Empirical Studies of Information Visualization: A Meta-Analysis

Summary:

The study Empirical Studies of Information Visualization: A Meta-Analysis by Chaomei Chen and Yue Yu provides a meta-analysis on a variety of empirical studies of information visualization.  The intent of the research is to conduct a meta-analysis on this topic in order to capture the theories and practices in empirical examinations of information visualization. The analysis focuses on three areas of information visualization including users, tasks, and tools.  As a meta-analysis, this article provides a simplified description and displays the underlying relations in the large amount of convoluted, contradictory, and confusing information often found in the literature.  

The article first provides an overview of the meta-analytical method and selection for studies used, then a subjective review of the studies is presented follow by identifying the most commonly used hypothesis, independent variables, and dependent variables, this is followed by the results of the study.  The research includes experimental studies with independent variables that comprise a relationship to one of the three contextual variables (users, tasks, and tools).  The two types of dependent variables used are accuracy and efficiency measures. 

The study’s results come in two parts looking at both users and tools.  Each section compares the empirical findings of individual studies and are synthesized in terms of effect sizes and significance levels.  The study found that users with strong cognitive abilities will benefit significantly more from visual-spatial interfaces than those with weaker cognitive abilities.  The study found that users with stronger cognitive abilities will perform more efficiently than users with weaker cognitive abilities while using visualization. Additionally, the study displayed that visual-spatial information-retrieval interfaces will enable users to perform better than traditional retrieval interfaces.  Finally, the study stated that users using visualization interfaces in information retrieval will perform more efficiently than those using a none visualization interface.  The following list the major, all encompassing conclusions of the study.

                1. Empirical studies of information visualization are diverse and applying meta-analysis methods is difficult.

                2. Future studies would benefit from systematically investigating individual differences, including a  variety of cognitive abilities systematically.

                3. When users displayed the same level of cognitive abilities they tended to perform better with simpler visual-spatial interfaces.

                4. The combined effect size of visualization in not statistically significant. A larger homogeneous sample of studies is necessary for conclusive results.

Critique:

This meta-analysis is especially helpful due to the increasing amount of literate on the topic of visualization.  Although its finding were not significant it provided a very effective start in providing an overview of the current literature on visualization.  As technology progresses this study will be one to build upon.

Considering the study came out when visualization was in its infancy, it is understandable that a minimal amount of articles were available.  Although, if the authors would have slightly expanded their criteria the study would have provided a better overview of visualization overall.  Furthermore, only five studies tested the effects of visualization on accuracy.  To strengthen the argument the authors should have considered increasing the amount of articles used under this category.  Additionally only three studies tested the efficiency of visualization. The same critique applies here in that more sources would have greatly benefited the analysis as well as the argument overall.  Again, by expanding the criteria for an article to be used in the study as well as the categories tested in the study more articles may have been available.    

Overall, this study provided a very useful synthesis of visualization and its benefits.  Expanding upon the approach by conducting a similar study in terms of visualization today would provide an interesting comparison and an overview of its progression.   

Source:

Chen, C. & Yue, Y. Empirical studies of information visualization: a meta-analysis. Int. J. Human-Computer Studies. 53(5), 851-866. Retrieved from: http://www.sciencedirect.com/science/article/pii/S1071581900904221.