Datamator: An Authoring Tool for Creating Datamations via Data Query Decomposition
Source abstract
Abstract
Datamation is designed to animate an analysis pipeline step by step, serving as an intuitive and efficient method for interpreting data analysis outcomes and facilitating easy sharing with others. However, the creation of a datamation is a difficult task that demands expertise in diverse skills. To simplify this task, we introduce Datamator, a language-oriented authoring tool developed to support datamation generation. In this system, we develop a data query analyzer that enables users to generate an initial datamation effortlessly by inputting a data question in natural language. Then, the datamation is displayed in an interactive editor that affords users the ability to both edit the analysis progression and delve into the specifics of each step undertaken. Notably, the Datamator incorporates a novel calibration network that is able to optimize the outputs of the query decomposition network using a small amount of user feedback. To demonstrate the effectiveness of Datamator, we conduct a series of evaluations including performance validation, a controlled user study, and expert interviews.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Data Visualization and Analytics · Semantic Web and Ontologies · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Design and Innovation, Tongji University, Shanghai 200092, China
Visualization Research Center (VISUS), University of Stuttgart, 70569 Stuttgart, Germany
Department of Information and Communications Engineering, Aalto University, 02150 Espoo, Finland
Funding
NSFC · NSF Shanghai · Deutsche Forschungsgemeinschaft
References (62)
Show source-provided references
1. Pu, X., Kross, S., Hofman, J.M., and Goldstein, D.G. (2021, January 8–13). Datamations: Animated Explanations of Data Analysis Pipelines. Proceedings of the CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.10.1145/3411764.3445063 ↗
2. Canis: A High-Level Language for Data-Driven Chart Animations10.1111/cgf.14005 ↗
3. Kim, Y., and Heer, J. (2021, January 24–29). Gemini 2: Generating Keyframe-Oriented Animated Transitions between Statistical Graphics. Proceedings of the IEEE Visualization Conference, New Orleans, LA, USA.10.1109/VIS49827.2021.9623291 ↗
4. AutoClips: An Automatic Approach to Video Generation from Data Facts10.1111/cgf.14324 ↗
5. D3 Data-Driven Documents10.1109/TVCG.2011.185 ↗
6. Amini, F., Riche, N.H., Lee, B., Leboe-McGowan, J., and Irani, P. (2018, January 1). Hooked on data videos: Assessing the effect of animation and pictographs on viewer engagement. Proceedings of the AVI, Castiglione della Pescaia, Italy.10.1145/3206505.3206552 ↗
7. Ge, T., Lee, B., and Wang, Y. (2021, January 8–13). CAST: Authoring Data-Driven Chart Animations. Proceedings of the CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.10.1145/3411764.3445452 ↗
8. Thompson, J.R., Liu, Z., and Stasko, J. (2021, January 8–13). Data Animator: Authoring Expressive Animated Data Graphics. Proceedings of the CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.10.1145/3411764.3445747 ↗
9. Cao, Y., E, J.L., Chen, Z., and Xia, H. (2023, January 23–28). DataParticles: Block-based and Language-oriented Authoring of Animated Unit Visualizations. Proceedings of the CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544548.3581472 ↗
10. Effectiveness of Animation in Trend Visualization10.1109/TVCG.2008.125 ↗
11. Animated Transitions in Statistical Data Graphics10.1109/TVCG.2007.70539 ↗
12. Animation for Visualization: Opportunities and Drawbacks
13. Principles of traditional animation applied to 3D computer animation10.1145/37401.37407 ↗
14. Bakhshi, S., Shamma, D.A., Kennedy, L., Song, Y., De Juan, P., and Kaye, J. (2016, January 7–12). Fast, cheap, and good: Why animated GIFs engage us. Proceedings of the CHI Conference on Human FACTORS in Computing Systems, San Jose, CA, USA.10.1145/2858036.2858532 ↗
15. Feng, C., Bartram, L., and Riecke, B.E. (2014, January 8–9). Evaluating affective features of 3D motionscapes. Proceedings of the ACM Symposium on Applied Perception (SAP), Vancouver, BC, Canada.10.1145/2628257.2628264 ↗
16. Affective motion textures10.1016/j.cag.2012.04.009 ↗
17. Shi, Y., Yan, X., Ma, X., Lou, Y., and Cao, N. (2018, January 21–26). Designing emotional expressions of conversational states for voice assistants: Modality and engagement. Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3170427.3188560 ↗
18. Yang, Z., Zhang, Y., and Luo, J. (2019, January 8–12). Human-centered emotion recognition in animated gifs. Proceedings of the 2019 IEEE International Conference on Multimedia and Expo, Shanghai, China.10.1109/ICME.2019.00191 ↗
19. Motion to Support Rapid Interactive Queries on Node–Link Diagrams10.1145/1008722.1008724 ↗
20. Hullman, J., Resnick, P., and Adar, E. (2015). Hypothetical outcome plots outperform error bars and violin plots for inferences about reliability of variable ordering. PLoS ONE, 10.10.1371/journal.pone.0142444 ↗
21. Drucker, S., and Fernandez, R. (2015). A Unifying Framework for Animated and Interactive Unit Visualizations, Microsoft Research.
22. Schroeder, W., Martin, K., and Lorensen, B. (2006). The Visualization Toolkit, Kitware. [4th ed.].10.1016/B978-012387582-2/50032-0 ↗
23. Gemini: A Grammar and Recommender System for Animated Transitions in Statistical Graphics10.1109/TVCG.2020.3030360 ↗
24. Li, W., Wang, Y., Huang, H., Cui, W., Zhang, H., Qu, H., and Zhang, D. (2021). AniVis: Generating Animated Transitions between Statistical Charts with a Tree Model. arXiv.
25. Authoring Data-Driven Videos with Dataclips10.1109/TVCG.2016.2598647 ↗
26. Atom: A grammar for unit visualizations10.1109/TVCG.2017.2785807 ↗
27. Lu, J., Chen, W., Ye, H., Wang, J., Mei, H., Gu, Y., Wu, Y., Zhang, X.L., and Ma, K.L. (2021, January 19–21). Automatic Generation of Unit Visualization-based Scrollytelling for Impromptu Data Facts Delivery. Proceedings of the IEEE PacificVis, Tianjin, China.10.1109/PacificVis52677.2021.00011 ↗
28. Liu, C., Han, Y., Jiang, R., and Yuan, X. (2021, January 19–21). Advisor: Automatic Visualization Answer for Natural-Language Question on Tabular Data. Proceedings of the IEEE Pacific Visualization Symposium, Tianjin, China.10.1109/PacificVis52677.2021.00010 ↗
29. Towards Natural Language-Based Visualization Authoring
30. Gao, T., Dontcheva, M., Adar, E., Liu, Z., and Karahalios, K.G. (2015, January 11–15). DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization. Proceedings of the User Interface Software and Technology, Charlotte, NC, USA.10.1145/2807442.2807478 ↗
31. Setlur, V., Battersby, S.E., Tory, M., Gossweiler, R., and Chang, A.X. (2016, January 16–19). Eviza: A Natural Language Interface for Visual Analysis. Proceedings of the User Interface Software and Technology, Tokyo, Japan.10.1145/2984511.2984588 ↗
32. Aurisano, J., Kumar, A., Gonzalez, A., Leigh, J., DiEugenio, B., and Johnson, A. (2016, January 23–28). Articulate2: Toward a Conversational Interface for Visual Data Exploration. Proceedings of the IEEE Visualization, Baltimore, MD, USA.
33. NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries10.1109/TVCG.2020.3030378 ↗
34. Applying Pragmatics Principles for Interaction with Visual Analytics10.1109/TVCG.2017.2744684 ↗
35. A Multi-Modal Natural Language Interface to an Information Visualization Environment10.1023/A:1011368926479 ↗
36. Wen, Z., Zhou, M.X., and Aggarwal, V. (2005, January 23–25). An Optimization-based Approach to Dynamic Visual Context Management. Proceedings of the IEEE InfoVis, Minneapolis, MN, USA.
37. Interweaving multimodal interaction with flexible unit visualizations for data exploration10.1109/TVCG.2020.2978050 ↗
38. (2022, January 11). Microsoft Power BI Q&A. Available online: https://powerbi.microsoft.com.
39. (2021, December 11). Tableau Ask Data. Available online: https://www.tableau.com/products/new-features/ask-data.
40. Flowsense: A Natural Language Interface for Visual Data Exploration within a Dataflow System10.1109/TVCG.2019.2934668 ↗
41. Sun, Y., Leigh, J., Johnson, A., and Lee, S. (2010). Smart Graphics, Proceedings of the 10th International Symposium on Smart Graphics, Banff, AB, Canada, 24–26 June 2010, Springer.
42. Natural Language to Visualization by Neural Machine Translation10.1109/TVCG.2021.3114848 ↗
43. Guo, Y., Cao, N., Qi, X., Li, H., Shi, D., Zhang, J., Chen, Q., and Weiskopf, D. (2023). Urania: Visualizing Data Analysis Pipelines for Natural Language-Based Data Exploration. arXiv.
44. Towards natural language interfaces for data visualization: A survey10.1109/TVCG.2022.3148007 ↗
45. IDvXLab (2023, January 12). Narrative Chart. Available online: https://github.com/narchart/narrative-chart.
46. Break it down: A question understanding benchmark10.1162/tacl_a_00309 ↗
47. Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., and Roman, S. (November, January 31). Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. Proceedings of the Empirical Methods in Natural Language Processing, Brussels, Belgium.
48. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020, January 16–20). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the Empirical Methods in Natural Language Processing, Online.10.18653/v1/2020.emnlp-demos.6 ↗
49. Attention is All You Need
50. Vaswani, A., Bengio, S., Brevdo, E., Chollet, F., Gomez, A.N., Gouws, S., Jones, L., Kaiser, Ł., Kalchbrenner, N., and Parmar, N. (2018). Tensor2tensor for Neural Machine Translation. arXiv.
51. Wang, B., Shin, R., Liu, X., Polozov, O., and Richardson, M. (2020, January 6–8). RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.677 ↗
52. Liu, Y., and Lapata, M. (2019). Text Summarization with Pretrained Encoders. arXiv.10.18653/v1/D19-1387 ↗
53. Saparina, I., and Osokin, A. (2021, January 7–11). SPARQLing Database Queries from Intermediate Question Decompositions. Proceedings of the Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.708 ↗
54. Pytorch: An imperative style, high-performance deep learning library
55. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
56. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the CVPR, Las Vegas, NV, USA.
57. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http://www.deeplearningbook.org.
58. Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C.D. (2021). Fast model editing at scale. arXiv.
59. Optimizing Statistical Machine Translation for Text Simplification10.1162/tacl_a_00107 ↗
60. Gu, J., Lu, Z., Li, H., and Li, V.O. (2016, January 7–12). Incorporating Copying Mechanism in Sequence-to-Sequence Learning. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1154 ↗
61. Astf: Visual abstractions of time-varying patterns in radio signals
62. Zhao, Y., Lv, S., Long, W., Fan, Y., Yuan, J., Jiang, H., and Zhao, F. (2023). Malicious webshell family dataset for webshell multi-classification research. Visual Inform.10.1016/j.visinf.2023.06.008 ↗