Getting over High-Dimensionality: How Multidimensional Projection Methods Can Assist Data Science
Source abstract
Abstract
The exploration and analysis of multidimensional data can be pretty complex tasks, requiring sophisticated tools able to transform large amounts of data bearing multiple parameters into helpful information. Multidimensional projection techniques figure as powerful tools for transforming multidimensional data into visual information according to similarity features. Integrating this class of methods into a framework devoted to data sciences can contribute to generating more expressive means of visual analytics. Although the Principal Component Analysis (PCA) is a well-known method in this context, it is not the only one, and, sometimes, its abilities and limitations are not adequately discussed or taken into consideration by users. Therefore, knowing in-depth multidimensional projection techniques, their strengths, and the possible distortions they can create is of significant importance for researchers developing knowledge-discovery systems. This research presents a comprehensive overview of current state-of-the-art multidimensional projection techniques and shows example codes in Python and R languages, all available on the internet. The survey segment discusses the different types of techniques applied to multidimensional projection tasks from their background, application processes, capabilities, and limitations, opening the internal processes of the methods and demystifying their concepts. We also illustrate two problems, from a genetic experiment (supervised) and text mining (non-supervised), presenting solutions through multidimensional projection application. Finally, we brought elements that reverberate the competitiveness of multidimensional projection techniques towards high-dimension data visualization, commonly needed in data sciences solutions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Data Visualization and Analytics · Time Series Analysis and Forecasting · Sensory Analysis and Statistical Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Mathematics and Computer Science, University of São Paulo, São Carlos 13566590, Brazil
Departamento de Matemática, Facultad de Ingeniería, Universidad de Atacama, Copiapó 1530000, Chile
Funding
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)
References (100+)
Show source-provided references
1. Graphical Perception of Multiple Time Series10.1109/TVCG.2010.162 ↗
2. Shneiderman, B. (1996, January 3–6). The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. Proceedings of the 1996 IEEE Symposium on Visual Languages, VL’96, Boulder, CO, USA.
3. Heer, J., Kong, N., and Agrawala, M. (2009, January 4–9). Sizing the Horizon: The Effects of Chart Size and Layering on the Graphical Perception of Time Series Visualizations. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI’09, Boston, MA, USA.10.1145/1518701.1518897 ↗
4. Chi, E.H. (2002). A Framework for Visualization Information, Springer.10.1007/978-94-017-0573-8 ↗
5. Telea, A.C. (2014). Data Visualization: Principles and Practice, CRC Press. [2nd ed.].10.1201/b17217 ↗
6. LoCH: A neighborhood-based multidimensional projection technique for high-dimensional sparse spaces10.1016/j.neucom.2014.07.071 ↗
7. Multidimensional projection for visual analytics: Linking techniques with distortions, tasks, and layout enrichment10.1109/TVCG.2018.2846735 ↗
8. Cantareira, G.D., Etemad, E., and Paulovich, F.V. (2020). Exploring Neural Network Hidden Layer Activity Using Vector Fields. Information, 11.10.3390/info11090426 ↗
9. Least Square Projection: A Fast High-Precision Multidimensional Projection Technique and Its Application to Document Mapping10.1109/TVCG.2007.70443 ↗
10. HiPP: A Novel Hierarchical Point Placement Strategy and Its Application to the Exploration of Document Collections10.1109/TVCG.2008.138 ↗
11. Information visualization techniques for sensing and biosensing
12. Local Affine Multidimensional Projection10.1109/TVCG.2011.220 ↗
13. Berkhin, P. (2006). A Survey of Clustering Data Mining Techniques, Springer.
14. Lee, J., and Verleysen, M. (2007). Nonlinear Dimensionality Reduction, Springer Science & Business Media.10.1007/978-0-387-39351-3 ↗
15. Data Visualization With Multidimensional Scaling10.1198/106186008X318440 ↗
16. Dimensionality Reduction: A Comparative Review
17. A Survey of CUDA-based Multidimensional Scaling on GPU Architecture
18. Visual Interaction with Dimensionality Reduction: A Structured Literature Analysis10.1109/TVCG.2016.2598495 ↗
19. Konyha, Z., Lez, A., Matković, K., Jelović, M., and Hauser, H. (2012, January 5–7). Interactive Visual Analysis of Families of Curves Using Data Aggregation and Derivation. Proceedings of the 12th International Conference on Knowledge Management and Knowledge Technologies, i-KNOW’12, Graz, Austria.10.1145/2362456.2362487 ↗
20. Fua, Y.H., Ward, M.O., and Rundensteiner, E.A. (1999, January 24–29). Hierarchical Parallel Coordinates for Exploration of Large Datasets. Proceedings of the Conference on Visualization’99: Celebrating Ten Years, VIS’99, Los Alamitos, CA, USA.
21. Ware, C. (2004). Information Visualization: Perception for Design, Morgan Kaufmann Publishers Inc.. [2nd ed.].
22. Novel Methods for Subset Selection with Respect to Problem Knowledge10.1109/5254.671094 ↗
23. A survey on feature selection methods10.1016/j.compeleceng.2013.11.024 ↗
24. An introduction to variable and feature selection
25. Kirby, M. (2000). Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns, John Wiley & Sons, Inc.. [1st ed.].
26. On Improved Projection Techniques to Support Visual Exploration of Multidimensional Data Sets10.1057/palgrave.ivs.9500054 ↗
27. Cox, T.F., and Cox, M.A.A. (2000). Multidimensional Scaling, Chapman and Hall–CRC. [2nd ed.].10.1201/9781420036121 ↗
28. Visualizing Data using t-SNE
29. McInnes, L., Healy, J., and Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv.10.21105/joss.00861 ↗
30. Zezula, P., Amato, G., Dohnal, V., and Batko, M. (2006). Similarity Search: The Metric Space Approach, Advances in Database Systems, Book 32; Springer Publishing Company, Inc.. [1st ed.].10.1007/0-387-29151-2 ↗
31. Paulovich, F.V. (2008). Mapeamento de Dados Multi-Dimensionais-Integrando Mineração e Visualização. [Ph.D. Thesis, Universidade de São Paulo].
32. Ward, M.O., Grinstein, G., and Keim, D. (2015). Interactive Data Visualizaton: Foundations, Techniques, and Applications, CRC Press. [2nd ed.].10.1201/b18379 ↗
33. On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration10.1023/A:1024988512476 ↗
34. Incremental Nonlinear Dimensionality Reduction by Manifold Learning10.1109/TPAMI.2006.56 ↗
35. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis10.1007/BF02289565 ↗
36. Heulot, N., Fekete, J.D., and Aupetit, M. (2017). Visualizing dimensionality reduction artifacts: An evaluation. arXiv.
37. Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis10.1016/0377-0427(87)90125-7 ↗
38. Coimbra, D.B. (2016). Multidimensional Projections for the Visual Exploration of Multimedia Data. [Ph.D. Thesis, Universidade de São Paulo].
39. Marcilio, W.E., Eler, D.M., and Garcia, R.E. (2017, January 17–18). An approach to perform local analysis on multidimensional projection. Proceedings of the 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Niteroi, Brazil.10.1109/SIBGRAPI.2017.53 ↗
40. Quality Metrics in High-Dimensional Data Visualization: An Overview and Systematization10.1109/TVCG.2011.229 ↗
41. Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods10.1080/01621459.1984.10478080 ↗
42. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization10.1037/a0029333 ↗
43. Brushing Scatterplots10.1080/00401706.1987.10488204 ↗
44. Analysis and visualization of multidimensional time series: Particulate matter (PM10) from São Carlos-SP (Brazil)10.1016/j.apr.2019.03.001 ↗
45. McLachlan, P., Munzner, T., Koutsofios, E., and North, S. (2008, January 5–10). LiveRAC: Interactive Visual Exploration of System Management Time-series Data. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’08, Florence, Italy.10.1145/1357054.1357286 ↗
46. M4: A Visualization-oriented Time Series Data Aggregation
47. Jolliffe, I. (2002). Principal Component Analysis, Springer. [2nd ed.].
48. Trend Modeling for Traffic Time Series Analysis: An Integrated Study10.1109/TITS.2015.2457240 ↗
49. Brunton, S.L., and Kutz, J.N. (2022). Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge University Press. [2nd ed.].10.1017/9781009089517 ↗
50. Multidimensional scaling: I. Theory and method10.1007/BF02288916 ↗
51. Silva, V.D., and Tenenbaum, J.B. (2004). Sparse Multidimensional Scaling Using Landmark Points, Stanford University. Technical Report.
52. Brandes, U., and Pich, C. (2007, January 18–20). Eigensolver Methods for Progressive Multidimensional Scaling of Large Data. Proceedings of the 14th International Conference on Graph Drawing, GD’06, Tübingen, Germany.
53. A Nonlinear Mapping for Data Structure Analysis10.1109/T-C.1969.222678 ↗
54. Pekalska, E., de Ridder, D., Duin, R.P., and Kraaijveld, M.A. (1999, January 15–17). A new method of generalizing Sammon mapping with application to algorithm speed-up. Proceedings of the 5th Annual Conference of the Advanced School for Computing and Imaging, ASCI’99, Heijen, The Netherlands.
55. A Global Geometric Framework for Nonlinear Dimensionality Reduction10.1126/science.290.5500.2319 ↗
56. TopoMap: A 0-dimensional homology preserving projection of high-dimensional data10.1109/TVCG.2020.3030441 ↗
57. A note on two problems in connexion with graphs10.1007/BF01386390 ↗
58. The Isomap Algorithm and Topological Stability10.1126/science.295.5552.7a ↗
59. Silva, V.D., and Tenenbaum, J.B. (2002, January 9–14). Global Versus Local Methods in Nonlinear Dimensionality Reduction. Proceedings of the 15th International Conference on Neural Information Processing Systems, NIPS’02, Vancouver, BC, Canada.
60. FastMap: A Fast Algorithm for Indexing, Data-mining and Visualization of Traditional and Multimedia Datasets10.1145/568271.223812 ↗
61. Chalmers, M. (November, January 27). A Linear Iteration Time Layout Algorithm for Visualising High-dimensional Data. Proceedings of the 7th Conference on Visualization’96, VIS ’96, San Francisco, CA, USA.
62. Morrison, A., Ross, G., and Chalmers, M. (November, January 27). A Hybrid Layout Algorithm for Sub-Quadratic Multidimensional Scaling. Proceedings of the IEEE Symposium on Information Visualization, INFOVIS’02, Boston, MA, USA.
63. Brodbeck, D., and Girardin, L. (1998, January 19–20). Combining topological clustering and multidimensional scaling for visualising large data sets. Proceedings of the IEEE Information Visualization 1998, Research Triangle Park, NC, USA.
64. Glimmer: Multilevel MDS on the GPU10.1109/TVCG.2008.85 ↗
65. Hinton, G., and Roweis, S. (2002, January 9–14). Stochastic Neighbor Embedding. Proceedings of the 15th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada. NIPS’02.
66. On Information and Sufficiency10.1214/aoms/1177729694 ↗
67. Accelerating t-SNE Using Tree-based Algorithms
68. Nocedal, J., and Wright, S. (2006). Numerical Optimization, Springer Science & Business Media.
69. Shift-invariant similarities circumvent distance concentration in stochastic neighbor embedding and variants10.1016/j.procs.2011.04.056 ↗
70. How to use t-SNE effectively10.23915/distill.00002 ↗
71. Paulovich, F.V., Eler, D.M., Poco, J., Botha, C.P., Minghim, R., and Nonato, L.G. (2011, January 1–3). Piecewise Laplacian-based Projection for Interactive Data Exploration and Organization. Proceedings of the 13th Eurographics/IEEE—VGTC Conference on Visualization, EuroVis’11, Bergen, Norway.10.1111/j.1467-8659.2011.01958.x ↗
72. Two-Phase Mapping for Projecting Massive Data Sets10.1109/TVCG.2010.207 ↗
73. Gower, J.C., and Dijksterhuis, G.B. (2004). Procrustes Problems, Oxford University Press.10.1093/acprof:oso/9780198510581.001.0001 ↗
74. Steinbach, M., Karypis, G., and Kumar, V. (2000, January 20). A comparison of document clustering techniques. Proceedings of the KDD Workshop on Text Mining, Boston, MA, USA.
75. Dias, F., and Minghim, R. (November, January 29). xHiPP: eXtended Hierarchical Point Placement Strategy. Proceedings of the 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Foz do Iguaçu, Brazil.10.1109/SIBGRAPI.2018.00053 ↗
76. The choice of a class interval10.1080/01621459.1926.10502161 ↗
77. Graph regularization multidimensional projection10.1016/j.patcog.2022.108690 ↗
78. Goerss, P.G., and Jardine, J.F. (2009). Simplicial Homotopy Theory, Springer Science & Business Media.10.1007/978-3-0346-0189-4 ↗
79. The norm of the Schur product operation10.1007/BF01386329 ↗
80. Nene, S.A., Nayar, S.K., and Murase, H. (1996). Columbia Object Image Library (COIL-20), Columbia University.
81. LeCun, Y., Cortes, C., and Burges, C.J. (2022, April 10). The MNIST Database of Handwritten Digits. Available online: http://yann.lecun.com/exdb/mnist.
82. Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. arXiv.
83. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5–8). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
84. Wagner, A., Solomon, E., and Bendich, P. (2021). Improving Metric Dimensionality Reduction with Distributed Topology. arXiv.
85. Nelson, B.J., and Luo, Y. (2022). Topology-Preserving Dimensionality Reduction via Interleaving Optimization. arXiv.
86. Ripser: Efficient computation of Vietoris–Rips persistence barcodes10.1007/s41468-021-00071-5 ↗
87. Attribute-based Explanation of Non-Linear Embeddings of High-Dimensional Data10.1109/TVCG.2021.3114870 ↗
88. Graph Signal Processing: Overview, Challenges, and Applications10.1109/JPROC.2018.2820126 ↗
89. A tutorial on spectral clustering10.1007/s11222-007-9033-z ↗
90. Lundberg, S.M., Erion, G.G., and Lee, S.I. (2018). Consistent individualized feature attribution for tree ensembles. arXiv.
91. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
92. Shapley, L.S. (1953). A value for n-person games Contributions to the Theory of Games (AM-28), Princeton University Press.10.1515/9781400881970-018 ↗
93. Molnar, C. (2019). Interpretable Machine Learning, Lulu Press, Inc.
94. Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
95. Hong, C.W., Lee, C., Lee, K., Ko, M.S., and Hur, K. (2020, January 21–23). Explainable Artificial Intelligence for the Remaining Useful Life Prognosis of the Turbofan Engines. Proceedings of the 2020 3rd IEEE International Conference on Knowledge Innovation and Invention (ICKII), Taiwan, China.10.1109/ICKII50300.2020.9318912 ↗
96. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery10.1038/s41551-018-0304-0 ↗
97. Vilarino, R., and Vicente, R. (2020). An Experiment on Leveraging SHAP Values to Investigate Racial Bias. arXiv.
98. Explaining dimensionality reduction results using Shapley values10.1016/j.eswa.2021.115020 ↗
99. Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., Srivastava, M., Preece, A., Julier, S., and Rao, R.M. (2017, January 4–8). Interpretability of deep learning models: A survey of results. Proceedings of the 2017 IEEE Smartworld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (Smartworld/SCALCOM/UIC/ATC/CBDcom/IOP/SCI), San Francisco, CA, USA.10.1109/UIC-ATC.2017.8397411 ↗
100. Zhang, Q., and Zhu, S.C. (2018). Visual interpretability for deep learning: A survey. arXiv.10.1631/FITEE.1700808 ↗