journal article·2021Open access

Cluster Analysis for IR and NIR Spectroscopy: Current Practices to Future Perspectives

Simon Crase, Benjamin Hall, Suresh N. Thennadil

Computers, Materials & Continua· Volume 69· Issue 2

Abstract

Supervised machine learning techniques have become well established in the study of spectroscopy data. However, the unsupervised learning technique of cluster analysis hasn’t reached the same level maturity in chemometric analysis. This paper surveys recent studies which apply cluster analysis to NIR and IR spectroscopy data. In addition, we summarize the current practices in cluster analysis of spectroscopy and contrast these with cluster analysis literature from the machine learning and pattern recognition domain. This includes practices in data pre-processing, feature extraction, clustering distance metrics, clustering algorithms and validation techniques. Special consideration is given to the specific characteristics of IR and NIR spectroscopy data which typically includes high dimensionality and relatively low sample size. The findings highlighted a lack of quantitative analysis and evaluation in current practices for cluster analysis of IR and NIR spectroscopy data. With this in mind, we propose an analysis model or workflow with techniques specifically suited for cluster analysis of IR and NIR spectroscopy data along with a pragmatic application strategy.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Spectroscopy and Chemometric Analyses · Spectroscopy Techniques in Biomedical and Chemical Research · Advanced Chemical Sensor Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (91)

Show source-provided references
1. Machine learning for molecular and materials science10.1038/s41586-018-0337-2 ↗
2. Duda · Pattern Classification · 2012
3. What are the true clusters?10.1016/j.patrec.2015.04.009 ↗
4. Jain · Algorithms for Clustering Data · 1988
5. Statistical pattern recognition: A review10.1109/34.824819 ↗
6. To cluster, or not to cluster: An analysis of clusterability methods10.1016/j.patcog.2018.10.026 ↗
7. Rapid identification of closely related muscle foods by vibrational spectroscopy and machine learning10.1039/b511484e ↗
8. Pollen discrimination and classification by Fourier transform infrared (FT-IR) microspectroscopy and machine learning10.1007/s00216-009-2794-9 ↗
9. Using FTIR spectra and pattern recognition for discrimination of tea varieties10.1016/j.ijbiomac.2015.03.025 ↗
10. Spectral pre and post processing for infrared and Raman spectroscopy of biological tissues and cells10.1039/C5CS00440C ↗
11. A comparison of Raman, FTIR and ATR-FTIR micro spectroscopy for imaging human skin tissue sections10.1039/c3ay40185e ↗
12. Derivative FTIR spectroscopy for cluster analysis and classification of morocco olive oils10.1016/j.foodchem.2010.07.010 ↗
13. Identification of lactic acid bacteria using FTIR spectroscopy and cluster analysis10.1016/j.idairyj.2006.02.013 ↗
14. Discrimination of tea varieties using FTIR spectroscopy and allied Gustafson–Kessel clustering10.1016/j.compag.2018.02.014 ↗
15. Fourier transform infrared imaging and unsupervised hierarchical clustering applied to cervical biopsies10.1071/CH04137 ↗
16. Differentiation of anatolian honey samples from different botanical origins by ATR-FTIR spectroscopy using multivariate analysis10.1016/j.foodchem.2014.08.040 ↗
17. Analysis and evaluation of essential oil components of cinnamon barks using GC-MS and FTIR spectroscopy10.1016/j.indcrop.2012.04.056 ↗
18. FTIR spectroscopy for detection and identification of fungal phytopathogenes10.1155/2010/723489 ↗
19. Simulated annealing fuzzy clustering in cancer diagnosis
20. A rapid ATR-FTIR spectroscopic method for detection of sibutramine adulteration in tea and coffee based on hierarchical cluster and principal component analyses10.1016/j.foodchem.2017.02.072 ↗
21. Efficient discrimination of oat and pea roots by cluster analysis of Fourier transform infrared (FTIR) spectra10.1016/j.fcr.2010.06.017 ↗
22. Characterization of food spoilage fungi by FTIR spectroscopy10.1111/jam.12092 ↗
23. Combined principal component analysis (PCA) and hierarchical cluster analysis (HCA): An efficient chemometric approach in aged gel inks discrimination10.1080/00450618.2018.1466913 ↗
24. Effects of selenium supplementation on rat heart apex and right ventricle myocardia by using FTIR spectroscopy: A cluster analysis and neural network approach10.1016/j.foodchem.2008.02.044 ↗
25. Determination of the dose-dependent toxic effects of mad honey on mouse liver using ATR-FTIR spectroscopy10.1016/j.saa.2019.117719 ↗
26. A rapid ATR-FTIR spectroscopic method for classification of gelatin gummy candies in relation to the gelatin source10.1016/j.foodchem.2018.10.125 ↗
27. Rapid detection of adulteration of cold pressed sesame oil adultered with hazelnut, canola, and sunflower oils using ATR-FTIR spectroscopy combined with chemometric10.1016/j.foodcont.2017.06.034 ↗
28. Authentication and discrimination of green tea samples using UV-vis, FTIR and HPLC techniques coupled with chemometrics analysis10.1016/j.jpba.2018.11.036 ↗
29. Classification of ANFO samples based on their fuel composition by GC-MS and FTIR combined with chemometrics10.1016/j.forsciint.2019.06.001 ↗
30. Imaging of colorectal adenocarcinoma using FT-IR microspectroscopy and cluster analysis10.1016/j.bbadis.2003.12.006 ↗
31. Selecting relevant Fourier transform infrared spectroscopy wavenumbers for clustering authentic and counterfeit drug samples10.1016/j.scijus.2014.04.005 ↗
32. Study of pharmaceutical samples by NIR chemical-image and multivariate analysis10.1016/j.trac.2008.05.010 ↗
33. Relationship between near-infrared (NIR) spectra and the geographical provenance of timber10.1007/s00226-010-0313-y ↗
34. Identification of magnesium oxychloride cement biomaterial heterogeneity using raman chemical mapping and NIR hyperspectral chemical imaging10.1038/s41598-018-31379-5 ↗
35. Classification of apple varieties using near infrared reflectance spectroscopy and fuzzy discriminant C-Means clustering model10.1111/jfpe.12355 ↗
36. Rapid evaluation and quantitative analysis of thyme, origano and chamomile essential oils by ATR-IR and NIR spectroscopy10.1016/S0022-2860(03)00517-9 ↗
37. FT-IR spectroscopy for identification of closely related lactobacilli10.1016/j.mimet.2004.06.011 ↗
38. Development and validation of a near-infrared method for the quantitation of caffeine in intact single tablets10.1021/ac026262w ↗
39. Chemometric analysis of combined NIR and MIR spectra to characterize French olives10.1002/ejlt.200900198 ↗
40. Do NIR spectra collected from laboratory-reared mosquitoes differ from those collected from wild mosquitoes?10.1371/journal.pone.0198245 ↗
41. Classification of edible oils by infrared spectroscopy with optimized k-means clustering by a hybrid particle swarm algorithm10.1080/00032719.2013.807816 ↗
42. Identification of fungal phytopathogens using Fourier transform infrared-attenuated total reflection spectroscopy and advanced statistical methods10.1117/1.JBO.17.1.017002 ↗
43. Gasoline quality prediction using gas chromatography and FTIR spectroscopy: An artificial intelligence approach10.1016/j.fuel.2005.07.019 ↗
44. Determination of total antioxidant capacity in green tea by near-infrared spectroscopy and multivariate calibration10.1016/S0039-9140(03)00397-7 ↗
45. Identification of flax oil by linear multivariate spectral analysis10.1007/s10812-020-00929-z ↗
46. Detecting and quantifying sunflower oil adulteration in extra virgin olive oils from the Eastern Mediterranean by visible and near-infrared spectroscopy10.1021/jf0257188 ↗
47. Combination of mid-and near-infrared spectroscopy for the determination of the quality properties of beers10.1016/j.aca.2006.04.070 ↗
48. Characteristics and classification of Fourier-transform near infrared spectra of the heartwood of different larch species (Larix sp.)10.1255/jnirs.415 ↗
49. Coal analysis by diffuse reflectance near-infrared spectroscopy: Hierarchical cluster and linear discriminant analysis10.1016/j.talanta.2007.01.050 ↗
50. Near infrared spectroscopy: An analytical tool to predict coffee roasting degree10.1016/j.aca.2008.07.013 ↗
51. A comparison of fuzzy and non-fuzzy clustering techniques in cancer diagnosis
52. Challenges in computational histopathology: The feasibility of FTIR spectroscopy in clustering
53. The complexities involved in the analysis of Fourier transform infrared spectroscopy of breast cancer data with clustering algorithms
54. Discrimination of amanita mushrooms using fourier transform infrared difference spectroscopy and cluster analysis
55. Classification of endospores of Bacillus and Clostridium species by FT-IR reflectance microspectroscopy and autoclaving
56. Application of the fuzzy C-means clustering method on the analysis of non pre-processed FTIR data for cancer diagnosis
57. Breaking with trends in pre-processing?10.1016/j.trac.2013.04.015 ↗
58. A contemporary review on data preprocessing (DP) practice strategy in ATR-FTIR spectrum10.1016/j.chemolab.2017.02.008 ↗
59. Feature selection for unsupervised learning
60. Towards improving cluster-based feature selection with a simplified silhouette filter10.1016/j.ins.2011.04.050 ↗
61. Boutsidis · Advances in Neural Information Processing Systems · 2009 · 153
62. Dash · Knowledge Discovery and Data Mining. Current Issues and New Applications · 2000 · 11010.1007/3-540-45571-X_13 ↗
63. Unsupervised feature selection for multi-cluster data
64. Unsupervised feature selection for linked social media data
65. Feature selection for clustering: A review
66. A review of unsupervised feature selection methods10.1007/s10462-019-09682-y ↗
67. Principal component analysis for clustering gene expression data10.1093/bioinformatics/17.9.763 ↗
68. Does principal component analysis improve cluster-based analysis?
69. Visualizing data using t-SNE
70. Visualization of SNPs with t-SNE10.1371/journal.pone.0056883 ↗
71. Rapid determination of aflatoxin B1 concentration in soybean oil using terahertz spectroscopy with chemometric methods10.1016/j.foodchem.2019.04.081 ↗
72. Dimensionality reduction and visualisation of hyperspectral ink data using t-SNE10.1016/j.forsciint.2020.110194 ↗
73. A New method for determining the type of distribution of plant individuals10.1093/oxfordjournals.aob.a083391 ↗
74. A density-based algorithm for discovering clusters in large spatial databases with noise
75. The dip test of unimodality10.1214/aos/1176346577 ↗
76. Using kernel density estimates to investigate multimodality10.1111/j.2517-6161.1981.tb01155.x ↗
77. Rokach · Data Mining and Knowledge Discovery Handbook · 2010 · Ch. 14
78. Aldenderfer · Cluster Analysis (Quantitative Applications in the Social Sciences) · 1984
79. Classification and identification of bacteria by Fourier-transform infrared spectroscopy10.1099/00221287-137-1-69 ↗
80. A Survey of Clustering With deep learning: From the perspective of network architecture10.1109/ACCESS.2018.2855437 ↗
81. Cluster ensembles—A knowledge reuse framework for combining multiple partitions
82. Estimating the number of clusters in a data set via the gap statistic10.1111/1467-9868.00293 ↗
83. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis10.1016/0377-0427(87)90125-7 ↗
84. Aggarwal · Data Clustering: Algorithms and Applications · 201310.1201/b15410 ↗
85. Desgraupes · Clustering Indices · 2017 · 1
86. V-measure: A conditional entropy-based external cluster evaluation measure
87. Comparing partitions10.1007/BF01908075 ↗
88. Objective criteria for the evaluation of clustering methods10.1080/01621459.1971.10482356 ↗
89. On the selection of appropriate distances for gene expression data clustering10.1186/1471-2105-15-S2-S2 ↗
90. Understanding of internal clustering validation measures
91. Pattern recognition in chemometrics10.1016/j.chemolab.2015.06.012 ↗