journal article·2021-12-16Open access

Sensory Panel Performance Evaluation—Comprehensive Review of Practical Approaches

László Sipos, Ákos Nyitrai, Géza Hitka, László Ferenc Friedrich, Zoltán Kókai

Applied Sciences· Volume 11· Issue 24

Abstract

Sensory assessors determine the result of sensory analysis; therefore, investigation of panel performance is inevitable to obtain well-established results. In the last few decades, numerous publications examine the performance of both panelists and panels. The initial point of any panelist measures are the applied selection methods, which are chosen according to the purpose (general suitability or product-specific skills). A practical overview is given on the available solutions, methods, protocols and software relating to all major panelist and panel measure indices (agreement, discrimination, repeatability, reproducibility and scale usage), with special focus on the utilized statistical methods. The novel approach of the presented methods is multi-faceted, concerning time factor (measuring performance at a given moment or over a period), the level of integration in the sensory testing procedure and the target of the measurements (panelist versus panel). The present paper supports the choice of the performance parameter and its related statistical procedure. Available software platforms, their accessibility (open-source status) and their functions are thoroughly analyzed concerning panelist or whole panel evaluation. The applied sensory test method strongly defines the applicable performance evaluation tools; therefore, these aspects are also discussed. A special field is related to proficiency testing. With the focus on special activities (product competitions, expert panels, food and horticultural goods), practical examples are given. In our research, special attention was given to sensory activity in companies and product experts or product-specific panels. Emerging future trends in this field will involve meta-analyses, application of AI and integration of psychophysics.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Sensory Analysis and Statistical Methods · Meat and Animal Product Quality · Biochemical Analysis and Sensing Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Postharvest, Supply Chain, Commerce and Sensory Science, Institute of Food Science and Technology, Hungarian University of Agriculture and Life Sciences, 39-43 Villányi Street, 1118 Budapest, Hungary

Department of Refrigeration and Livestock Product Technology, Institute of Food Science and Technology, Hungarian University of Agriculture and Life Sciences, 43-45 Ménesi Street, 1118 Budapest, Hungary

Funding

Hungarian Academy of Sciences · European Union and co-financed by the European Social Fund · National Research, Development and Innovation Office of Hungary · Ministry for Innovation and Technology

References (97)

Show source-provided references
1. Example Food: What Are Its Sensory Properties and Why Is That Important?10.1038/s41538-018-0019-3 ↗
2. Lawless, H.T., and Heymann, H. (2010). Sensory Evaluation of Food: Principles and Practices. Food Science Text Series, Springer. [2nd ed.].10.1007/978-1-4419-6488-5 ↗
3. ISO 8586 (2012). Sensory Analysis—General Guidelines for the Selection, Training and Monitoring of Selected Assessors and Expert Sensory Assessors, ISO.
4. Remote Testing: Sensory Test during COVID-19 Pandemic and Beyond10.1016/j.foodqual.2021.104437 ↗
5. Sensory Analysis of Cosmetic Powders: Personal Care Ingredients and Emulsions10.1111/ics.12352 ↗
6. Rogers, L. (2017). Sensory Panel Management: A Practical Handbook for Recruitment, Training and Performance, Woodhead Publishing.
7. ISO 5492 (2008). Sensory Analysis—Vocabulary, ISO.
8. Meilgaard, M.C., Carr, B.T., and Civille, G.V. (2006). Sensory Evaluation Techniques, CRC Press. [4th ed.].10.1201/b16452 ↗
9. Raithatha, C., and Rogers, L. (2018). Panel Quality Management: Performance, Monitoring and Proficiency. Descriptive Analysis in Sensory Evaluation, John Wiley & Sons.10.1002/9781118991657.ch4 ↗
10. Bourne, M. (2002). Food Texture and Viscosity, Academic Press. [2nd ed.].10.1016/B978-012119062-0/50001-2 ↗
11. ISO 11036 (2020). Sensory Analysis—Methodology—Texture Profile, ISO.
12. Sweet Taste in Apple: The Role of Sorbitol, Individual Sugars, Organic Acids and Volatile Compounds10.1038/srep44950 ↗
13. Næs, T., Brockhoff, P.B., and Tomic, O. (2010). Statistics for Sensory and Consumer Science, John Wiley & Sons, Ltd.. [1st ed.].10.1002/9780470669181 ↗
14. Fichet, B., Piccolo, D., Verde, R., and Vichi, M. (2011). Monitoring Panel Performance Within and between Sensory Experiments by Multi-Way Analysis. Classification and Multivariate Analysis for Complex Data Structures, Springer.10.1007/978-3-642-13312-1 ↗
15. Meiselman, H.L. (2019). 24—Contextual Product Testing for Small to Medium Sized Enterprises (SMEs). Context, Woodhead Publishing.
16. Review on Characteristics of Trained Sensory Panels in Food Science10.1111/jtxs.12616 ↗
17. Sensory optimisation in new food product development: A case study of polish apple juice
18. Cavicchi, A., and Santini, C. (2018). Chapter 4—Sensory and Consumer Approaches for Targeted Product Development in the Agro-Food Sector. Case Studies in the Traditional Food Sector, Woodhead Publishing.
19. Cost/Efficiency Evaluation of Descriptive Analysis Panels—I. Panel Size10.1016/0950-3293(95)00024-0 ↗
20. Performance Comparison Between Trained Assessors and Wine Experts Using Specific Sensory Attributes10.1111/j.1745-459X.2004.051404.x ↗
21. A Method for Panelists’ Consistency Assessment in Sensory Evaluations Based on Cronbach’s Alpha Coefficient10.1016/j.foodqual.2013.06.006 ↗
22. Monteleone, E., and Langstaff, S. (2013). Olive Oil Sensory Science, John Wiley & Sons.10.1002/9781118332511 ↗
23. A Comparative Study of the Main International Extra Virgin Olive Oil Competitions: Their Impact on Producers and Consumers10.1016/j.tifs.2020.11.014 ↗
24. Prediction of Olive Oil Sensory Descriptors Using Instrumental Data Fusion and Partial Least Squares (PLS) Regression10.1016/j.talanta.2016.04.040 ↗
25. How Do Consumer Hedonic Ratings for Extra Virgin Olive Oil Relate to Quality Ratings by Experts and Descriptive Analysis Ratings?10.1016/j.foodqual.2010.10.004 ↗
26. An Investigation of the Pivot© Profile Sensory Analysis Method Using Wine Experts: Comparison with Descriptive Analysis and Results from Two Expert Panels10.1016/j.foodqual.2019.103858 ↗
27. Sensory and Volatile Profiles of Monofloral Honeys Produced by Native Stingless Bees of the Brazilian Semiarid Region10.1016/j.foodres.2017.10.043 ↗
28. Sensory Analysis Applied to Honey: State of the Art10.1051/apido:2004048 ↗
29. Current Status and Perspectives of the Official Sensory Control Methods in Protected Designation of Origin Food Products and Wines10.1016/j.foodcont.2018.01.010 ↗
30. ISO 20613 (2019). Sensory Analysis—General Guidance for the Application of Sensory Analysis in Quality Control, ISO.
31. Partial Projective Mapping and Ultra-Flash Profile with and without Red Light: A Case Study with White Wine10.1111/joss.12528 ↗
32. Coherences of Instrumental and Sensory Characteristics: Case Study on Cherry Tomatoes10.1111/1750-3841.12685 ↗
33. New Multiresistant Apple Cultivars—Complex Assessment of Sensory and Some Instrumental Attributes10.1556/AAlim.42.2013.2.14 ↗
34. Importance of Fruit Variability in the Assessment of Apple Quality by Sensory Evaluation10.1016/j.postharvbio.2012.11.005 ↗
35. A Combined Sensory-Instrumental Tool for Apple Quality Evaluation10.1016/j.postharvbio.2014.05.016 ↗
36. Comparison of Different Ocimum Basilicum L. Gene Bank Accessions Analyzed by GC–MS and Sensory Profile10.1016/j.indcrop.2015.01.013 ↗
37. Sipos, L., Végh, R., Bodor, Z., Zaukuu, J.-L.Z., Hitka, G., Bázár, G., and Kovacs, Z. (2020). Classification of Bee Pollen and Prediction of Sensory and Colorimetric Attributes—A Sensometric Fusion Approach by e-Nose, e-Tongue and NIR. Sensors, 20.10.3390/s20236768 ↗
38. Chun, S., Chambers, E., and Han, I. (2020). Development of a Sensory Flavor Lexicon for Mushrooms and Subsequent Characterization of Fresh and Dried Mushrooms. Foods, 9.10.3390/foods9080980 ↗
39. Training Effects on Performance of Descriptive Panelists10.1111/j.1745-459X.2004.082402.x ↗
40. Comparison of Results from Cupping and Descriptive Sensory Analysis of Colombian Brewed Coffee10.1111/joss.12104 ↗
41. Sensory Profiling, the Blurred Line between Sensory and Consumer Science. A Review of Novel Methods for Product Characterization10.1016/j.foodres.2012.06.037 ↗
42. Who Told You the Triangle Test Was Simple?10.1016/0950-3293(95)00022-4 ↗
43. ISO 16820 (2019). Sensory Analysis—Methodology—Sequential Analysis, ISO.
44. ISO 11132 (2012). Sensory Analysis—Methodology—Guidelines for Monitoring the Performance of a Quantitative Sensory Panel, ISO.
45. ISO 13299 (2016). Sensory Analysis—Methodology—General Guidance For Establishing A Sensory Profile, ISO.
46. Kilcast, D. (2010). Sensory Analysis for Food and Beverage Quality Control: A Practical Guide, Elsevier.10.1533/9781845699512 ↗
47. Statistical Testing of Individual Differences in Sensory Profiling10.1016/S0950-3293(03)00007-7 ↗
48. Taking Individual Scaling Differences into Account by Analyzing Profile Data with the Mixed Assessor Model10.1016/j.foodqual.2014.07.005 ↗
49. A Panel Performance Procedure Implemented in R10.1111/j.1745-459X.2005.00021.x ↗
50. Assessing the Performance of a Sensory Panel–Panellist Monitoring and Tracking10.1002/cem.918 ↗
51. Panel Performance Monitoring by Poincaré Plot: A Case Study on Flavoured Bottled Waters10.1016/j.foodres.2017.04.029 ↗
52. Design of Experiment with Sensory Data: A Pragmatic Data Analysis Approach10.1111/joss.12489 ↗
53. ISO 29842 (2011). Sensory Analysis—Methodology—Balanced Incomplete Block Designs, ISO.
54. Multiplicative Decomposition of the Scaling Effect in the Mixed Assessor Model into a Descriptor-Specific and an Overall Coefficients10.1016/j.foodqual.2015.10.005 ↗
55. The MAM-CAP Table: A New Tool for Monitoring Panel Performances10.1016/j.foodqual.2013.07.004 ↗
56. Panel Performance for Temporal Dominance of Sensations10.1016/j.foodqual.2014.05.002 ↗
57. Visualization of Sensory Profiling Data for Performance Monitoring10.1016/j.lwt.2005.09.014 ↗
58. Analysing Sensory Panel Performance in a Proficiency Test Using the PanelCheck Software10.1007/s00217-009-1185-y ↗
59. Assessing Sensory Panelist Performance Using Repeatability and Reproducibility Measures10.1016/S0950-3293(01)00038-6 ↗
60. Sensominer: A Package for Sensory Data Analysis10.1111/j.1745-459X.2007.00137.x ↗
61. Le, S., and Worch, T. (2018). Analyzing Sensory Data with R, CRC Press.10.1201/9781315373416 ↗
62. Measurement of the Reliability of Sensory Panel Performances10.1016/j.foodqual.2005.04.010 ↗
63. Exploring Consumer Product Profiling Techniques and Their Linkage to a Quantitative Descriptive Analysis10.1016/j.foodqual.2010.09.005 ↗
64. López-López, A., Sánchez-Gómez, A.H., Montaño, A., Cortés-Delgado, A., and Garrido-Fernández, A. (2019). Panel and Panelist Performance in the Sensory Evaluation of Black Ripe Olives from Spanish Manzanilla and Hojiblanca Cultivars. Foods, 8.10.3390/foods8110562 ↗
65. A Procedure for Sensory Evaluation of Bread: Protocol Developed by a Trained Panel10.1111/j.1745-459X.2011.00342.x ↗
66. Monitoring Calibration of Descriptive Sensory Panels Using Distance from Target Measurements10.1016/j.foodqual.2005.03.011 ↗
67. Agreement and Reliability Assessments for Performance of Sensory Descriptive Panel10.1111/j.1745-459X.2003.tb00373.x ↗
68. Intraclass Correlation Coefficient (ICC): A Framework for Monitoring and Assessing Performance of Trained Sensory Panels and Panelists10.1111/j.1745-459X.2012.00399.x ↗
69. Application of a Panel Performance Reliability versus Product Effect Size (PR-ES) Framework: A Protein Powder Case Study10.1016/j.foodqual.2020.104152 ↗
70. Testing Panel Consistency with GCAP Method in Food Profile Analysis10.1002/cem.1135 ↗
71. Comparison of Novel Sensory Panel Performance Evaluation Techniques with E-Nose Analysis Integration10.1002/cem.1391 ↗
72. Canonical Variate Analysis of Sensory Profiling Data10.1111/joss.12160 ↗
73. CLV3W: A Clustering around Latent Variables Approach to Detect Panel Disagreement in Three-Way Conventional Sensory Profiling Data10.1016/j.foodqual.2015.03.013 ↗
74. Method and Model Comparison by Sum of Ranking Differences in Cases of Repeated Observations (Ties)10.1016/j.chemolab.2013.06.007 ↗
75. Olive Oil Sensory Evaluation: An Alternative to the Robust Coefficient of Variation (CVr %) for Measuring Panel Group Performance in Official Tasting Sessions10.1016/j.tifs.2019.02.044 ↗
76. Use of Feedback Calibration to Reduce the Training Time for Wine Panels10.1016/j.foodqual.2005.07.005 ↗
77. Grapes: A Method and a Sas® Program for Graphical Representations of Assessor Performances10.1111/j.1745-459X.1994.tb00238.x ↗
78. Refined Approximations to Permutation Tests for Multivariate Inference10.1016/0167-9473(94)00064-2 ↗
79. Defining and Validating Assessor Compromises about Product Distances and Attribute Correlations
80. Modeling the Evolution of the Performance of a Sensory Panel: A Mixed-Model and Control Chart Approach10.1111/j.1745-459X.2007.00103.x ↗
81. Monte Carlo Based Test for Inferring about the Unidimensionality of a Brazilian Coffee Sensory Panel10.1016/j.foodqual.2009.08.018 ↗
82. Multi-Way Models for Sensory Profiling Data10.1002/cem.1097 ↗
83. CONS an R Based Graphical Interface to Perform Consonance Analysis10.1016/j.foodqual.2017.07.008 ↗
84. Elgaard, L., Mielby, L.A., Hopfer, H., and Byrne, D.V. (2019). A Comparison of Two Sensory Panels Trained with Different Feedback Calibration Range Specifications via Sensory Description of Five Beers. Foods, 8.10.3390/foods8110534 ↗
85. Assessing Sensory Panel Performance Using Generalizability Theory10.1016/j.foodqual.2015.02.019 ↗
86. Performance Indices in Descriptive Sensory Analysis—A Complimentary Screening Tool for Assessor and Panel Performance10.1016/j.foodqual.2012.06.012 ↗
87. ISO EA-4/09 (2017). Accreditation for Sensory Testing Laboratories, ISO.
88. ISO/IEC 17043 (2010). Conformity Assessment—General Requirements for Proficiency Testing, ISO.
89. ISO/IEC 17025 (2017). Testing and Calibration Laboratories, ISO.
90. Kilcast, D. (2010). 3-Proficiency Testing of Sensory Panels. Sensory Analysis for Food and Beverage Quality Control, Woodhead Publishing.10.1533/9781845699512 ↗
91. Inter-Laboratory Comparison of Sensory Profiles: Methodology and Results10.1016/S0950-3293(01)00015-5 ↗
92. Proficiency Testing for Sensory Profile Panels: Measuring Panel Performance10.1016/S0950-3293(02)00022-8 ↗
93. Proficiency Testing for Sensory Ranking Panels: Measuring Panel Performance10.1016/S0950-3293(02)00083-6 ↗
94. A Comparison between Nine Laboratories Performing Triangle Tests10.1016/j.foodqual.2011.04.007 ↗
95. Inter-Laboratory Assessment by Trained Panelists from France and the United Kingdom of Beef Cooked at Two Different End-Point Temperatures10.1016/j.meatsci.2016.07.026 ↗
96. Barbieri, S., Brkić Bubola, K., Bendini, A., Bučar-Miklavčič, M., Lacoste, F., Tibet, U., Winkelmann, O., García-González, D.L., and Gallina Toschi, T. (2020). Alignment and Proficiency of Virgin Olive Oil Sensory Panels: The OLEUM Approach. Foods, 9.10.3390/foods9030355 ↗
97. Meullenet, J.-F., Xiong, R., and Findlay, C.J. (2008). Multivariate and Probabilistic Analyses of Sensory Science Problems, John Wiley & Sons.10.1002/9780470277539 ↗