A Review of Methods for Assessing the Quality of Measurement Systems and Results
Source abstract
Abstract
This paper provides an overview of key methods for assessing the quality of measurement systems and measurement results. Fundamental concepts are analyzed and their definitions compared across various metrological standards. Particular emphasis is placed on the issue of terminological inconsistency, as the same terms are interpreted differently in different documents, which may lead to misinterpretation or incorrect application of methods. This paper also presents practical examples of how these methods are applied to real data. Clear indications are provided regarding the conditions under which each method is recommended, depending on factors such as model complexity, data availability, and intended application. In addition, it offers a discussion and recommendations for future directions, highlighting the need for harmonized terminology, standardization of evaluation procedures, and the adoption of advanced technologies such as artificial intelligence and machine learning in the assessment of measurement quality.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Scientific Measurement and Uncertainty Evaluation · Advanced Statistical Process Monitoring · Pesticide Residue Analysis and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Polytechnics, Dr. Franjo Tuđman Defense and Security University, Ilica 256b, 10000 Zagreb, Croatia
Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ivana Lučića 5, 10000 Zagreb, Croatia
Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovića 6, 21000 Novi Sad, Serbia
References (99)
Show source-provided references
1. Using Measurement Uncertainty in Decision-Making and Conformity Assessment10.1088/0026-1394/51/4/S206 ↗
2. BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPA, and OIML (2012). Evaluation of Measurement Data—The Role of Measurement Uncertainty in Conformity Assessment, JCGM.
3. AIAG (2010). Measurement Systems Analysis: Reference Manual, Automotive Industry Action Group. [4th ed.].
4. (2023). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 1: General Principles and Definitions (Standard No. ISO 5725-1:2023).
5. (2019). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 2: Basic Method for the Determination of Repeatability and Reproducibility of a Standard Measurement Method (Standard No. ISO 5725-2:2019).
6. (2023). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 3: Intermediate Precision and Alternative Designs for Collaborative Studies (Standard No. ISO 5725-3:2023).
7. (2020). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 4: Basic Methods for the Determination of the Trueness of a Standard Measurement Method (Standard No. ISO 5725-4:2020).
8. (1998). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 5: Alternative Methods for the Determination of the Precision of a Standard Measurement Method (Standard No. ISO/DIS 5725-5).
9. (1994). Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 6: Use in Practice of Accuracy Values (Standard No. ISO 5725-6:1994).
10. (2008). Evaluation of Measurement Data—Guide to the Expression of Uncertainty in Measurement (Standard No. JCGM 100:2008).
11. (2008). Evaluation of Measurement Data—Supplement 1 to the “Guide to the Expression of Uncertainty in Measurement”—Propagation of Distributions Using a Monte Carlo Method (Standard No. JCGM 101:2008).
12. Informative Bayesian Type A Uncertainty Evaluation, Especially Applicable to a Small Number of Observations10.1088/1681-7575/aa787f ↗
13. Bayesian Methods for Type A Evaluation of Standard Uncertainty10.1088/1681-7575/aad103 ↗
14. Bayesian Assessment of Uncertainty in Metrology: A Tutorial10.1088/0026-1394/47/3/R01 ↗
15. (2017). Guidance for the Use of Repeatability, Reproducibility and Trueness Estimates in Measurement Uncertainty Evaluation (Standard No. ISO 21748:2017).
16. Revision of the ‘Guide to the Expression of Uncertainty in Measurement’10.1088/0026-1394/49/6/702 ↗
17. Linear Mixed Models: Gum and Beyond10.2478/msr-2014-0009 ↗
18. Runje, B., Novak, A.H., Razumić, A., Piljek, P., Štrbac, B., and Orosnjak, M. (2019, January 23–26). Evaluation of Consumer and Producer Risk in Conformity Assessment Decisions. Proceedings of the 30th Daaam International Symposium on Inteligent Manufacturing and Automation, Zadar, Croatia.10.2507/30th.daaam.proceedings.007 ↗
19. Cox, M.G., and Harris, P.M. (2025, July 24). Software Support for Metrology Best Practice Guide No. 6. Uncertainty Evaluation. Available online: https://eprintspublications.npl.co.uk/4655/.
20. Bayesian Alternative to the ISO-GUM’s Use of the Welch–Satterthwaite Formula10.1088/0026-1394/43/1/001 ↗
21. Measurement Uncertainty Assessment of Coordinate Measuring Machines by Simulation and Planned Experimentation10.1016/j.cirpj.2011.01.007 ↗
22. Božić, D., Runje, B., and Razumić, A. (2024). Risk Assessment for Linear Regression Models in Metrology. Appl. Sci., 14.10.20944/preprints202402.0948.v1 ↗
23. Traceable Measurements Using Sensor Networks
24. Bell, S.A. (2001). A Beginner’s Guide to Uncertainty of Measurement, National Physical Laboratory.
25. (2017). Testing and Calibration Laboratories (Standard No. ISO-ISO/IEC 17025).
26. Measurement Uncertainty and Optimized Conformance Assessment10.1016/j.measurement.2006.04.007 ↗
27. Allard, A., Fischer, N., Smith, I., Harris, P., and Pendrill, L. (2019, January 24–26). Risk Calculations for Conformity Assessment in Practice. Proceedings of the 19th International Congress of Metrology (CIM2019), Paris, France.10.1051/metrology/201916001 ↗
28. Ellison, S.L.R., and Williams, A. (2012). Quantifying Uncertainty in Analytical Measurement, Eurachem/CITAC. [3rd ed.].
29. A Review of Methods for Measurement Systems Capability Analysis10.1080/00224065.2003.11980232 ↗
30. Repeatability and Reproducibility Techniques for the Analysis of Measurement Systems10.1016/j.measurement.2016.02.041 ↗
31. Assessing Bias and Linearity of a Measurement System10.4028/www.scientific.net/AMM.760.621 ↗
32. Analysis on Accuracy of Bias, Linearity and Stability of Measurement System in Ball Screw Processes by Simulation10.3390/su71115464 ↗
33. Variable Data Measurement Systems Analysis: Advances in Gage Bias and Linearity Referencing and Acceptability10.1051/ijmqe/2020010 ↗
34. Effects of Within Part Variation on Measurement System Analysis
35. Runje, B., Novak, A.H., and Razumić, A. (2017, January 4–6). Measurement System Analysis in Production Process. Proceedings of the XVII International Scientific Conference on Industrial Systems (IS’17), Novi Sad, Serbia.
36. Wang, Y., Fu, M., Xu, L., and Zou, J. (2020). Quality Confirmation of Electrical Measurement Data for Space Parts Based on MSA Method. Signal and Information Processing, Networking and Computers, Proceedings of the 7th International Conference on Signal and Information Processing, Networking and Computers (ICSINC), Rizhao, China, 21–23 September 2020, Springer.10.1007/978-981-15-4163-6_41 ↗
37. Evaluation of Adequacy of Quantitative Laboratory Chemical Analysis Techniques10.1134/S0020168509140283 ↗
38. BES TESTTM Accuracy Evaluation by Means of 3D-Printed Phantoms10.1177/09544062221133675 ↗
39. Interlaboratory Comparison of Dicentric Chromosome Assay Using Electronically Transmitted Images10.1093/rpd/ncs139 ↗
40. The GUM Perspective on Straight-Line Errors-in-Variables Regression10.1016/j.measurement.2021.110340 ↗
41. Non-Linear Models and Best Estimates in the GUM10.1088/0026-1394/43/4/S05 ↗
42. Linear and Non-Linear Flow Behavior of Welan Gum Solutions10.1007/s00397-018-1120-x ↗
43. On the Application of Supplement 1 to the GUM to Non-Linear Problems10.1088/0026-1394/48/5/014 ↗
44. Evaluation of Measurement Uncertainty Based on the Propagation of Distributions Using Monte Carlo Simulation10.1023/B:METE.0000008439.82231.ad ↗
45. Comparison of GUM and Monte Carlo Methods for Evaluating Measurement Uncertainty of Perspiration Measurement Systems10.1016/j.measurement.2016.03.007 ↗
46. Comparison of GUM and Monte Carlo Methods for the Uncertainty Estimation in Hardness Measurements10.1051/ijmqe/2017014 ↗
47. Uncertainty Assessment in Geodetic Network Adjustment by Combining GUM and Monte-Carlo-Simulations10.1515/jag-2016-0017 ↗
48. Uncertainty Estimation and Monte Carlo Simulation Method10.1016/S0955-5986(01)00015-2 ↗
49. Monte Carlo Simulation in Uncertainty Evaluation: Strategy, Implications and Future Prospects10.1007/s12647-019-00345-5 ↗
50. Quantification of Model Uncertainty and Variability for Landslide Displacement Prediction Based on Monte Carlo Simulation10.1016/j.gr.2023.03.006 ↗
51. Asymmetrical Uncertainties10.1088/1681-7575/ab2a8d ↗
52. Modeling of Asymmetry between Gasoline and Crude Oil Prices: A Monte Carlo Comparison10.1007/s10614-010-9218-y ↗
53. A Propensity-Based Framework for Measurement Uncertainty Analysis10.1016/j.measurement.2023.112693 ↗
54. The Soft Monte Carlo Method10.1016/j.apm.2021.01.022 ↗
55. A Bayesian Approach to Quantifying Uncertainties and Improving Generalizability in Traffic Prediction Models10.1016/j.trc.2024.104585 ↗
56. Ode to Bayesian Methods in Metrology10.1088/1681-7575/acf66b ↗
57. MCMC-Based Image Reconstruction with Uncertainty Quantification10.1137/11085760X ↗
58. Bayesian and Markov Chain Monte Carlo Methods for Identifying Nonlinear Systems in the Presence of Uncertainty10.1098/rsta.2014.0405 ↗
59. Bayesian Neural Networks via MCMC: A Python-Based Tutorial10.1109/ACCESS.2024.3401234 ↗
60. Bayesian Conformity Assessment in Presence of Systematic Measurement Errors10.1088/0026-1394/53/2/S74 ↗
61. Calculation of Uncertainty in the Presence of Prior Knowledge10.1088/0026-1394/44/2/002 ↗
62. Bayesian Model Selection for Group Studies10.1016/j.neuroimage.2009.03.025 ↗
63. Bayesian Statistics and Modelling10.1038/s43586-020-00001-2 ↗
64. Guidance on Bayesian Uncertainty Evaluation for a Class of GUM Measurement Models10.1088/1681-7575/abb065 ↗
65. Outlier Detection in the Analysis of Nested Gage R&R, Random Effect Model10.21307/stattrans-2019-023 ↗
66. A Study of the AIAG Measurement System Analysis (MSA) Method for Quality Control
67. Evaluating Measurement and Process Capabilities by GR&R with Four Quality Measures10.1016/j.measurement.2010.02.016 ↗
68. Analysis of the Wind Average Speed in Different Brazilian States Using the Nested GR&R Measurement System10.1016/j.measurement.2017.10.048 ↗
69. Measurement System Analyses–Gauge Repeatability and Reproducibility Methods10.1515/msr-2018-0004 ↗
70. Implementation of Measurement System Analysis System (MSA): In the Piston Ring Company Case Study
71. Multivariate GR&R through Factor Analysis10.1016/j.measurement.2019.107107 ↗
72. A New Multivariate Gage R&R Method for Correlated Characteristics10.1016/j.ijpe.2013.02.018 ↗
73. Reproducibility of Areal Topography Parameters Obtained by Atomic Force Microscope10.31803/tg-20250324183037 ↗
74. Bontempi, M., Cardinale, U., Bragonzoni, L., Muccioli, G.M.M., Alesi, D., di Matteo, B., Marcacci, M., and Zaffagnini, S. (2020). A Computer Simulation Protocol to Assess the Accuracy of a Radio Stereometric Analysis (RSA) Image Processor According to the ISO-5725. arXiv.
75. A Fully Customizable MATLAB Framework for MSA Based on ISO 5725 Standard10.1088/1361-6501/aa542f ↗
76. Calculation of Measurement Uncertainty in Clinical Chemistry10.1373/clinchem.2004.041525 ↗
77. Basics of Interlaboratory Studies: The Trends in the New ISO 5725 Standard Edition10.1016/0165-9936(95)93243-Z ↗
78. Razumić, A. (2024). Procjena Mjerne Nesigurnosti Rezultata Mjerenja na Području Mikroskopije Atomskih Sila u Dimenzijskom Nanomjeriteljstvu. [Ph.D. Thesis, Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb].
79. Alar, V., Razumić, A., Runje, B., Stojanović, I., Kurtela, M., and Štrbac, B. (2025). Application of Areal Topography Parameters in Surface Characterization. Appl. Sci., 15.10.3390/app15126573 ↗
80. Atomic Force Microscopy: Step Height Measurement Uncertainty Evaluation10.31803/tg-20230829155921 ↗
81. van der Veen, A., Cox, M., Greenwood, J., Bošnjaković, A., Karahodžić, V., Martens, S., Klauenberg, K., Elster, C., Demeyer, S., and Fischer, N. (2021). Good Practice in Evaluating Measurement Uncertainty—Compendium of Examples, EURAMET.
82. Cox, M.G., Greenwood, J., Bošnjaković, A., and Karahodžić, V. (2021). Two-point and Multipoint Calibration. Good Practice in Evaluating Measurement Uncertainty, EURAMET.
83. Ribeiro, A.S., Cox, M.G., Almeida, M.C., Sousa, J.A., Martins, L.L., Simoes, C., Brito, R., Loureiro, D., Silva, M.A., and Soares, A.C. (2021). Evaluation of Measurement Uncertainty in Average Areal Rainfall—Uncertainty Propagation for Three Methods. Good Practice in Evaluating Measurement Uncertainty, EURAMET.
84. Martins, L.L., Ribeiro, A.S., Cox, M.G., Sousa, J.A., Loureiro, D., Almeida, M.C., Silva, M.A., Brito, R., and Soares, A.C. (2021). Evaluation of Measurement Uncertainty in SBI—Single Burning Item Reaction to Fire Test. Good Practice in Evaluating Measurement Uncertainty, EURAMET.10.1142/9789811242380_0019 ↗
85. Cox, M.G., Gardiner, T., Robinson, R., Smith, T., Ellison, S.L.R., and Van der Veen, A.M.H. (2021). Greenhouse Gas Emission Inventories. Good Practice in Evaluating Measurement Uncertainty, EURAMET.
86. Demeyer, S., Fischer, N., Cox, M.G., Van der Veen, A.M.H., Sousa, J.A., Pellegrino, O., Bošnjaković, A., Karahodžić, V., and Elster, C. (2021). Bayesian Approach Applied to the Mass Calibration Example in JCGM 101:2008. Good Practice in Evaluating Measurement Uncertainty, EURAMET.
87. Caebergs, T., De Boeck, B., Pétry, J., Sebaïhi, N., Cox, M.G., Fischer, N., and Greenwood, J. (2021). Uncertainty Evaluation of Nanoparticle Size by AFM, by Means of an Optimised Design of Experiment for a Hierarchical Mixed Model in a Bayesian Framework Approach. Good Practice in Evaluating Measurement Uncertainty, EURAMET.
88. (2022). Medical Laboratories—Requirements for Quality and Competence (Standard No. ISO 15189:2022).
89. (2012). Conformity Assessment—Requirements for the Operation of Various Types of Bodies Performing Inspection (Standard No. ISO/IEC 17020:2012).
90. (2003). Measurement Management Systems—Requirements for Measurement Processes and Measuring Equipment (Standard No. ISO 10012:2003).
91. (2017). Geometrical Product Specifications (GPS)—Inspection by Measurement of Workpieces and Measuring Equipment Part 1: Decision Rules for Verifying Conformity or Nonconformity with Specifications (Standard No. ISO 14253-1:2017).
92. (2015). Quality Management Systems—Requirements (Standard No. ISO 9001:2015).
93. Keran, Z., Runje, B., Piljek, P., and Razumić, A. (2025). Roboforming in ISF—Characteristics, Development, and the Step Towards Industry 5.0. Sustainability, 17.10.3390/su17062562 ↗
94. Modelling Risks in Transition from Industry 4.0 to Industry 5.010.1007/s10479-024-06055-9 ↗
95. Industry 5.0: Do Risk Assessment and Risk Management Need to Update? And If Yes, How?
96. The Need for Uncertainty Quantification in Machine-Assisted Medical Decision Making10.1038/s42256-018-0004-1 ↗
97. Quantification of Interpatient 12-Lead ECG Variabilities within a Healthy Cohort10.1515/cdbme-2020-3127 ↗
98. Aston, P.J., Mehari, T., Bosnjakovic, A., Harris, P.M., Sundar, A., Williams, S.E., Dössel, O., Loewe, A., Nagel, C., and Strodthoff, N. (2022, January 4–7). Multi-Class ECG Feature Importance Rankings: Cardiologists vs Algorithms. Proceedings of the 2022 Computing in Cardiology (CinC), Tampere, Finland.10.22489/CinC.2022.087 ↗
99. Ecg Feature Importance Rankings: Cardiologists vs. Algorithms10.1109/JBHI.2024.3354301 ↗