journal article·2022-12-01

Confidence Interval Comparisons For Probability of Detection On Hit/Miss Data

Christine E. Knott, Christine Schubert Kabban

Materials Evaluation· Volume 80· Issue 12
Publisher page

Abstract

Probability of detection (POD) studies for evaluating the capabilities of an inspection system for Air Force aircraft structural components commonly use a Logistic Regression model with a Wald 95% confidence interval. However, hit/miss POD data is distributed as a Binomial, and the sample sizes are commonly too small for Wald’s identically and independently normality distributed assumption to be true. This paper uses a large set of simulated representative hit/miss data to compare and contrast the performance of the four confidence intervals methods: Standard Wald, Modified Wald, Profile Likelihood Ratio, and Profile Modified Likelihood Ratio. Performance is measured in terms of bias and existence of a90/95 with respect to data distribution, sample size, overlap, and evenness. This paper provides guidance and methodology on new POD methods that more reliably and accurately estimate a90/95.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Statistical Methods and Models · Probabilistic and Robust Engineering Design · Advanced Statistical Process Monitoring

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (18)

Show source-provided references
1. Agresti, A. 2002. Categorical Data Analysis. 2nd ed. Hoboken, New Jersey: Wiley. https://doi:10.1002/0471249688.10.1002/0471249688 ↗
2. Annis, Charles P.E. 2014. “Influence of sample characteristics on probability of detection curves.” AIP Conference Proceedings 1581, 2039-2046. https://doi.org/10.1063/1.486507410.1063/1.4865074 ↗
3. Annis, C., L. Gandossi. 2012. ENIQ TGR Technical Document - Influence of Sample Size and Other Factors on Hit/Miss Probability of Detection Curves (ENIQ report N. 47). EUR 25200 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC68677
4. Annis, C., J. C. Aldrin, and H. A. Sabbagh. 2015. “Profile likelihood: What to do when maximum probability of detection never gets to one.” Materials Evaluation, 73 (1): 96–99.
5. Annis, C., L. Gandossi, and O. Martin. 2013. “Optimal sample size for probability of detection curves.” Nuclear Engineering and Design, 262 (September 2013): 98-105. JRC7495110.1016/j.nucengdes.2013.03.059 ↗
6. Barndorff-Nielsen, O. E. 1986. “Inference on Full or Partial Parameters Based on the Standardized Signed Log Likelihood Ratio.” Biometrika 73 (2): 307–22. https://doi.org/10.2307/2336207.10.2307/2336207 ↗
7. Barndorff-Nielsen, O., and D.R. Cox. 1979. “Edgeworth and Saddle-Point Approximations with Statistical Applications.” Journal of the Royal Statistical Society: Series B (Methodological), 41: 279-299. https://doi.org/10.1111/j.2517-6161.1979.tb01085.x10.1111/j.2517-6161.1979.tb01085.x ↗
8. Bates, O., and D. Watts. 1988. Nonlinear Regression Analysis and Its Applications. Hoboken, New Jersey: Wiley. doi:10.1002/9780470316757.10.1002/9780470316757 ↗
9. Brazzale, A.R., A.C. Davison, and N. Reid. 2007. Applied Asymptotics: Case Studies in Small-Sample Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge, UK: Cambridge University Press.10.1017/cbo9780511611131 ↗
10. Brazzale, Alessandra R., and Anthony C. Davison. 2008. “Accurate Parametric Inference for Small Samples.” Statist. Sci. 23 (4): 465–484. https://doi.org/10.1214/08-STS27310.1214/08-sts273 ↗
11. Casella, G., and R. Berger. 2001. Statistical Inference. 2nd ed. Boston, Massachusetts: Cengage Learning.
12. Gandossi, L. and C. Annis. 2010. ENIQ TGR Technical Document – Probability of Detection Curves: Statistical Best-Practices (ENIQ report N. 41). EUR 24429 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC56672.
13. Gerhard, D. 2016. “Simultaneous small sample inference for linear
14. combinations of generalized linear model parameters.” Communications in Statistics – Simulation and Computation 45 (8): 2678–90. https://doi:10.1080/03610918.2014.895836.10.1080/03610918.2014.895836 ↗
15. Hothorn, T., F. Bretz, and P. Westfall. 2008. Simultaneous Inference in General Parametric Models. Technical Report. Department of Statistics, University of Munich.10.1002/bimj.200810425 ↗
16. Mameli, V. and A.R. Brazzale. 2016. “Modern likelihood inference for the maximum/minimum of a bivariate normal vector.” Journal of Statistical Computation and Simulation, 86 (10): 1869-1890, https://doi.org/10.1080/00949655.2015.108987210.1080/00949655.2015.1089872 ↗
17. US DOD (Department of Defense). 2010. MIL-HDBK-1823A: Nondestructive Evaluation System Reliability Assessment, Department of Defense Handbook.10.21236/ada308209 ↗
18. Venables, W.N. and B.D. Ripley. 2002. Modern Applied Statistics with S. 4th edition. Statistics and Computing. New York, New York: Springer. https://doi.org/10.1007/978-0-387-21706-210.1007/978-0-387-21706-2 ↗