Confidence Interval Comparisons For Probability of Detection On Hit/Miss Data
Source abstract
Abstract
Probability of detection (POD) studies for evaluating the capabilities of an inspection system for Air Force aircraft structural components commonly use a Logistic Regression model with a Wald 95% confidence interval. However, hit/miss POD data is distributed as a Binomial, and the sample sizes are commonly too small for Wald’s identically and independently normality distributed assumption to be true. This paper uses a large set of simulated representative hit/miss data to compare and contrast the performance of the four confidence intervals methods: Standard Wald, Modified Wald, Profile Likelihood Ratio, and Profile Modified Likelihood Ratio. Performance is measured in terms of bias and existence of a90/95 with respect to data distribution, sample size, overlap, and evenness. This paper provides guidance and methodology on new POD methods that more reliably and accurately estimate a90/95.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Statistical Methods and Models · Probabilistic and Robust Engineering Design · Advanced Statistical Process Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (18)
Show source-provided references
1. Agresti, A. 2002. Categorical Data Analysis. 2nd ed. Hoboken, New Jersey: Wiley. https://doi:10.1002/0471249688.10.1002/0471249688 ↗
2. Annis, Charles P.E. 2014. “Influence of sample characteristics on probability of detection curves.” AIP Conference Proceedings 1581, 2039-2046. https://doi.org/10.1063/1.486507410.1063/1.4865074 ↗
3. Annis, C., L. Gandossi. 2012. ENIQ TGR Technical Document - Influence of Sample Size and Other Factors on Hit/Miss Probability of Detection Curves (ENIQ report N. 47). EUR 25200 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC68677
4. Annis, C., J. C. Aldrin, and H. A. Sabbagh. 2015. “Profile likelihood: What to do when maximum probability of detection never gets to one.” Materials Evaluation, 73 (1): 96–99.
5. Annis, C., L. Gandossi, and O. Martin. 2013. “Optimal sample size for probability of detection curves.” Nuclear Engineering and Design, 262 (September 2013): 98-105. JRC7495110.1016/j.nucengdes.2013.03.059 ↗
6. Barndorff-Nielsen, O. E. 1986. “Inference on Full or Partial Parameters Based on the Standardized Signed Log Likelihood Ratio.” Biometrika 73 (2): 307–22. https://doi.org/10.2307/2336207.10.2307/2336207 ↗
7. Barndorff-Nielsen, O., and D.R. Cox. 1979. “Edgeworth and Saddle-Point Approximations with Statistical Applications.” Journal of the Royal Statistical Society: Series B (Methodological), 41: 279-299. https://doi.org/10.1111/j.2517-6161.1979.tb01085.x10.1111/j.2517-6161.1979.tb01085.x ↗
8. Bates, O., and D. Watts. 1988. Nonlinear Regression Analysis and Its Applications. Hoboken, New Jersey: Wiley. doi:10.1002/9780470316757.10.1002/9780470316757 ↗
9. Brazzale, A.R., A.C. Davison, and N. Reid. 2007. Applied Asymptotics: Case Studies in Small-Sample Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge, UK: Cambridge University Press.10.1017/cbo9780511611131 ↗
10. Brazzale, Alessandra R., and Anthony C. Davison. 2008. “Accurate Parametric Inference for Small Samples.” Statist. Sci. 23 (4): 465–484. https://doi.org/10.1214/08-STS27310.1214/08-sts273 ↗
11. Casella, G., and R. Berger. 2001. Statistical Inference. 2nd ed. Boston, Massachusetts: Cengage Learning.
12. Gandossi, L. and C. Annis. 2010. ENIQ TGR Technical Document – Probability of Detection Curves: Statistical Best-Practices (ENIQ report N. 41). EUR 24429 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC56672.
13. Gerhard, D. 2016. “Simultaneous small sample inference for linear
14. combinations of generalized linear model parameters.” Communications in Statistics – Simulation and Computation 45 (8): 2678–90. https://doi:10.1080/03610918.2014.895836.10.1080/03610918.2014.895836 ↗
15. Hothorn, T., F. Bretz, and P. Westfall. 2008. Simultaneous Inference in General Parametric Models. Technical Report. Department of Statistics, University of Munich.10.1002/bimj.200810425 ↗
16. Mameli, V. and A.R. Brazzale. 2016. “Modern likelihood inference for the maximum/minimum of a bivariate normal vector.” Journal of Statistical Computation and Simulation, 86 (10): 1869-1890, https://doi.org/10.1080/00949655.2015.108987210.1080/00949655.2015.1089872 ↗
17. US DOD (Department of Defense). 2010. MIL-HDBK-1823A: Nondestructive Evaluation System Reliability Assessment, Department of Defense Handbook.10.21236/ada308209 ↗
18. Venables, W.N. and B.D. Ripley. 2002. Modern Applied Statistics with S. 4th edition. Statistics and Computing. New York, New York: Springer. https://doi.org/10.1007/978-0-387-21706-210.1007/978-0-387-21706-2 ↗