journal article·2021-11-22Open access

How to Use Machine Learning to Improve the Discrimination between Signal and Background at Particle Colliders

Xabier Cid Cid Vidal, Lorena Dieste Dieste Maroñas, Álvaro Dosil Dosil Suárez

Applied Sciences· Volume 11· Issue 22

Abstract

The popularity of Machine Learning (ML) has been increasing in recent decades in almost every area, with the commercial and scientific fields being the most notorious ones. In particle physics, ML has been proven a useful resource to make the most of projects such as the Large Hadron Collider (LHC). The main advantage provided by ML is a reduction in the time and effort required for the measurements carried out by experiments, and improvements in the performance. With this work we aim to encourage scientists working with particle colliders to use ML and to try the different alternatives that are available, focusing on the separation of signal and background. We assess some of the most-used libraries in the field, such as Toolkit for Multivariate Data Analysis with ROOT, and also newer and more sophisticated options such as PyTorch and Keras. We also assess the suitability of some of the most common algorithms for signal-background discrimination, such as Boosted Decision Trees, and propose the use of others, namely Neural Networks. We compare the overall performance of different algorithms and libraries in simulated LHC data and produce some guidelines to help analysts deal with different situations. Examples include the use of low or high-level features from particle detectors or the amount of statistics that are available for training the algorithms. Our main conclusion is that the algorithms and libraries used more frequently at LHC collaborations might not always be those that provide the best results for the classification of signal candidates, and fully connected Neural Networks trained with Keras can improve the performance scores in most of the cases we formulate.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Particle Detector Development and Performance · Particle physics theoretical and experimental studies · Computational Physics and Python Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Instituto Galego de Física de Altas Enerxías (IGFAE), Universidade de Santiago de Compostela, 15782 Santiago de Compostela, Spain

Triple Alpha Innovation, Cuns, 4, Outes, 15286 A Coruña, Spain

Funding

Spanish Research Agency · MINECO (Spain) · XuntaGAL

References (73)

Show source-provided references
1. Feickert, M., and Nachman, B. (2021, November 17). A Living Review of Machine Learning for Particle Physics. Available online: https://arxiv.org/abs/2102.02770.
2. The LHCb Detector at the LHC
3. LHCb Detector Performance10.1142/S0217751X15300227 ↗
4. (2021, October 01). CERN Storage. Available online: https://home.cern/science/computing/storage.
5. Learning representations of irregular particle-detector geometry with distance-weighted graph networks10.1140/epjc/s10052-019-7113-9 ↗
6. Reframing Jet Physics with New Computational Methods10.1051/epjconf/202125103059 ↗
7. Ariadne: PyTorch Library for Particle Track Reconstruction Using Deep Learning10.1063/5.0063300 ↗
8. End-to-End Jet Classification of Boosted Top Quarks with CMS Open Data10.1051/epjconf/202125104030 ↗
9. Akchurin, N., Cowden, C., Damgov, J., Hussain, A., and Kunori, S. (2021, November 17). Perspectives on the Calibration of CNN Energy Reconstruction in Highly Granular Calorimeters. Available online: https://arxiv.org/abs/2108.10963.
10. Cornell, A.S., Doorsamy, W., Fuks, B., Harmsen, G., and Mason, L. (2021, November 17). Boosted Decision Trees in the Era of New Physics: A Smuon Analysis Case Study 2021. Available online: https://arxiv.org/abs/2109.11815.
11. Searching for Exotic Particles in High-Energy Physics with Deep Learning10.1038/ncomms5308 ↗
12. Learning the latent structure of collider events10.1007/JHEP10(2020)206 ↗
13. Dahbi, S.E., Choma, J., Mellado, B., Mokgatitswane, G., Ruan, X., Lieberman, B., and Celik, T. (2021, November 17). Machine Learning Approach for the Search of Resonances with Topological Features at the Large Hadron Collider. Available online: https://arxiv.org/abs/2011.09863.
14. Search for the rare decays Bs0→μ+μ- and B0→μ+μ-10.1016/j.physletb.2011.04.031 ↗
15. Williams, M., Gligorov, V.V., Thomas, C., Dijkstra, H., Nardulli, J., and Spradlin, P. (2011). The HLT2 Topological Lines, CERN. Technical Report.
16. LHCb Topological Trigger Reoptimization10.1088/1742-6596/664/8/082025 ↗
17. Adam-Bourdarios, C., Cowan, G., Germain, C., Guyon, I., Kégl, B., and Rousseau, D. (2014, January 8–13). The Higgs Boson Machine Learning Challenge. Proceedings of the 2014 International Conference on High-Energy Physics and Machine Learning, JMLR.org, HEPML’14, Montreal, QC, Canada.
18. Aarrestad, T., van Beekveld, M., Bona, M., Boveia, A., Caron, S., Davies, J., De Simone, A., Doglioni, C., Duarte, J.M., and Farbin, A. (2021, November 17). The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider. Available online: https://arxiv.org/abs/2105.14027.
19. Kasieczka, G., Nachman, B., Shih, D., Amram, O., Andreassen, A., Benkendorfer, K., Bortolato, B., Brooijmans, G., Canelli, F., and Collins, J.H. (2021, November 17). The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics. Available online: https://arxiv.org/abs/2101.08320.10.1088/1361-6633/ac36b9 ↗
20. Hocker, A., Speckmayer, P., Stelzer, J., Therhaag, J., von Toerne, E., Voss, H., Backes, M., Carli, T., Cohen, O., and Christov, A. (2007). TMVA–Toolkit for Multivariate Data Analysis with ROOT: Users guide. TMVA–Toolkit for Multivariate Data Analysis, CERN. Available online: https://arxiv.org/abs/physics/0703039.
21. Brun, R., Rademakers, F., Canal, P., Naumann, A., Couet, O., Moneta, L., Vassilev, V., Linev, S., Piparo, D., and Ganis, G. (2019). Root-Project/Root: v6.18/02. Zenodo August.
22. Scikit-learn: Machine learning in Python
23. Géron, A. (2017). Hands-on Machine Learning with Scikit-Learn and TensorFlow Concepts, Tools, and Techniques to Build Intelligentsystems, O’Reilly Media.
24. Matplotlib: A 2D graphics environment10.1109/MCSE.2007.55 ↗
25. McKinney, W. (July, January 28). Data structures for statistical computing in python. Proceedings of the 9th Python in Science Conference, Austin, TX, USA.
26. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python10.1038/s41592-019-0686-2 ↗
27. Array programming with NumPy10.1038/s41586-020-2649-2 ↗
28. Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc.
29. Chollet, F. (2021, November 17). Keras Software. Available online: https://keras.io.
30. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2021, November 17). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: https://tensorflow.org.
31. Al-Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., and Belopolsky, A. (2016). Theano: A Python framework for fast computation of mathematical expressions. arXiv.
32. Seide, F., and Agarwal, A. (2016, January 13–17). CNTK: Microsoft’s Open-Source Deep-Learning Toolkit. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2945397 ↗
33. Neural networks and cellular automata in experimental high energy physics10.1016/0010-4655(88)90004-5 ↗
34. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
35. A Novel Artificial Intelligence Technique to Predict Compressive Strength of Recycled Aggregate Concrete Using ICA-XGBoost Model10.1007/s00366-020-01003-0 ↗
36. Qiu, Y., Zhou, J., Khandelwal, M., Yang, H., Yang, P., and Li, C. (2021). Performance evaluation of hybrid WOA-XGBoost, GWO-XGBoost and BO-XGBoost models to predict blast-induced ground vibration. Eng. Comput., 1–18.10.1007/s00366-021-01393-9 ↗
37. Hasani, R., Lechner, M., Amini, A., Rus, D., and Grosu, R. (2021, January 2–9). Liquid Time-constant Networks. Proceedings of the AAAI Conference on Artificial Intelligence (Virtual).10.1609/aaai.v35i9.16936 ↗
38. An introduction to quantum machine learning10.1080/00107514.2014.964942 ↗
39. The ATLAS Experiment at the CERN Large Hadron Collider
40. The CMS Experiment at the CERN LHC
41. An introduction to PYTHIA 8.210.1016/j.cpc.2015.01.024 ↗
42. Kazeev, N. Machine Learning for Particle Identification in the LHCb Detector. [Ph.D. Thesis, Sapienza—University of Rome]. 21 October 2020.10.1088/1742-6596/1525/1/012100 ↗
43. Buarque Franzosi, D., Cacciapaglia, G., Cid Vidal, X., Ferretti, G., Flacke, T., and Vázquez Sierra, C. (2021, November 17). Exploring New Possibilities to Discover a Light Pseudo-Scalar at LHCb. Available online: https://arxiv.org/abs/2106.12615.10.1140/epjc/s10052-021-09930-y ↗
44. Discovering True Muonium at LHCb10.1103/PhysRevD.100.053003 ↗
45. Hynds, D.P.M. Resolution Studies and Performance Evaluation of the LHCb VELO Upgrade. [Ph.D. Thesis, University of Glasgow]. 27 November 2014.
46. uBoost: A boosting method for producing uniform selection efficiencies from multivariate classifiers10.1088/1748-0221/8/12/P12013 ↗
47. New approaches for boosting to uniformity10.1088/1748-0221/10/03/T03002 ↗
48. Pivarski, J., Das, P., Smirnov, D., Burr, C., Feickert, M., Biederbeck, N., Smith, N., Rembser, J., Schreiner, H., and Dembinski, H. Scikit-Hep/Uproot: 3.10.7.
49. Dawe, N., Ongmongkolkul, P., Deil, C., Stark, G., Waller, P., Howard, J., and Babuschkin, I. Root_NUMPY: 4.2.0.
50. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach10.2307/2531595 ↗
51. Fast Implementation of DeLong’s Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves10.1109/LSP.2014.2337313 ↗
52. Boosted decision trees as an alternative to artificial neural networks for particle identification10.1016/j.nima.2004.12.018 ↗
53. Recent advances in deep learning10.1007/s13042-020-01096-5 ↗
54. Stanev, D., Riva, R., and Umassi, M. (2021, November 17). Deep Neural Network as an Alternative to Boosted Decision Trees for PID. Available online: https://arxiv.org/abs/2104.14045.
55. Alvestad, D., Fomin, N., Kersten, J., Maeland, S., and Strümke, I. (2021, November 17). Beyond Cuts in Small Signal Scenarios–Enhanced Sneutrino Detectability Using Machine Learning. Available online: https://arxiv.org/abs/2108.03125.
56. Tannenwald, B., Neu, C., Li, A., Buehlmann, G., Cuddeback, A., Hatfield, L., Parvatam, R., and Thompson, C. (2021, November 17). Benchmarking Machine Learning Techniques with Di-Higgs Production at the LHC. Available online: https://arxiv.org/abs/2009.06754.
57. Heredge, J., Hill, C., Hollenberg, L., and Sevior, M. (2021, November 17). Quantum Support Vector Machines for Continuum Suppression in B Meson Decays. Available online: https://arxiv.org/abs/2103.12257.10.1007/s41781-021-00075-x ↗
58. Event Classification with Quantum Machine Learning in High-Energy Physics10.1007/s41781-020-00047-7 ↗
59. Bendavid, J. (2021, November 17). Efficient Monte Carlo Integration Using Boosted Decision Trees and Generative Deep Neural Networks. Available online: https://arxiv.org/abs/1707.00028.
60. Strong, G.C. (2021, November 17). On the Impact of Selected Modern Deep-Learning Techniques to the Performance and Celerity of Classification Models in an Experimental High-Energy Physics Use Case. Available online: https://arxiv.org/abs/2002.01427.
61. Strong, G.C. (2021). GilesStrong/Lumin: v0.8.0–Mistake Not.
62. Hung, C.Y., Chen, W.C., Lai, P.T., Lin, C.H., and Lee, C.C. (2017, January 11–15). Comparing deep neural network and other machine learning algorithms for stroke prediction in a large-scale population-based electronic medical claims database. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, Korea.10.1109/EMBC.2017.8037515 ↗
63. Improving the Diagnosis of Liver Disease Using Multilayer Perceptron Neural Network and Boosted Decision Trees10.1007/s40846-017-0360-z ↗
64. Botchkarev, A. (2021, November 17). Evaluating Hospital Case Cost Prediction Models Using Azure Machine Learning Studio. CoRR, Available online: https://arxiv.org/abs/1804.01825.
65. Chen, C., Liu, Z., Zhou, J., Li, X., Qi, Y., Jiao, Y., and Zhong, X. (2021, November 17). How Much Can A Retailer Sell? Sales Forecasting on Tmall. CoRR, Available online: https://arxiv.org/abs/2002.11940.
66. Partin, A., Brettin, T., Evrard, Y.A., Zhu, Y., Yoo, H., Xia, F., Jiang, S., Clyde, A., Shukla, M., and Fonstein, M. (2021). Learning curves for drug response prediction in cancer cell lines. BMC Bioinform., 22.10.1186/s12859-021-04163-y ↗
67. Chen, Y., Chen, W., Pal, S.C., Saha, A., Chowdhuri, I., Adeli, B., Janizadeh, S., Dineva, A.A., Wang, X., and Mosavi, A. (2021). Evaluation efficiency of hybrid deep learning algorithms with neural network decision tree and boosting methods for predicting groundwater potential. Geocarto Int., 1–21.10.1080/10106049.2021.1920635 ↗
68. Machine learning and the physical sciences10.1103/RevModPhys.91.045002 ↗
69. Machine learning in medicine10.1056/NEJMra1814259 ↗
70. Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems10.1021/acs.chemrev.1c00107 ↗
71. Machine learning for predicting thermal transport properties of solids10.1016/j.mser.2021.100642 ↗
72. Machine learning in predicting mechanical behavior of additively manufactured parts10.1016/j.jmrt.2021.07.004 ↗
73. A Review of User Interface Design for Interactive Machine Learning10.1145/3185517 ↗