journal article·2023-03-29Open access

PREFMoDeL: A Systematic Review and Proposed Taxonomy of Biomolecular Features for Deep Learning

Jacob L. North, Victor L. Hsu

Applied Sciences· Volume 13· Issue 7

Abstract

Of fundamental importance in biochemical and biomedical research is understanding a molecule’s biological properties—its structure, its function(s), and its activity(ies). To this end, computational methods in Artificial Intelligence, in particular Deep Learning (DL), have been applied to further biomolecular understanding—from analysis and prediction of protein–protein and protein–ligand interactions to drug discovery and design. While choosing the most appropriate DL architecture is vitally important to accurately model the task at hand, equally important is choosing the features used as input to represent molecular properties in these DL models. Through hypothesis testing, bioinformaticians have created thousands of engineered features for biomolecules such as proteins and their ligands. Herein we present an organizational taxonomy for biomolecular features extracted from 808 articles from across the scientific literature. This objective view of biomolecular features can reduce various forms of experimental and/or investigator bias and additionally facilitate feature selection in biomolecular analysis and design tasks. The resulting dataset contains 1360 nondeduplicated features, and a sample of these features were classified by their properties, clustered, and used to suggest new features. The complete feature dataset (the Public Repository of Engineered Features for Molecular Deep Learning, PREFMoDeL) is released for collaborative sourcing on the web.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Computational Drug Discovery Methods · Machine Learning in Materials Science · Protein Structure and Dynamics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Biochemistry, University of Washington, Seattle, WA 98195, USA

Institute for Protein Design, University of Washington, Seattle, WA 98195, USA

Graduate Program in Biological Physics, Structure and Design, University of Washington, Seattle, WA 98195, USA

Department of Biochemistry and Biophysics, Oregon State University, Corvallis, OR 97331-7305, USA

Funding

NIGMS · OSU/CoS Summer Undergraduate Research Experience (SURE) Award

References (91)

Show source-provided references
1. Drug Discovery: A Historical Perspective10.1126/science.287.5460.1960 ↗
2. Phenotypic drug discovery: Recent successes, lessons learned and new directions10.1038/s41573-022-00472-w ↗
3. Machine Learning in Drug Discovery: A Review10.1007/s10462-021-10058-4 ↗
4. Rethinking drug design in the artificial intelligence era10.1038/s41573-019-0050-3 ↗
5. Long, A.a.W., Nayler, J.H.C., Smith, H., Taylor, T., and Ward, N. (1971). Derivatives of 6-aminopenicillanic acid. Part XI. α-Amino-p-hydroxybenzylpenicillin. J. Chem. Soc. Org., 1920–1922.10.1039/J39710001920 ↗
6. Accurate prediction of protein structures and interactions using a three-track neural network10.1126/science.abj8754 ↗
7. Protein structure predictions to atomic accuracy with AlphaFold10.1038/s41592-021-01362-6 ↗
8. Robust deep learning based protein sequence design using ProteinMPNN10.1126/science.add2187 ↗
9. Adjuvanting a subunit COVID-19 vaccine to induce protective immunity10.1038/s41586-021-03530-2 ↗
10. Predicting the Reliability of Drug-target Interaction Predictions with Maximum Coverage of Target Space10.1038/s41598-017-04264-w ↗
11. Cerisier, N., Petitjean, M., Regad, L., Bayard, Q., Réau, M., Badel, A., and Camproux, A.C. (2019). High Impact: The Role of Promiscuous Binding Sites in Polypharmacology. Molecules, 24.10.3390/molecules24142529 ↗
12. Prediction of Promiscuity Cliffs Using Machine Learning10.1002/minf.202000196 ↗
13. Machine learning reveals that structural features distinguishing promiscuous and non-promiscuous compounds depend on target combinations10.1038/s41598-021-87042-z ↗
14. Promiscuous Ligands from Experimentally Determined Structures, Binding Conformations, and Protein Family-Dependent Interaction Hotspots10.1021/acsomega.8b03481 ↗
15. A review of molecular representation in the age of machine learning10.1002/wcms.1603 ↗
16. Scientific intuition inspired by machine learning-generated hypotheses10.1088/2632-2153/abda08 ↗
17. Recent advances in convolutional neural networks10.1016/j.patcog.2017.10.013 ↗
18. A Comprehensive Survey on Graph Neural Networks10.1109/TNNLS.2020.2978386 ↗
19. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
20. Advanced graph and sequence neural networks for molecular property prediction and drug discovery10.1093/bioinformatics/btac112 ↗
21. Geometry-enhanced molecular representation learning for property prediction10.1038/s42256-021-00438-4 ↗
22. A Comprehensive Survey of Loss Functions in Machine Learning10.1007/s40745-020-00253-5 ↗
23. Ciampiconi, L., Elwood, A., Leonardi, M., Mohamed, A., and Rozza, A. (2023). A survey and taxonomy of loss functions in machine learning. arXiv.
24. Chauvin, Y., and Rumelhart, D.E. (1995). Backpropagation: Theory, Architectures, and Applications, Psychology Press.
25. Backpropagation and the brain10.1038/s41583-020-0277-3 ↗
26. Abdolrasol, M.G.M., Hussain, S.M.S., Ustun, T.S., Sarker, M.R., Hannan, M.A., Mohamed, R., Ali, J.A., Mekhilef, S., and Milad, A. (2021). Artificial Neural Networks Based Optimization Techniques: A Review. Electronics, 10.10.3390/electronics10212689 ↗
27. Differentiable biology: Using deep learning for biophysics-based and data-driven modeling of molecular mechanisms10.1038/s41592-021-01283-4 ↗
28. König, G., Molnar, C., Bischl, B., and Grosse-Wentrup, M. (2021, January 10–15). Relative Feature Importance. Proceedings of the 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9413090 ↗
29. A comprehensive survey on feature selection in the various fields of machine learning10.1007/s10489-021-02550-9 ↗
30. Feature Selection: A Review and Comparative Study10.1051/e3sconf/202235101046 ↗
31. Machine Learning for Molecular Simulation10.1146/annurev-physchem-042018-052331 ↗
32. Learning to Make Chemical Predictions: The Interplay of Feature Representation, Data, and Machine Learning Methods10.1016/j.chempr.2020.05.014 ↗
33. Chemist versus Machine: Traditional Knowledge versus Machine Learning Techniques10.1016/j.trechm.2020.10.007 ↗
34. Feature Selection: A Literature Review10.6029/smartcr.2014.03.007 ↗
35. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration10.1136/bmj.b2700 ↗
36. Orange: Data Mining Toolbox in Python
37. Stochastic Neighbor Embedding
38. Visualizing Data using t-SNE
39. Miller, B.K., Geiger, M., Smidt, T.E., and Noé, F. (2020). Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties. arXiv.
40. Fuchs, F.B., Worrall, D.E., Fischer, V., and Welling, M. (2020, January 6–12). SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
41. The Protein Data Bank10.1093/nar/28.1.235 ↗
42. PDB-wide Collection of Binding Data: Current Status of the PDBbind Database10.1093/bioinformatics/btu626 ↗
43. Experimental Protein Molecular Dynamics: Broadband Dielectric Spectroscopy coupled with nanoconfinement10.1038/s41598-019-54562-8 ↗
44. Pradeepkiran, J., and Reddy, P. (2019). Structure Based Design and Molecular Docking Studies for Phosphorylated Tau Inhibitors in Alzheimer’s Disease. Cells, 8.10.3390/cells8030260 ↗
45. Fout, A., Byrd, J., Shariat, B., and Ben-Hur, A. (2017). NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, MIT Press.
46. Persistent spectral–based machine learning (PerSpect ML) for protein-ligand binding affinity prediction10.1126/sciadv.abc5329 ↗
47. Cyclophilin A stabilizes the HIV-1 capsid through a novel non-canonical binding site10.1038/ncomms10714 ↗
48. Beyond Rotatable Bond Counts: Capturing 3D Conformational Flexibility in a Single Descriptor10.1021/acs.jcim.6b00565 ↗
49. Effect of Tris, MOPS, and phosphate buffers on the hydrolysis of polyethylene terephthalate films by polyester hydrolases10.1002/2211-5463.12097 ↗
50. A theory of implicit and explicit knowledge10.1017/S0140525X99002186 ↗
51. Implicit and explicit categorization: A tale of four species10.1016/j.neubiorev.2012.09.003 ↗
52. Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., and Sun, G. (2018, January 19–23). xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3220023 ↗
53. Comparative Assessment of Scoring Functions: The CASF-2016 Update10.1021/acs.jcim.8b00545 ↗
54. A review of mathematical representations of biomolecular data10.1039/C9CP06554G ↗
55. Machine learning approaches for analyzing and enhancing molecular dynamics simulations10.1016/j.sbi.2019.12.016 ↗
56. TorchMD: A Deep Learning Framework for Molecular Simulations10.1021/acs.jctc.0c01343 ↗
57. Choosing the right molecular machine learning potential10.1039/D1SC03564A ↗
58. Structural Reorganization and Preorganization in Enzyme Active Sites: Comparisons of Experimental and Theoretically Ideal Active Site Geometries in the Multistep Serine Esterase Reaction Cycle10.1021/ja803213p ↗
59. Strain analysis of protein structures and low dimensionality of mechanical allosteric couplings10.1073/pnas.1609462113 ↗
60. Intrinsic dynamics of an enzyme underlies catalysis10.1038/nature04105 ↗
61. Promoting Vibrations and the Function of Enzymes. Emerging Theoretical and Experimental Convergence10.1021/acs.biochem.8b00201 ↗
62. Energy Bilocalization Effect and the Emergence of Molecular Functions in Proteins10.3389/fmolb.2021.736376 ↗
63. Oscillatory Active-site Motions Correlate with Kinetic Isotope Effects in Formate Dehydrogenase10.1021/acscatal.9b03345 ↗
64. Machine Learning Identifies Chemical Characteristics That Promote Enzyme Catalysis10.1021/jacs.8b13879 ↗
65. Dynamics of Substrate Processing by PPIP5K2, a Versatile Catalytic Machine10.1016/j.str.2019.03.007 ↗
66. ProDy 2.0: Increased Scale and Scope after 10 Years of Protein Dynamics Modelling with Python10.1093/bioinformatics/btab187 ↗
67. Molecular contrastive learning of representations via graph neural networks10.1038/s42256-022-00447-x ↗
68. Importance of Engineered and Learned Molecular Representations in Predicting Organic Reactivity, Selectivity, and Chemical Properties10.1021/acs.accounts.0c00745 ↗
69. Ramsundar, B., Eastman, P., Walters, P., Pande, V., Leswing, K., and Wu, Z. (2019). Deep Learning for the Life Sciences, O’Reilly Media.
70. Jamasb, A.R., Lió, P., and Blundell, T.L. (2020). Graphein—a Python Library for Geometric Deep Learning and Network Analysis on Protein Structures. bioRxiv.10.1101/2020.07.15.204701 ↗
71. Approaches to Multi-Objective Feature Selection: A Systematic Literature Review10.1109/ACCESS.2020.3007291 ↗
72. A multi-objective optimization algorithm for feature selection problems10.1007/s00366-021-01369-9 ↗
73. Selecting critical features for data classification based on machine learning methods10.1186/s40537-020-00327-4 ↗
74. Zhu, G., Xu, Z., Guo, X., Yuan, C., and Huang, Y. (2021). DIFER: Differentiable Automated Feature Engineering. arXiv.
75. Gada, M., Haria, Z., Mankad, A., Damania, K., and Sankhe, S. (2021, January 19–20). Automated Feature Engineering and Hyperparameter Optimization for Machine Learning. Proceedings of the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.10.1109/ICACCS51430.2021.9441668 ↗
76. FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches10.1109/TVCG.2022.3141040 ↗
77. MDTraj: A Modern Open Library for the Analysis of Molecular Dynamics Trajectories10.1016/j.bpj.2015.08.015 ↗
78. MSMBuilder2: Modeling Conformational Dynamics on the Picosecond to Millisecond Scale10.1021/ct200463m ↗
79. PyEMMA 2: A Software Package for Estimation, Validation, and Analysis of Markov Models10.1021/acs.jctc.5b00743 ↗
80. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. arXiv.10.1109/CVPR52688.2022.01042 ↗
81. OpenAI (2023, February 04). ChatGPT: Optimizing Language Models for Dialogue. Available online: https://openai.com/blog/chatgpt/.
82. Single-sequence protein structure prediction using a language model and deep learning10.1038/s41587-022-01432-w ↗
84. CryoDRGN: Reconstruction of heterogeneous cryo-EM structures using neural networks10.1038/s41592-020-01049-4 ↗
85. EMPIAR: The Electron Microscopy Public Image Archive10.1093/nar/gkac1062 ↗
86. Jamali, K., Kimanius, D., and Scheres, S. (2022). ModelAngelo: Automated Model Building in Cryo-EM Maps. arXiv.
87. EMDataBank unified data resource for 3DEM10.1093/nar/gkv1126 ↗
88. BioMagResBank10.1093/nar/gkm957 ↗
89. SASBDB, a repository for biological small-angle scattering data10.1093/nar/gku1047 ↗
90. Mechanism and Catalytic Site Atlas (M-CSA): A database of enzyme reaction mechanisms and active sites10.1093/nar/gkx1012 ↗
91. ProtaBank: A repository for protein design and engineering data10.1002/pro.3585 ↗