Developing ChemDFM as a large language foundation model for chemistry

1. Scientific discovery in the age of artificial intelligence10.1038/s41586-023-06221-2 ↗

2. Accelerated chemical science with AI10.1039/D3DD00213F ↗

3. Zhou · 2022

4. An adaptive graph learning method for automated molecular interactions and properties predictions10.1038/s42256-022-00501-8 ↗

5. Molformer: Motif-based transformer on 3d heterogeneous molecular graphs

6. GeoMFormer: A General Architecture for Geometric Molecular Representation Learning

7. GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation

8. Translation between Molecules and Natural Language

9. Deep generative models for ligand-based de novo design applied to multi-parametric optimization10.1002/jcc.26826 ↗

10. Domain-Agnostic Molecular Generation with Chemical Feedback

11. Machine learning-aided generative molecular design10.1038/s42256-024-00843-5 ↗

12. Xu P., Fu T., Gao W., Sun J. REINVENT-Transformer: Molecular De Novo Design through Transformer-based Reinforcement Learning. In KDD 2024 Workshop AIDSH; 2024. https://openreview.net/forum?id=XykiSFid41.

13. 3D Molecular Generation via Virtual Dynamics

14. Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction10.1021/acscentsci.9b00576 ↗

15. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy10.1039/C9SC05704H ↗

16. Retroprime: A diverse, plausible and transformer-based method for single-step retrosynthesis predictions10.1016/j.cej.2021.129845 ↗

17. Deep learning driven biosynthetic pathways navigation for natural products with BioNavi-NP10.1038/s41467-022-30970-9 ↗

18. Reaction performance prediction with an extrapolative and interpretable graph model based on chemical knowledge10.1038/s41467-023-39283-x ↗

19. Enantioselectivity prediction of pallada-electrocatalysed C–H activation using transition state knowledge in machine learning10.1038/s44160-022-00233-y ↗

20. Retrosynthesis prediction with an iterative string editing model10.1038/s41467-024-50617-1 ↗

21. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

22. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. Improving Language Understanding by Generative Pre-Training. 2018. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf

23. MolXPT: Wrapping Molecules with Text for Generative Pre-training

24. One Transformer Can Understand Both 2D & 3D Molecular Data

25. UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation

26. GLM: General Language Model Pretraining with Autoregressive Blank Infilling

27. LLaMA: Open and Efficient Foundation Language Models

28. Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data

29. Le Q.V. Finetuned Language Models Are Zero-Shot Learners

30. Chain-of-thought prompting elicits reasoning in large language models

31. Large language models are zero-shot reasoners

32. Scialom T. Toolformer: Language Models Can Teach Themselves to Use Tools

33. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

34. Unifying Molecular and Textual Representations via Multi-Task Language Modelling

35. Fang Y., Liang X., Zhang N., Liu K., Huang R., Chen Z., Fan X., Chen H. Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models. ICLR; 2024. https://openreview.net/forum?id=Tlsdsb6l9n.

36. InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery

37. Shaping the water-harvesting behavior of metal–organic frameworks aided by fine-tuned GPT models10.1021/jacs.3c12086 ↗

38. Large Language Models for Inorganic Synthesis Predictions10.1021/jacs.4c05840 ↗

39. Yu B., Baker F.N., Chen Z., Ning X., Sun H. Llasmol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset. COLM; 2024. https://arxiv.org/abs/2402.09391.

40. Prompt engineering of GPT-4 for chemical research: what can/cannot be done?

41. Moformer: self-supervised transformer model for metal–organic framework property prediction10.1021/jacs.2c11420 ↗

42. Autonomous chemical research with large language models10.1038/s41586-023-06792-0 ↗

43. Large language models for chemistry robotics10.1007/s10514-023-10136-2 ↗

44. Augmenting large language models with chemistry tools10.1038/s42256-024-00832-8 ↗

45. Ruan, Y., Lu, C., Xu, N., He, Y., Chen, Y,. Zhang, J., Xuan, J., Pan, J., and Fang, Q., Gao, H., Shen, X., Ye, N., et al., (2024). An automatic end-to-end chemical synthesis development platform powered by large language models. Nat. Commun. 2024;15:10160. https://doi.org/10.1038/s41467-024-54457-x10.1038/s41467-024-54457-x ↗

46. A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?

47. Large language models encode clinical knowledge10.1038/s41586-023-06291-2 ↗

48. PMC-LLaMA: Towards Building Open-source Language Models for Medicine10.1093/jamia/ocae045 ↗

49. ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge

50. DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs

51. EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education

52. DARWIN Series: Domain Specific Large Language Models for Natural Science

53. K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization

54. GPT-4 Technical Report

55. Llama 2: Open foundation and fine-tuned chat models

56. Galactica: A Large Language Model for Science

57. What Can Large Language Models Do in Chemistry? A Comprehensive Benchmark on Eight Tasks

58. ChemEval: A Comprehensive Multi-Level Chemical Evaluation for Large Language Models

59. Are large language models superhuman chemists?

60. SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research

61. SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models

62. Bleu: a method for automatic evaluation of machine translation

63. Lin C.-Y. Rouge: A package for automatic evaluation of summaries. Text summarization branches out; 2004. p. 74–81. https://aclanthology.org/W04-1013/.

64. Banerjee S., Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization; 2005. p. 65–72. https://aclanthology.org/W05-0909/.

65. Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries

66. Tanimoto · 1958

67. Ratcliff · 1988

68. G Landrum. (2023). RDKit: Open-Source Cheminformatics Software. https://www.rdkit.org/.

69. The Generation of a Unique Machine Description for Chemical Structures-A Technique Developed at Chemical Abstracts Service10.1021/c160017a018 ↗

70. Miller · 2009

71. MoleculeNet: a benchmark for molecular machine learning10.1039/C7SC02664A ↗

72. The use of the area under the ROC curve in the evaluation of machine learning algorithms10.1016/S0031-3203(96)00142-2 ↗

73. A Closer Look at AUROC and AUPRC under Class Imbalance

74. Ramsundar · 2019

75. The properties of known drugs. 1. Molecular frameworks10.1021/jm9602928 ↗

76. Mix-Key: graph mixup with key structures for molecular property prediction10.1093/bib/bbae165 ↗

77. Multi-Modal Representation Learning for Molecular Property Prediction: Sequence, Graph, Geometry

78. Predicting reaction performance in C–N cross-coupling using machine learning10.1126/science.aar5169 ↗

79. Suzuki–Miyaura cross-coupling optimization enabled by automated feedback10.1039/C6RE00153J ↗

80. Jin, W., Coley, C., Barzilay, R., and Jaakkola, T. (2017). Predicting organic reaction outcomes with weisfeiler-lehman network. https://proceedings.neurips.cc/paper_files/paper/2017/file/ced556cd9f9c0c8315cfbe0744a3baf0-Paper.pdf.

81. What’s what: The (nearly) definitive guide to reaction role assignment10.1021/acs.jcim.6b00564 ↗

82. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow10.1126/science.aap9112 ↗

83. Uncertainty-aware prediction of chemical reaction yields with graph neural networks10.1186/s13321-021-00579-z ↗

84. Chemformer: a pre-trained transformer for computational chemistry10.1088/2632-2153/ac3ffb ↗

85. Root-aligned SMILES: a tight representation for chemical reaction prediction10.1039/D2SC02763A ↗

86. Deep Retrosynthetic Reaction Prediction using Local Reactivity and Global Attention10.1021/jacsau.1c00246 ↗

87. G2Retro as a two-step graph generative models for retrosynthesis prediction10.1038/s42004-023-00897-3 ↗

88. Total Syntheses of Polycyclic Diterpenes Phomopsene, Methyl Phomopsenonate, and iso-Phomopsene via Reorganization of C–C Single Bonds10.1021/jacs.3c07044 ↗

89. MeSesamol, a bio-based and versatile polar aprotic solvent for organic synthesis and depolymerization10.1016/j.cej.2023.144365 ↗

90. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

91. LibreTexts. (n.d.). (2023, Mar 1). Home - Chemistry LibreTexts. LibreTexts. https://chem.libretexts.org/.

92. 2023

93. Wudaocorpora: A super large-scale chinese corpora for pre-training language models10.1016/j.aiopen.2021.06.001 ↗

94. Applied Deep Learning Research team at NVIDIA (2023, Mar 1). deepspeedai/Megatron-DeepSpeed GitHub. https://github.com/deepspeedai/Megatron-DeepSpeed?tab=readme-ov-file.59.

95. Think you have solved question answering? try arc, the ai2 reasoning challenge

96. Piqa: Reasoning about physical commonsense in natural language

97. Crowdsourcing multiple choice science questions

98. Measuring Massive Multitask Language Understanding

99. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules10.1021/ci00057a005 ↗

100. Lowe · 2012