MIRA-ChatGLM: A Fine-Tuned Large Language Model for Intelligent Risk Assessment in Coal Mining
Source abstract
Abstract
Intelligent mining risk assessment (MIRA) is a vital approach for enhancing safety and operational efficiency in mining. In this study, we introduce MIRA-ChatGLM, which leverages pre-trained large language models (LLMs) for the domain of gas risk assessment in coal mines. We meticulously constructed a dataset specifically designed for mining risk analysis and performed parameter-efficient fine-tuning on the locally deployed GLM-4-9B-chat base model to develop MIRA-ChatGLM. By utilizing consumer-grade GPUs and employing LoRA and various levels of quantization algorithms such as QLoRA, we investigated the impact of different data scales and instruction settings on model performance. The evaluation results show that MIRA-ChatGLM achieved excellent performance with BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L scores of 84.47, 90.63, 86.88, and 90.63, respectively, highlighting its outstanding performance in coal mine gas risk assessment. Through comparative experiments with other large language models of similar size and manual evaluation, MIRA-ChatGLM demonstrated superior performance across multiple key metrics, fully demonstrating its tremendous potential in intelligent mine risk assessment and decision support.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Oil and Gas Production Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Communication and Information Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
References (42)
Show source-provided references
1. Li, X., Cao, Z., and Xu, Y. (2021). Characteristics and trends of coal mine safety development. Energy Sources Part A Recover. Util. Environ. Eff., 1–19.10.1080/15567036.2020.1852339 ↗
2. Risk assessment of gas explosion in coal mines based on fuzzy AHP and bayesian network10.1016/j.psep.2020.01.003 ↗
3. Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., and Dong, Z. (2023). A survey of large language models. arXiv.
4. Research on method of coal mine gas and coal dust explosion perception alarm and explosion source judgment
5. Safety measurements and risk assessment of coal mining industry using artificial intelligence and machine learning10.4236/ojbm.2021.93064 ↗
6. Risk identification for coal and gas outburst in underground coal mines: A critical review and future directions10.1016/j.jgsce.2023.205106 ↗
7. Research on coal mine hidden danger analysis and risk early warning technology based on data mining in China10.1016/j.psep.2022.12.077 ↗
8. Hybrid CNN-LSTM and IoT-based coal mine hazards monitoring and prediction system10.1016/j.psep.2021.06.005 ↗
9. Big data monitoring and early warning cloud platform for coal mine gas disaster risk and potential danger and its application
10. A decision system based on intelligent perception and decision for scene ventilation safety
11. A comprehensive risk assessment method for coal and gas outburst in underground coal mines based on variable weight theory and uncertainty analysis10.1016/j.psep.2022.08.065 ↗
12. Risk assessment of coal mine gas explosion based on cloud integrated similarity and fuzzy DEMATEL10.1016/j.psep.2023.07.043 ↗
13. Applications of artificial intelligence for coal mine gas risk assessment10.1016/j.ssci.2021.105420 ↗
14. Risk assessment of dynamic disasters in deep coal mines based on multi-source, multi-parameter indexes, and engineering application10.1016/j.psep.2021.09.034 ↗
15. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv.
16. Exploring the potential of large language models (llms) in learning on graphs10.1145/3655103.3655110 ↗
17. Training language models to follow instructions with human feedback
18. Liu, C., Sun, K., Zhou, Q., Duan, Y., Shu, J., Kan, H., Gu, Z., and Hu, J. (2024). CPMI-ChatGLM: Parameter-Efficient Fine-Tuning of ChatGLM with Chinese Patent Medicine Instructions. Sci. Rep., 14.10.1038/s41598-024-56874-w ↗
19. FinBERT: A large language model for extracting information from financial text10.1111/1911-3846.12832 ↗
20. ChatGPT for good? On opportunities and challenges of large language models for education10.1016/j.lindif.2023.102274 ↗
21. Using large language models in psychology
22. Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., and Liao, K.-D. (2024, January 3–8). A survey on multimodal large language models for autonomous driving. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACVW60836.2024.00106 ↗
23. Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.-Y.K., Li, Z., and Zhao, H. (2024). Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robot. Autom. Lett.10.1109/LRA.2024.3440097 ↗
24. GLMT, Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., and Zhao, H. (2024). ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv.
25. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning
26. Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9–15). Parameter-efficient transfer learning for NLP. Proceedings of the International Conference on Machine Learning, PMLR 2019, Long Beach, CA, USA.
27. Li, X.L., and Liang, P. (2021). Prefix-tuning: Optimizing continuous prompts for generation. arXiv.10.18653/v1/2021.acl-long.353 ↗
28. Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., and Tang, J. (2023). GPT Understands, too, AI Open.10.1016/j.aiopen.2023.08.012 ↗
29. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv.
30. Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2024). Qlora: Efficient finetuning of quantized llms. arXiv.
31. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002, January 6–12). Bleu: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
32. Lin, C.Y. (2004). Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out, Association for Computational Linguistics.
33. Urbizu, G., San Vicente, I., Saralegi, X., and Corral, A. (2023, January 9–14). Not Enough Data to Pre-train Your Language Model? MT to the Rescue!. Proceedings of the Findings of the Association for Computational Linguistics, Toronto, ON, Canada.10.18653/v1/2023.findings-acl.235 ↗
34. Diao, S., Xu, R., Su, H., Jiang, Y., Song, Y., and Zhang, T. (2021, January 1–6). Taming pre-trained language models with n-gram representations for low-resource domain adaptation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Virtual Event.10.18653/v1/2021.acl-long.259 ↗
35. Edwards, A., Camacho-Collados, J., De Ribaupierre, H., and Preece, A. (2020, January 8–13). Go simple and pre-train on domain-specific corpora: On the role of training data for text classification. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.481 ↗
36. Bird, S. (2006, January 17–18). NLTK: The natural language toolkit. Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, Sydney, Australia.10.3115/1225403.1225421 ↗
37. Zheng, Y., Zhang, R., Zhang, J., Ye, Y., Luo, Z., Feng, Z., and Ma, Y. (2024). Llamafactory: Unified Efficient Fine-Tuning of 100+ Language Models. arXiv.10.18653/v1/2024.acl-demos.38 ↗
38. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
39. Le Scao, T., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A.S., Yvon, F., and Gallé, M. (2023). Bloom: A 176b-parameter open-access multilingual language model. arXiv.
40. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv.
41. Yang, A., Xiao, B., Wang, B., Zhang, B., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., and Yang, F. (2023). Baichuan 2: Open large-scale language models. arXiv.
42. Wang, H., Liu, C., Xi, N., Qiang, Z., Zhao, S., Qin, B., and Liu, T. (2023). Huatuo: Tuning llama model with chinese medical knowledge. arXiv.