Chinese Named Entity Recognition of Geological News Based on BERT Model
Source abstract
Abstract
With the ongoing progress of geological survey work and the continuous accumulation of geological data, extracting accurate information from massive geological data has become increasingly difficult. To fully mine and utilize geological data, this study proposes a geological news named entity recognition (GNNER) method based on the bidirectional encoder representations from transformers (BERT) pre-trained language model. This solves the problems of traditional word vectors that are difficult to represent context semantics and the single extraction effect and can also help construct the knowledge graphs of geological news. First, the method uses the BERT pre-training model to embed words in the geological news text, and the dynamically obtained word vector is used as the model’s input. Second, the word vector is sent to a bidirectional long short-term memory model for further training to obtain contextual features. Finally, the corresponding six entity types are extracted using conditional random field sequence decoding. Through experiments on the constructed Chinese geological news dataset, the average F1 score identified by the model is 0.839. The experimental results show that the model can better identify news entities in geological news.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Web Data Mining and Analysis · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, China University of Geosciences, Beijing 100083, China
Funding
China University of Geosciences (Beijing)
References (24)
Show source-provided references
1. Artificial intelligence and sustainable development
2. Khurana, D., Koli, A., Khatter, K., and Singh, S. (2017). Natural language processing: State of the art, current trends and challenges. arXiv.
3. Shaalan, K., and Raza, H. (2008, January 25–27). Arabic named entity recognition from diverse text types. Proceedings of the International Conference on Natural Language Processing, Gothenburg, Sweden.10.1007/978-3-540-85287-2_42 ↗
4. Malay named entity recognition based on rule-based approach10.7763/IJMLC.2014.V4.428 ↗
5. NERA: Named entity recognition for Arabic10.1002/asi.21090 ↗
6. Todorovic, B.T., Rancic, S.R., Markovic, I.M., Mulalic, E.H., and Ilic, V.M. (2008, January 25–27). Named entity recognition and classification using context Hidden Markov Model. Proceedings of the 2008 9th Symposium on Neural Network Applications in Electrical Engineering, Belgrade, Serbia.10.1109/NEUREL.2008.4685557 ↗
7. What is a hidden Markov model?10.1038/nbt1004-1315 ↗
8. Och, F.J., and Ney, H. (2002, January 7–12). Discriminative training and maximum entropy models for statistical machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073133 ↗
9. Ratnaparkhi, A. (2017). Maximum Entropy Models for Natural Language Processing. Encyclopedia of Machine Learning and Data Mining, Springer.10.1007/978-1-4899-7687-1_525 ↗
10. Lafferty, J., McCallum, A., and Pereira, F.C. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning, San Francisco, CA, USA.
11. Geological entity recognition method based on deep belief network
12. Research on coal mine safety knowledge graph construction and intelligent query method
13. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
14. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.J.a.p.a. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
15. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv.
16. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv.10.18653/v1/N16-1030 ↗
17. A personal history of the Viterbi algorithm10.1109/MSP.2006.1657823 ↗
18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems 30, Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017, Curran Associates Inc.
19. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
20. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
21. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
22. Yang, J., Zhang, Y., Li, L., and Li, X. (2017). YEDDA: A lightweight collaborative text span annotation tool. arXiv.10.18653/v1/P18-4006 ↗
23. Joint extraction of named entities and relations from Chinese rock description text based on BERT-BiLSTM-CRF Model
24. Chinese entity recognition based on BERT-BiLSTM-CRF model