Chains-BERT: A High-Performance Semi-Supervised and Contrastive Learning-Based Automatic Question-and-Answering Model for Agricultural Scenarios
Source abstract
Abstract
Achieving automatic question-and-answering for agricultural scenarios based on machine reading comprehension can facilitate production staff to query information and process data efficiently. Nevertheless, when studying agricultural question-and-answer classification, there are barriers, such as small-scale corpus, narrow content range of corpus, or the need for manual annotation. In the context of such production needs, this paper proposed a text classification model based on text-relational chains and applied it to machine reading comprehension and open-ended question-and-answer tasks in agricultural scenarios. This paper modified the BERT network based on semi-supervised and contrastive learning to enhance the model’s performance. By incorporating the text-relational chains with the BERT network, the Chains-BERT model is constructed. Our efficient mode method outperformed other methods on the CAIL2018 dataset. Ultimately, we developed an automatic question-and-answering application to embed the contrastive-learning information aggregation model in this paper. The accuracy of the proposed model exceeded that of several contrasting mainstream models in many open-source datasets. In agricultural scenarios, the model has achieved state-of-the-art levels and is the best in efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Expert finding and Q&A systems · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
Funding
Natural Science Fund Project
References (33)
Show source-provided references
1. Zhang, Y., Wa, S., Sun, P., and Wang, Y. (2021). Pear Defect Detection Method Based on ResNet and DCGAN. Information, 12.10.3390/info12100397 ↗
2. Zhang, Y., Wa, S., Liu, Y., Zhou, X., Sun, P., and Ma, Q. (2021). High-Accuracy Detection of Maize Leaf Diseases CNN Based on Multi-Pathway Activation Function Module. Remote Sens., 13.10.3390/rs13214218 ↗
3. Automatic Plant Disease Detection Based on Tranvolution Detection Network with GAN Modules Using Leaf Images10.3389/fpls.2022.875693 ↗
4. Lin, X., Wa, S., Zhang, Y., and Ma, Q. (2022). A Dilated Segmentation Network with the Morphological Correction Method in Farming Area Image Series. Remote Sens., 14.10.3390/rs14081771 ↗
5. Huang, L., Bras, R.L., Bhagavatula, C., and Choi, Y. (2019). Cosmos QA: Machine reading comprehension with contextual commonsense reasoning. arXiv.10.18653/v1/D19-1243 ↗
6. Yang, A., Wang, Q., Liu, J., Liu, K., Lyu, Y., Wu, H., She, Q., and Li, S. (August, January 28). Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
7. Zhang, Z., Wu, Y., Zhou, J., Duan, S., Zhao, H., and Wang, R. (2020, January 7–12). SG-Net: Syntax-guided machine reading comprehension. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6511 ↗
8. Gao, S., Agarwal, S., Chung, T., Jin, D., and Hakkani-Tur, D. (2020). From machine reading comprehension to dialogue state tracking: Bridging the gap. arXiv.10.18653/v1/2020.nlp4convai-1.10 ↗
9. Tanaka, R., Nishida, K., and Yoshida, S. (2021, January 2–9). Visualmrc: Machine reading comprehension on document images. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i15.17635 ↗
10. Lehnert, W.G. (1977). The Process of Question Answering, Yale University.
11. Hirschman, L., Light, M., Breck, E., and Burger, J.D. (1999, January 20–26). Deep read: A reading comprehension system. Proceedings of the 37th annual meeting of the Association for Computational Linguistics, Stroudsburg, PA, USA.10.3115/1034678.1034731 ↗
12. Riloff, E., and Thelen, M. (2000, January 4). A rule-based question answering system for reading comprehension tests. Proceedings of the ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-Based Language Understanding Systems, Seattle, WA, USA.10.3115/1117595.1117598 ↗
13. Poon, H., Christensen, J., Domingos, P., Etzioni, O., Hoffmann, R., Kiddon, C., Lin, T., Ling, X., Ritter, A., and Schoenmackers, S. (2010, January 6). Machine reading at the university of washington. Proceedings of the NAACL HLT 2010 First International Workshop on Formalisms and Methodology for Learning by Reading, Los Angeles, CA, USA.
14. Richardson, M., Burges, C.J., and Renshaw, E. (2013, January 18–21). Mctest: A challenge dataset for the open-domain machine comprehension of text. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
15. Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P. (2015). Teaching machines to read and comprehend. Adv. Neural Inf. Process. Syst., 28.
16. Hill, F., Bordes, A., Chopra, S., and Weston, J. (2015). The goldilocks principle: Reading children’s books with explicit memory representations. arXiv.
17. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv.10.18653/v1/D16-1264 ↗
18. Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng, L. (2016, January 9). MS MARCO: A human generated machine reading comprehension dataset. Proceedings of the CoCo@ NIPs, Barcelona, Spain.
19. Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., and Suleman, K. (2016). Newsqa: A machine comprehension dataset. arXiv.10.18653/v1/W17-2623 ↗
20. The narrativeqa reading comprehension challenge10.1162/tacl_a_00023 ↗
21. Joshi, M., Choi, E., Weld, D.S., and Zettlemoyer, L. (2017). Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv.10.18653/v1/P17-1147 ↗
22. Lai, G., Xie, Q., Liu, H., Yang, Y., and Hovy, E. (2017). Race: Large-scale reading comprehension dataset from examinations. arXiv.10.18653/v1/D17-1082 ↗
23. Xie, Q., Lai, G., Dai, Z., and Hovy, E. (2017). Large-scale cloze test dataset designed by teachers. arXiv.10.18653/v1/D18-1257 ↗
24. Wang, S., and Jiang, J. (2016). Machine comprehension using match-lstm and answer pointer. arXiv.
25. Seo, M., Kembhavi, A., Farhadi, A., and Hajishirzi, H. (2016). Bidirectional attention flow for machine comprehension. arXiv.
26. Lyu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., and Le, Q.V. (2018). Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv.
27. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
28. “Cloze procedure”: A new tool for measuring readability
29. Paperno, D., Kruszewski, G., Lazaridou, A., Pham, Q.N., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., and Fernández, R. (2016). The LAMBADA dataset: Word prediction requiring a broad discourse context. arXiv.10.18653/v1/P16-1144 ↗
30. Šuster, S., and Daelemans, W. (2018). Clicr: A dataset of clinical case reports for machine reading comprehension. arXiv.10.18653/v1/N18-1140 ↗
31. Dunn, M., Sagun, L., Higgins, M., Guney, V.U., Cirik, V., and Cho, K. (2017). Searchqa: A new q&a dataset augmented with context from a search engine. arXiv.
32. He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H., and She, Q. (2017). Dureader: A chinese machine reading comprehension dataset from real-world applications. arXiv.10.18653/v1/W18-2605 ↗
33. HAR-Depth: A Novel Framework for Human Action Recognition Using Sequential Learning and Depth Estimated History Images10.1109/TETCI.2020.3014367 ↗