journal article·2023Open access

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

Chaoyu Fan

Computers, Materials & Continua· Volume 77· Issue 2

Abstract

There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them. However, the volume of internet data is beyond the processing capabilities of the current internet infrastructure. Therefore, engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia. The goal of this paper is to explore the entity relationship based on deep learning, introduce semantic knowledge by using the prepared language model, develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach (RoBERTa) and multi-task learning, and combine the intelligent characters in the field of linguistic, called Robustly Optimized BERT Approach + Multi-Task Learning (RoBERTa + MTL). To improve the effectiveness of model interaction, multi-task teaching is used to implement the observation information of auxiliary tasks. Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction, and a further it has achieved 86.35% of accuracy after being combined with multi-task learning.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Topic Modeling · Web Data Mining and Analysis · Data Quality and Management

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (27)

Show source-provided references
1. Using semantic roles to improve text classification in the requirements domain10.1007/s10579-017-9406-7 ↗
2. Data analytics for the identification of fake reviews using supervised learning10.32604/cmc.2022.019625 ↗
3. Integrating external dictionary knowledge in conference scenarios the field of personalized machine-translation method
4. Rough set-based approaches for discretization: A compact review10.1007/s10462-014-9426-2 ↗
5. Neural network acceptability judgments10.1162/tacl_a_00290 ↗
6. A brief review on multi-task learning10.1007/s11042-018-6463-x ↗
7. Gradient surgery for multi-task learning
8. Classifying relations via long short-term memory networks along shortest dependency paths
9. Long short-term memory projection recurrent neural network architectures for piano’s continuous note recognition10.1155/2017/2061827 ↗
10. ImprovedGCN: An efficient and accurate recommendation system employing lightweight graph convolutional networks in social media10.1016/j.elerap.2022.101191 ↗
11. Stronger, faster and more explainable: A graph convolutional baseline for skeleton-based action recognition
12. Efficiently identifying task groupings for multi-task learning
13. An overview of multi-task learning10.1093/nsr/nwx105 ↗
14. Adashare: Learning what to share for efficient deep multi-task learning
15. Federated multi-task learning under a mixture of distributions
16. Multi-task learning model based on multi-scale CNN and LSTM for sentiment classification10.1109/ACCESS.2020.2989428 ↗
17. A comparison of loss weighting strategies for multi-task learning in deep neural networks10.1109/ACCESS.2019.2943604 ↗
18. A. Waldis and L. Mazzola, “Nested and balanced entity recognition using multi-task learning,” arXiv preprint arXiv:2106.06216, 2021.
19. Entity relationship extraction based on a multi-neural network cooperation model10.3390/app13116812 ↗
20. A Chinese entity-relation extraction method via improved machine reading comprehension
21. Improving relation extraction using semantic role and multi-task learning
22. Relation extraction: Advancements through deep learning and entity-related features10.1007/s13278-023-01095-8 ↗
23. Span-based single-stage joint entity-relation extraction model10.1371/journal.pone.0281055 ↗
24. Position-aware attention and supervised data improve slot filling
25. I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó. Séaghdha et al., “Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals,” arXiv preprint, arXiv:1911.10422.
26. A bidirectional tree tagging scheme for joint medical relation extraction
27. https://figshare.com/articles/dataset/Wiki80/19323371 (accessed on 01/10/2022).