Automatic Construction of Fine-Grained Paraphrase Corpora System Using Language Inference Model
Source abstract
Abstract
Paraphrase detection and generation are important natural language processing (NLP) tasks. Yet the term paraphrase is broad enough to include many fine-grained relations. This leads to different tolerance levels of semantic divergence in the positive paraphrase class among publicly available paraphrase datasets. Such variation can affect the generalisability of paraphrase classification models. It may also impact the predictability of paraphrase generation models. This paper presents a new model which can use few corpora of fine-grained paraphrase relations to construct automatically using language inference models. The fine-grained sentence level paraphrase relations are defined based on word and phrase level counterparts. We demonstrate that the fine-grained labels from our proposed system can make it possible to generate paraphrases at desirable semantic level. The new labels could also contribute to general sentence embedding techniques.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, Faculty of Engineering, Darlington Campus, The University of Sydney, Darlington, NSW 2008, Australia
References (24)
Show source-provided references
1. Dolan, W.B., and Brockett, C. (2005, January 14). Automatically constructing a corpus of sentential paraphrases. Proceedings of the Third International Workshop on Paraphrasing (IWP2005), Jeju Island, Korea.
2. De Beaugrande, R.A., and Dressler, W.U. (1981). Introduction to Text Linguistics, Longman.10.4324/9781315835839 ↗
3. Shinyama, Y., Sekine, S., Sudo, K., and Grishman, R. (, 2002). Automatic paraphrase acquisition from news articles. Proceedings of the HLT, San Diego, CA, USA.10.3115/1289189.1289218 ↗
4. Barzilay, R., and Lee, L. (2003). Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. arXiv.10.3115/1073445.1073448 ↗
5. What is a paraphrase?10.1162/COLI_a_00166 ↗
6. Qiu, L., Kan, M.Y., and Chua, T.S. (2006, January 22–23). Paraphrase recognition via dissimilarity significance classification. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia.10.3115/1610075.1610079 ↗
7. Introduction to the special issue on summarization10.1162/089120102762671927 ↗
8. Elhadad, N., and Sutaria, K. (2007, January 29). Mining a lexicon of technical terms and lay equivalents. Proceedings of the Biological, Translational, and Clinical Language Processing, Prague, Czech Republic.10.3115/1572392.1572402 ↗
9. Tomuro, N. (2003, January 11). Interrogative reformulation patterns and acquisition of question paraphrases. Proceedings of the Second International Workshop on Paraphrasing, Sapporo, Japan.10.3115/1118984.1118989 ↗
10. Identification of sentence-to-sentence relations using a textual entailer10.1007/s11168-009-9065-y ↗
11. Is this a paraphrase? What kind? Paraphrase boundaries and typology10.4236/ojml.2014.41016 ↗
12. Iyer, S., Dandekar, N., and Csernai, K. (2021, November 20). First Quora Dataset Release: Question Pairs. Available online: https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs.
13. Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., and Specia, L. (2017). Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv.10.18653/v1/S17-2001 ↗
14. Pavlick, E., Rastogi, P., Ganitkevitch, J., Van Durme, B., and Callison-Burch, C. (2015, January 26–31). PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Beijing, China.10.3115/v1/P15-2070 ↗
15. Bhagat, R., Pantel, P., and Hovy, E. (2007, January 28–30). LEDIR: An unsupervised algorithm for learning directionality of inference rules. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic.
16. Pavlick, E., Bos, J., Nissim, M., Beller, C., Van Durme, B., and Callison-Burch, C. (2015, January 26–31). Adding semantics to data-driven paraphrasing. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Beijing, China.10.3115/v1/P15-1146 ↗
17. MacCartney, B. (2009). Natural Language Inference. [Ph.D. Thesis, Stanford University].
18. Bowman, S.R., Angeli, G., Potts, C., and Manning, C.D. (2015). A large annotated corpus for learning natural language inference. arXiv.10.18653/v1/D15-1075 ↗
19. Williams, A., Nangia, N., and Bowman, S.R. (2017). A broad-coverage challenge corpus for sentence understanding through inference. arXiv.10.18653/v1/N18-1101 ↗
20. Anaphora and coreference resolution: A review10.1016/j.inffus.2020.01.010 ↗
21. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
22. Lin, C.Y., and Och, F.J. (2004, January 21–26). Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), Barcelona, Spain.10.3115/1218955.1219032 ↗
23. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P.J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv.
24. Loshchilov, I., and Hutter, F. (2019). Decoupled Weight Decay Regularization. arXiv.