Similarity Calculation via Passage-Level Event Connection Graph
Source abstract
Abstract
Recently, many information processing applications appear on the web on the demand of user requirement. Since text is one of the most popular data formats across the web, how to measure text similarity becomes the key challenge to many web applications. Web text is often used to record events, especially for news. One text often mentions multiple events, while only the core event decides its main topic. This core event should take the important position when measuring text similarity. For this reason, this paper constructs a passage-level event connection graph to model the relations among events mentioned in one text. This graph is composed of many subgraphs formed by triggers and arguments extracted sentence by sentence. The subgraphs are connected via the overlapping arguments. In term of centrality measurement, the core event can be revealed from the graph and utilized to measure text similarity. Moreover, two improvements based on vector tunning are provided to better model the relations among events. One is to find the triggers which are semantically similar. By linking them in the event connection graph, the graph can cover the relations among events more comprehensively. The other is to apply graph embedding to integrate the global information carried by the entire event connection graph into the core event to let text similarity be partially guided by the full-text content. As shown by experimental results, after measuring text similarity from a passage-level event representation perspective, our calculation acquires superior results than unsupervised methods and even comparable results with some supervised neuron-based methods. In addition, our calculation is unsupervised and can be applied in many domains free from the preparation of training data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Advanced Graph Neural Networks · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
State Key Laboratory of Communication Content Cognition, People’s Daily Online, Beijing 100733, China
Harbin Institute of Technology, School of Computer Science and Technology, Harbin 150001, China
Peng Cheng Laboratory, Shenzhen 518055, China
International Business and Management Research Center, Beijing Normal University, Zhuhai 519087, China
Funding
National Key Research and Development Project · Project of State Key Laboratory of Communication Content Cognition · National Science Foundation of China · Shenzhen Foundational Research Funding
References (56)
Show source-provided references
1. Ji, H., and Grishman, R. (2008, January 16–18). Refining event extraction through cross-document inference. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Columbus, OH, USA.
2. Baker, C., Fillmore, C., and Lowe, J. (1998, January 10–14). The berkeley framenet project. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Montreal, QC, Canada.10.3115/980845.980860 ↗
3. Jacksi, K., Ibrahim, R., Zeebaree, S., Zebari, R., and Sadeeq, M. (2020, January 23–24). Clustering documents based on semantic similarity using HAC and K-mean algorithms. Proceedings of the 2020 International Conference on Advanced Science and Engineering, Duhok, Iraq.10.1109/ICOASE51841.2020.9436570 ↗
4. Huang, X., Qi, J., Sun, Y., and Zhang, R. (2020, January 7–12). Mala: Cross-domain dialogue generation with action learning. Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6306 ↗
5. Kieu, B., Unanue, I., Pham, S., Phan, H., and Piccardi, M. (2021, January 10–15). Learning neural textual representations for citation recommendation. Proceedings of the 25th International Conference on Pattern Recognition, Milan, Italy.10.1109/ICPR48806.2021.9412725 ↗
6. Chen, Y., Zhou, M., and Wang, S. (2006, January 17–21). Reranking answers for definitional QA using language modeling. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia.10.3115/1220175.1220311 ↗
7. Turian, J., Ratinov, L., and Bengio, Y. (2010, January 11–16). Word representations: A simple and general method for semi-supervised learning. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden.
8. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
9. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013, January 2–4). Efficient estimation of word representations in vector space. Proceedings of the 1st International Conference on Learning Representations, Scottsdale, AZ, USA.
10. Pennington, J., Socher, R., and Manning, C. (2014, January 25–29). GloVe: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1162 ↗
11. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018, January 1–6). Deep contextualized word representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.10.18653/v1/N18-1202 ↗
12. Chen, J., Dai, X., Yuan, Q., Lu, C., and Huang, H. (2020, January 5–10). Towards interpretable clinical diagnosis with Bayesian network ensembles stacked on entity-aware CNNs. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.286 ↗
13. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25–29). Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1179 ↗
14. Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data10.1016/j.ipm.2020.102435 ↗
15. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
16. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2021, December 31). Improving Language Understanding by Generative Pre-Training. Available online: https://S3-Us-West-2.Amazonaws.Com.
17. Devlin, J., Chang, W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
18. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, V. (2019, January 8–14). XLNET: Generalized autoregressive pretraining for language understanding. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
19. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv.
20. Pan, Y., Yao, T., Li, Y., and Mei, T. (2020, January 14–19). X-Linear attention networks for image captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01098 ↗
21. Dual self-attention with co-attention networks for visual question answering10.1016/j.patcog.2021.107956 ↗
22. An enhanced Support Vector Machine classification framework by using Euclidean distance function for text document categorization10.1007/s10489-011-0314-z ↗
23. Zhu, H., Zhang, P., and Gao, Z. (2018, January 6–8). K-means text dynamic clustering algorithm based on KL divergence. Proceedings of the 17th IEEE/ACIS International Conference on Computer and Information Science, Singapore.
24. Huang, A. (2008, January 14–18). Similarity measures for text document clustering. Proceedings of the 6th New Zealand Computer Science Research Student Conference, Hamilton, New Zealand.
25. Efficient Hybrid Semantic Text Similarity using Wordnet and a Corpus
26. A survey of text similarity approaches
27. Understanding inverse document frequency: On theoretical arguments for IDF10.1108/00220410410560582 ↗
28. Mihalcea, R., and Tarau, P. (2004, January 25–26). Textrank: Bringing order into text. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain.
29. Latent dirichlet allocation
30. Text classification method based on self-training and LDA topic models10.1016/j.eswa.2017.03.020 ↗
31. Improvement of building field association term dictionary using passage retrieval10.1016/j.ipm.2006.12.006 ↗
32. Extraction, selection and ranking of Field Association (FA) Terms from domain-specific corpora for building a comprehensive FA terms dictionary10.1007/s10115-010-0296-x ↗
33. Malkiel, I., Ginzburg, D., Barkan, O., Caciularu, A., Weill, J., and Koenigstein, N. (2022, January 25–29). Interpreting BERT-based Text Similarity via Activation and Saliency Maps. Proceedings of the 2022 ACM Web Conference, Lyon, France.10.1145/3485447.3512045 ↗
34. Grishman, R., and Sundheim, B. (1996, January 5–9). Message understanding conference-6: A brief history. Proceedings of the 16th International Conference on Computational Linguistics, Copenhagen, Denmark.10.3115/992628.992709 ↗
35. Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., and Weischedel, R. (2004, January 24–30). The Automatic Content Extraction (ACE) program-tasks, data, and evaluation. Proceedings of the 4th International Conference on Language Resources and Evaluation, Centro Cultural de Belem, Lisbon.
36. Yang, S., Feng, D., Qiao, L., Kan, Z., and Li, D. (August, January 28). Exploring pre-trained language models for event extraction and generation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
37. Wang, Z., Wang, X., Han, X., Lin, Y., Hou, L., Liu, Z., Li, P., Li, J., and Zhou, J. (2021, January 1–6). CLEVE: Contrastive pre-training for event extraction. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online.10.18653/v1/2021.acl-long.491 ↗
38. Du, X., and Cardie, C. (2020, January 5–10). Document-Level event role filler extraction using multi-granularity contextualized encoding. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Washington, DC, USA.10.18653/v1/2020.acl-main.714 ↗
39. Hu, Z., Liu, M., Wu, Y., Xu, J., Qin, B., and Li, J. (2020, January 14–18). Document-level event subject pair recognition. Proceedings of the 9th CCF International Conference on Natural Language Processing and Chinese Computing, Zhengzhou, China.10.1007/978-3-030-60450-9_23 ↗
40. Lin, Y., Ji, H., Huang, F., and Wu, L. (2020, January 5–10). A joint neural model for information extraction with global features. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Washington, DC, USA.10.18653/v1/2020.acl-main.713 ↗
41. Liao, S., and Grishman, R. (2010, January 11–16). Using document level cross-event inference to improve event extraction. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden.
42. Automatic building of new Field Association word candidates using search engine10.1016/j.ipm.2005.08.006 ↗
43. Finding communities in directed networks by PageRank random walk induced network embedding10.1016/j.physa.2010.02.014 ↗
44. Li, P., Zhou, G., Zhu, Q., and Hou, L. (2012, January 12–14). Employing compositional semantics and discourse consistency in Chinese event extraction. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea.
45. Amir, H., and Béatrice, D. (2018, January 7–12). Word embedding approach for synonym extraction of multi-word terms. Proceedings of the Eleventh International Conference on Language Resources and Evaluation, Miyazaki, Japan.
46. Knowledge management: Semantic drift or conceptual shift?
47. Grover, A., and Leskovec, J. (2016, January 13–17). Node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939754 ↗
48. Sasano, R., and Korhonen, A. (2020, January 5–10). Investigating word-class distributions in word vector spaces. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.337 ↗
49. Kennington, C. (2021, January 10–11). Enriching language models with visually-grounded word vectors and the lancaster sensorimotor norms. Proceedings of the 25th Conference on Computational Natural Language Learning, Online.10.18653/v1/2021.conll-1.11 ↗
50. Mysore, S., Cohan, A., and Hope, T. (2022, January 10–15). Multi-vector models with textual guidance for fine-grained scientific document similarity. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, Washington, DC, USA.10.18653/v1/2022.naacl-main.331 ↗
51. Sentence similarity based on contexts10.1162/tacl_a_00477 ↗
52. Lee, S., Lee, D., Jang, S., and Yu, H. (2022, January 22–27). Toward interpretable semantic textual similarity via optimal transport-based contrastive sentence learning. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland.10.18653/v1/2022.acl-long.412 ↗
53. Li, Q., Ji, H., and Huang, L. (2013, January 4–9). Joint event extraction via structured prediction with global features. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria.
54. Zhang, J., Qin, Y., Zhang, Y., Liu, M., and Ji, D. (2019, January 10–16). Extracting entities and events as a single task using a transition-based neural model. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/753 ↗
55. Liu, X., Huang, H., and Zhang, Y. (August, January 28). Open domain event extraction using neural latent variable models. Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Italy.
56. Levy, O., Søgaard, A., and Goldberg, Y. (2017, January 3–7). A strong baseline for learning cross-lingual word embeddings from sentence alignments. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain.10.18653/v1/E17-1072 ↗