Word Sense Disambiguation Using Clustered Sense Labels
Source abstract
Abstract
Sequence labeling models for word sense disambiguation have proven highly effective when the sense vocabulary is compressed based on the thesaurus hierarchy. In this paper, we propose a method for compressing the sense vocabulary without using a thesaurus. For this, sense definitions in a dictionary are converted into sentence vectors and clustered into the compressed senses. First, the very large set of sense vectors is partitioned for less computational complexity, and then it is clustered hierarchically with awareness of homographs. The experiment was done on the English Senseval and Semeval datasets and the Korean Sejong sense annotated corpus. This process demonstrated that the performance greatly increased compared to that of the uncompressed sense model and is comparable to that of the thesaurus-based model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Semantic Web and Ontologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Chungbuk National University, Cheongju 28644, Korea
Funding
National Research Foundation of Korea
References (48)
Show source-provided references
1. Word sense disambiguation: A survey10.1145/1459352.1459355 ↗
2. Word sense disambiguation: An overview10.1111/j.1749-818X.2009.00131.x ↗
3. Word sense disambiguation: An empirical survey
4. Approaches for word sense disambiguation—A survey
5. Zhong, Z., and Ng, H.T. (2010, January 11–16). It Makes Sense: A Wide-Coverage Word Sense Disambiguation System for Free Text. Proceedings of the ACL 2010 System Demonstrations, Association for Computational Linguistics, Uppsala, Sweden.
6. Iacobacci, I., Pilehvar, M.T., and Navigli, R. (2016, January 7–12). Embeddings for word sense disambiguation: An evaluation study. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. Long Papers.10.18653/v1/P16-1085 ↗
7. Agirre, E., Martínez, D., De Lacalle, O.L., and Soroa, A. (2006, January 22–23). Two graph-based algorithms for state-of-the-art WSD. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia.10.3115/1610075.1610157 ↗
8. Clustering and diversifying web search results with graph-based word sense induction10.1162/COLI_a_00148 ↗
9. Lesk, M.E. (1986, January 8–11). Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. Proceedings of the 5th Annual International Conference on Systems Documentation SIGDOC ’86, Toronto, ON, Canada.10.1145/318723.318728 ↗
10. Basile, P., Caputo, A., and Semeraro, G. (2014, January 23–29). An Enhanced Lesk Word Sense Disambiguation Algorithm through a Distributional Semantic Model. Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics, Dublin, Ireland. Technical Papers.
11. Trends in word sense disambiguation10.1007/s10462-012-9331-5 ↗
12. Neural network models for word sense disambiguation: An overview
13. Loureiro, D., Rezaee, K., Pilehvar, M.T., and Camacho-Collados, J. (2020). Language models and word sense disambiguation: An overview and analysis. arXiv.
14. Bevilacqua, M., Pasini, T., Raganato, A., and Navigli, R. (2021, January 19–27). Recent trends in word sense disambiguation: A survey. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Montreal, QC, Canada.10.24963/ijcai.2021/593 ↗
15. Huang, L., Sun, C., Qiu, X., and Huang, X. (2019). GlossBERT: BERT for word sense disambiguation with gloss knowledge. arXiv.10.18653/v1/D19-1355 ↗
16. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
17. Kumar, S., Jat, S., Saxena, K., and Talukdar, P. (August, January 28). Zero-shot word sense disambiguation using sense definition embeddings. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
18. Vial, L., Lecouteux, B., and Schwab, D. (2019). Sense vocabulary compression through the semantic knowledge of wordnet for neural word sense disambiguation. arXiv.
19. (2020, May 13). Global WordNet Association. Available online: http://globalwordnet.org.
20. Raganato, A., Bovi, C.D., and Navigli, R. (2017, January 7–11). Neural sequence learning models for word sense disambiguation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1120 ↗
21. Cer, D., Yang, Y., Kong, S.Y., Hua, N., Limtiaco, N., John, R.S., Constant, N., Guajardo-Céspedes, M., Yuan, S., and Tar, C. (2018). Universal sentence encoder. arXiv.10.18653/v1/D18-2029 ↗
22. Manning, C.D., Raghavan, P., and Schütze, H. (2008). Introduction to Information Retrieval, Cambridge University Press.10.1017/CBO9780511809071 ↗
23. Hierarchical Model-Based Clustering for Large Datasets10.1198/106186001317115072 ↗
24. Clustering by passing messages between data points10.1126/science.1136800 ↗
25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
26. Bjerva, J. (2017, January 23–24). Will my auxiliary tagging task help? estimating auxiliary tasks effectivity in multitask learning. Proceedings of the 21st Nordic Conference on Computational Linguistics, Gothenburg, Sweden.
27. (2020, April 11). Scikit-Learn. Available online: https://scikit-learn.org.
28. (2021, October 27). BERT-Base-Cased. Available online: https://github.com/huggingface/transformers.
29. (2019, June 10). KorBERT. Available online: http://aiopen.etri.re.kr.
30. Fellbaum, C. (2010). WordNet. Theory and Applications of Ontology: Computer applications, Springer.10.1007/978-90-481-8847-5_10 ↗
31. (2020, March 19). Standard Korean Language Dictionary. Available online: https://stdict.korean.go.kr/.
32. The National Institute of the Korean Language (2011). 21st Century Sejong Project Final Result, Revised Edition.
33. Raganato, A., Camacho-Collados, J., and Navigli, R. (2017, January 3–7). Word sense disambiguation: A unified evaluation framework and empirical comparison. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain. Long Papers.10.18653/v1/E17-1010 ↗
34. Statistical morphological disambiguation for agglutinative languages10.1023/A:1020271707826 ↗
35. A morphological tagger for Korean: Statistical tagging combined with corpus-based morphological rule application10.1007/s10590-004-7693-4 ↗
36. The National Institute of the Korean Language (2020). The Construction of a Disambiguated Tagged Corpus.
37. (2021, March 02). Urimalsaem Dictionary. Available online: https://opendict.korean.go.kr/.
38. Choi, K.S., Bae, H.S., Kang, W., Lee, J., Kim, E., Kim, H., Kim, D., Song, Y., and Shin, H. (2004, January 26–28). Korean-Chinese-Japanese Multilingual Wordnet with Shared Semantic Hierarchy. Proceedings of the LREC2004, Lisbon, Portugal.
39. Korean WordNet, KorLex 2.0-A Language Resource for Semantic Processing and Knowledge Engineering10.22557/HG.2012.03.295.163 ↗
40. Bae, Y.j., and Ock, C. (2014, January 18–20). Introduction to the Korean word map (UWordMap) and API. Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology, Chuncheon, Korea.
41. Lee, C.K., and Lee, G.B. (1999, January 8–9). Using WordNet for the automatic construction of Korean thesaurus. Proceedings of the 11th Annual Conference on Human and Cognitive Language Technology, Jeonju, Korea.
42. Vu, V.H., Nguyen, Q.P., Shin, J.C., and Ock, C.Y. (2020). UPC: An Open Word-Sense Annotated Parallel Corpora for Machine Translation Study. Appl. Sci., 10.10.3390/app10113904 ↗
43. On Convergence Properties of the EM Algorithm for Gaussian Mixtures10.1162/neco.1996.8.1.129 ↗
44. A density-based algorithm for discovering clusters in large spatial databases with noise
45. Märzinger, T., Kotík, J., and Pfeifer, C. (2021). Application of Hierarchical Agglomerative Clustering (HAC) for Systemic Classification of Pop-Up Housing (PUH) Environments. Appl. Sci., 11.10.3390/app112311122 ↗
46. Dinh, D.T., Fujinami, T., and Huynh, V.N. (December, January 29). Estimating the optimal number of clusters in categorical data clustering by silhouette coefficient. Proceedings of the International Symposium on Knowledge and Systems Sciences, Da Nang, Vietnam.
47. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv.
48. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compositionality. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada.