journal article·2021-11-19Open access

A Novel Deep Neural Network-Based Approach to Measure Scholarly Research Dissemination Using Citations Network

Naif Radi Aljohani, Ayman Fayoumi, Saeed-Ul Hassan

Applied Sciences· Volume 11· Issue 22

Abstract

We investigated the scientific research dissemination by analyzing the publications and citation data, implying that not all citations are significantly important. Therefore, as alluded to existing state-of-the-art models that employ feature-based techniques to measure the scholarly research dissemination between multiple entities, our model implements the convolutional neural network (CNN) with fastText-based pre-trained embedding vectors, utilizes only the citation context as its input to distinguish between important and non-important citations. Moreover, we speculate using focal-loss and class weight methods to address the inherited class imbalance problems in citation classification datasets. Using a dataset of 10 K annotated citation contexts, we achieved an accuracy of 90.7% along with a 90.6% f1-score, in the case of binary classification. Finally, we present a case study to measure the comprehensiveness of our deployed model on a dataset of 3100 K citations taken from the ACL Anthology Reference Corpus. We employed state-of-the-art graph visualization open-source tool Gephi to analyze the various aspects of citation network graphs, for each respective citation behavior.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Complex Network Analysis Techniques · Advanced Text Analysis Techniques · Data Visualization and Analytics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia

Department of Computing and Mathematics, Manchester Metropolitan University, Manchester M15 6BH, UK

Funding

King Abdulaziz University

References (62)

Show source-provided references
1. A decade of in-text citation analysis based on natural language processing and machine learning techniques: An overview of empirical studies10.1007/s11192-021-04055-1 ↗
2. Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics10.1007/s11192-020-03613-3 ↗
3. DGSD: Distributed graph representation via graph statistical properties10.1016/j.future.2021.02.005 ↗
4. Arshad, N., Bakar, A., Soroya, S.H., Safder, I., Haider, S., Hassan, S.U., Aljohani, N.R., Alelyani, S., and Nawaz, R. (2019). Extracting scientific trends by mining topics from Call for Papers. Libr. Hi Tech.10.1108/LHT-02-2019-0048 ↗
5. Mining network-level properties of Twitter altmetrics data10.1007/s11192-019-03112-0 ↗
6. Imran, M., Akhtar, A., Said, A., Safder, I., Hassan, S.U., and Aljohani, N.R. (2018). Exploiting social networks of Twitter in altmetrics big data. STI 2018 Conference Proceedings, Centre for Science and Technology Studies (CWTS).
7. Hassan, S.U., Aljohani, N.R., Tarar, U.I., Safder, I., Sarwar, R., Alelyani, S., and Nawaz, R. (2020). Exploiting Tweet Sentiments in Altmetrics Large-Scale Data. arXiv.
8. An in-text citation classification predictive model for a scholarly search system10.1007/s11192-021-03986-z ↗
9. Hassan, S.U., Saleem, A., Soroya, S.H., Safder, I., Iqbal, S., Jamil, S., Bukhari, F., Aljohani, N.R., and Nawaz, R. (2020). Sentiment analysis of tweets through Altmetrics: A machine learning approach. J. Inf. Sci.10.1177/0165551520930917 ↗
10. Influential tweeters in relation to highly cited articles in altmetric big data10.1007/s11192-019-03044-9 ↗
11. Agarwal, N., Haque, E., Liu, H., and Parsons, L. (2005, January 11–13). Research paper recommender systems: A subspace clustering approach. Proceedings of the International Conference on Web-Age Information Management, Hangzhou, China.10.1007/11563952_42 ↗
12. Gori, M., and Pucci, A. (2006, January 18–22). Research paper recommender systems: A random-walk based approach. Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2006 Main Conference Proceedings) (WI’06), Hong Kong, China.10.1109/WI.2006.149 ↗
13. Küçüktunç, O., Saule, E., Kaya, K., and Çatalyürek, Ü.V. (2012). Recommendation on academic networks using direction aware citation analysis. arXiv.
14. cite2vec: Citation-driven document exploration via word embeddings10.1109/TVCG.2016.2598667 ↗
15. Ganguly, S., and Pudi, V. (2017). Paper2vec: Combining graph and text information for scientific paper representation. European Conference on Information Retrieval, Springer.10.1007/978-3-319-56608-5_30 ↗
16. Ebesu, T., and Fang, Y. (2017, January 7). Neural citation network for context-aware citation recommendation. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan.10.1145/3077136.3080730 ↗
17. Big data service architecture: A survey
18. R2N: A novel deep learning architecture for rain removal from single image
19. Blockchain-based systems and applications: A survey
20. A fast two-stage black-box deep learning network attacking method based on cross-correlation
21. Effective crude oil price forecasting using new text-based and big-data-driven model10.1016/j.measurement.2020.108468 ↗
22. Measuring the evolution of a scientific field through citation frames10.1162/tacl_a_00028 ↗
23. Teufel, S., Siddharthan, A., and Tidhar, D. (2006, January 22–23). Automatic classification of citation function. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia.10.3115/1610075.1610091 ↗
24. Leveraging deep learning and SNA approaches for smart city policing in the developing world10.1016/j.ijinfomgt.2019.102045 ↗
25. Bot prediction on social networks of Twitter in altmetrics using deep graph convolutional networks10.1007/s00500-020-04689-y ↗
26. Forecasting the US oil markets based on social media information during the COVID-19 pandemic10.1016/j.energy.2021.120403 ↗
27. Research on the automatic extraction method of web data objects based on deep learning10.32604/iasc.2020.013939 ↗
28. Research on privacy disclosure detection method in social networks based on multi-dimensional deep learning
29. A novel scene text recognition method based on deep learning
30. Review of text classification methods on deep learning
31. Liu, Y., Niculescu-Mizil, A., and Gryc, W. (2009, January 14–18). Topic-link LDA: Joint models of topic and author community. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.10.1145/1553374.1553460 ↗
32. Extended co-citation search: Graph-based document retrieval on a co-citation network containing citation context information10.1016/j.ipm.2019.05.007 ↗
33. Chang, J., and Blei, D. (2009). Relational topic models for document networks. Artificial Intelligence and Statistics, PMLR.10.1214/09-AOAS309 ↗
34. A LSTM based model for personalized context-aware citation recommendation10.1109/ACCESS.2018.2872730 ↗
35. Visualizing the context of citations referencing papers published by Eugene Garfield: A new type of keyword co-occurrence analysis10.1007/s11192-017-2591-8 ↗
36. Meng, F., Gao, D., Li, W., Sun, X., and Hou, Y. (November, January 27). A unified graph model for personalized query-oriented reference paper recommendation. Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, San Francisco, CA, USA.
37. VOPRec: Vector representation learning of papers with text information and structural identity for recommendation10.1109/TETC.2018.2830698 ↗
38. A knowledge-aware attentional reasoning network for recommendation
39. A theoretical model of scientific impact based on citations
40. Relative citation impact of various study designs in the health sciences10.1001/jama.293.19.2362 ↗
41. Understanding the impact change of a highly cited article: A content-based citation analysis10.1007/s11192-017-2398-7 ↗
42. Discovery of knowledge flow in science10.1145/1125944.1125948 ↗
43. Characterizing scientific production and consumption in physics
44. Rowlands, I. (2002). Journal diffusion factors: A new approach to measuring research influence. Aslib Proceedings, MCB UP Ltd.10.1108/00012530210435211 ↗
45. Frandsen, T.F. (2004). Journal Diffusion Factors–a measure of diffusion?. Aslib Proceedings, Emerald Group Publishing Limited.10.1108/00012530410516822 ↗
46. Measuring international knowledge flows and scholarly impact of scientific research10.1007/s11192-012-0786-6 ↗
47. A comparison between the China Scientific and Technical Papers and Citations Database and the Science Citation Index in terms of journal hierarchies and interjournal citation relations10.1002/asi.20475 ↗
48. Analyzing knowledge flows of scientific literature through semantic links: A case study in the field of energy10.1007/s11192-015-1528-3 ↗
49. Human behavior analysis in the production and consumption of scientific knowledge across regions: A case study on publications in Scopus10.1108/LHT-04-2017-0076 ↗
50. Hassan, S.U., Akram, A., Asghar, A., and Aljohani, N.R. (2017). Measuring Scientific Knowledge Flows by Deploying Citation Context Analysis using Machine Learning Approach on PLoS ONE Full Text, In ISSI.
51. Aljohani, N.R., Fayoumi, A., and Hassan, S.U. (2021). A novel focal-loss and class-weight-aware convolutional neural network for the classification of in-text citations. J. Inf. Sci.10.1177/0165551521991022 ↗
52. ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis10.1016/j.future.2020.08.005 ↗
53. Rahi, S., Safder, I., Iqbal, S., Hassan, S.U., Reid, I., and Nawaz, R. (2019, January 26–28). Citation classification using natural language processing and machine learning models. Proceedings of the International Conference on Smart Information & Communication Technologies, Saidia, Morocco.10.1007/978-3-030-53187-4_39 ↗
54. Sagnika, S., Mishra, B.S., and Meher, S.K. (2021). An attention-based CNN-LSTM model for subjectivity detection in opinion-mining. Neural Comput. Appl., 1–4.10.1007/s00521-021-06328-5 ↗
55. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, Curran Associates, Inc.
56. Bastian, M., Heymann, S., and Jacomy, M. (2009, January 17–20). Gephi: An open source software for exploring and manipulating networks. Proceedings of the Third international AAAI Conference on Weblogs and Social Media, San Jose, CA, USA.10.1609/icwsm.v3i1.13937 ↗
57. Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities10.1103/PhysRevE.80.016118 ↗
58. Fast unfolding of communities in large networks10.1088/1742-5468/2008/10/P10008 ↗
59. Depth-first search and linear graph algorithms10.1137/0201010 ↗
60. Main-memory triangle computations for very large (sparse (power-law)) graphs10.1016/j.tcs.2008.07.017 ↗
61. A faster algorithm for betweenness centrality10.1080/0022250X.2001.9990249 ↗
62. Some unique properties of eigenvector centrality10.1016/j.socnet.2007.04.002 ↗