MBHAN: Motif-Based Heterogeneous Graph Attention Network
Source abstract
Abstract
Graph neural networks are graph-based deep learning technologies that have attracted significant attention from researchers because of their powerful performance. Heterogeneous graph-based graph neural networks focus on the heterogeneity of the nodes and links in a graph. This is more effective at preserving semantic knowledge when representing data interactions in real-world graph structures. Unfortunately, most heterogeneous graph neural networks tend to transform heterogeneous graphs into homogeneous graphs when using meta-paths for representation learning. This paper therefore presents a novel motif-based hierarchical heterogeneous graph attention network algorithm, MBHAN, that addresses this problem by incorporating a hierarchical dual attention mechanism at the node-level and motif-level. Node-level attention aims to learn the importance between a node and its neighboring nodes within its corresponding motif. Motif-level attention is capable of learning the importance of different motifs in the heterogeneous graph. In view of the different vector space features of different types of nodes in heterogeneous graphs, MBHAN also aggregates the features of different types of nodes, so that they can jointly participate in downstream tasks after passing through segregated independent shallow neural networks. MBHAN’s superior network representation learning capability has been validated by extensive experiments on two real-world datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Graph Theory and Algorithms · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Media and Communications, Guizhou Normal University, Guiyang 550000, China
School of Informatics, Xiamen University, Xiamen 361000, China
School of mathematical Science, Guizhou Normal University, Guiyang 550000, China
Funding
Guizhou Normal University 2021 Provincial Teaching Content and Curriculum Reform Foundation
References (51)
Show source-provided references
1. Cascaded Parsing of Human-Object Interaction Recognition10.1109/TPAMI.2021.3049156 ↗
2. Group-Wise Learning for Weakly Supervised Semantic Segmentation10.1109/TIP.2021.3132834 ↗
3. Li, Y., Zemel, R., Brockschmidt, M., and Tarlow, D. (2016, January 2–4). Gated Graph Sequence Neural Networks. Proceedings of the ICLR’16, San Juan, Puerto Rico.
4. The Graph Neural Network Model10.1109/TNN.2008.2005605 ↗
5. Gori, M., Monfardini, G., and Scarselli, F. (August, January 31). A New Model for Learning in Graph Domains. Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada.
6. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den, B.R., Titov, I., and Welling, M. (2018). Modeling Relational Data with Graph Convolutional Networks, Springer.10.1007/978-3-319-93417-4_38 ↗
7. Defferrard, M., Bresson, X., and Vandergheynst, P. (2016, January 5–10). Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain.
8. Hamilton, W.L., Ying, R., and Leskovec, J. (2017, January 4–9). Inductive Representation Learning on Large Graphs. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
9. Kipf, T., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks. arXiv.
10. Ashish, V., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
11. Dzmitry, B., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv.
12. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
13. Zhixing, T., Wang, M., Xie, J., Chen, Y., and Shi, X. (2017, January 4–9). Deep Semantic Role Labeling with Self-Attention. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.
14. Patrick, V., Strubell, E., and McCallum, A. (2018). Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction. arXiv.
15. Zichao, Y., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E.H. (2016, January 12–17). Hierarchical Attention Networks for Document Classification. Proceedings of the NAACL, San Diego, CA, USA.
16. Petar, V., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2018). Graph Attention Networks. arXiv.
17. Zhouhan, L., Feng, M., dos Santos, C.N., Yu, M., Xiang, B., Zhou, B., and Bengio, Y. (2017). A Structured Self-Attentive Sentence Embedding. arXiv.
18. Yizhou, Z., Xiong, Y., Kong, X., Li, S., and Zhu, Y. (2018, January 23–27). Deep Collective Classification in Heterogeneous Information Networks. Proceedings of the 2018 World Wide Web Conference, Lyon, France.
19. Xiao, W., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., and Yu, P.S. (2019). Heterogeneous Graph Attention Network. The World Wide Web Conference, Association for Computing Machinery.
20. Shaohua, F., Zhu, J., Han, X., Shi, C., Hu, L., Ma, B., and Li, Y. (2019, January 4–8). Metapath-Guided Heterogeneous Graph Neural Network for Intent Recommendation. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA.
21. Seongjun, Y., Jeong, M., Kim, R., Kang, J., and Kim, H.J. (2019, January 8–14). Graph Transformer Networks. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
22. Carranza, A.G., Rossi, R.A., Rao, A., and Koh, E. (2020, January 6–10). Higher-Order Clustering in Complex Heterogeneous Networks. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA.10.1145/3394486.3403045 ↗
23. Yang, C., Liu, M., Zheng, V.W., and Han, J. (2018, January 28–31). Node, Motif and Subgraph: Leveraging Network Functional Blocks through Structural Convolution. Proceedings of the 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), Barcelona, Spain.10.1109/ASONAM.2018.8508729 ↗
24. Mbrep: Motif-Based Representation Learning in Heterogeneous Networks
25. Dareddy, M.R., Das, M., and Yang, H. (2019, January 9–12). Motif2vec: Motif Aware Node Representation Learning for Heterogeneous Networks. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9005670 ↗
26. Li, Y., Lou, Z., Shi, Y., and Han, J. (2018, January 20). Temporal Motifs in Heterogeneous Information Networks. Proceedings of the 14th International Workshop on Mining and Learning with Graphs (MLG), London, UK.
27. Network Motifs: Simple Building Blocks of Complex Networks10.1126/science.298.5594.824 ↗
28. Lei, X.U., Huang, L., and Wang, C. (2019). Motif-Preserving Network Representation Learning. J. Front. Comput. Sci. Technol.
29. Yu, Y., Lu, Z., Liu, J., Zhao, G., and Wen, J. (2019, January 8–11). Rum: Network Representation Learning Using Motifs. Proceedings of the 2019 IEEE 35th International Conference on Data Engineering (ICDE), Macau, China.10.1109/ICDE.2019.00125 ↗
30. Boaz, L.J., Rossi, R.A., Kong, X., Kim, S., Koh, E., and Rao, A. (2019, January 3–7). Graph Convolutional Networks with Motif-Based Attention. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.
31. Hao, P., Li, J., Gong, Q., Wang, S., Ning, Y., and Yu, P.S. (2019). Graph Convolutional Neural Networks Via Motif-Based Attention. arXiv.
32. A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications10.1109/TKDE.2018.2807452 ↗
33. A Survey of Deep Learning Techniques for Weed Detection from Images10.1016/j.compag.2021.106067 ↗
34. Deep Learning-Based Text Classification: A Comprehensive Review
35. Chuxu, Z., Song, D., Huang, C., Swami, A., and Chawla, N.V. (2019, January 4–8). Heterogeneous Graph Neural Network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.
36. Ziniu, H., Dong, Y., Wang, K., and Sun, Y. (2020, January 20–24). Heterogeneous Graph Transformer. Proceedings of the Web Conference 2020, Taipei, Taiwan.
37. Higher-Order Organization of Complex Networks10.1126/science.aad9029 ↗
38. Dawei, Z., Zhang, S., Yildirim, M.Y., Alcorn, S., Tong, H., Davulcu, H., and He, J. (2017, January 13–17). A Local Algorithm for Structure-Preserving Graph Cut. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.
39. Hao, Y., Benson, A.R., Leskovec, J., and Gleich, D.F. (2017, January 13–17). Local Higher-Order Graph Clustering. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.
40. Le-kui, Z., Yang, Y., Ren, X., Wu, F., and Zhuang, Y. (2018, January 2–7). Dynamic Network Embedding by Modeling Triadic Closure Process. Proceedings of the AAAI, Hilton, New Orleans.
41. Distributed Representations of Words and Phrases and Their Compositionality
42. Jiawei, H. (2009, January 3–5). Mining Heterogeneous Information Networks by Exploring the Power of Links. Proceedings of the 20th International Conference on Algorithmic Learning Theory, Porto, Portugal.10.1007/978-3-642-04414-4_3 ↗
43. Bryan, P., Al-Rfou, R., and Skiena, S. (2014, January 24–27). Deepwalk: Online Learning of Social Representations. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, New York, NY, USA.
44. Yuxiao, D., Chawla, N.V., and Swami, A. (2017, January 13–17). Metapath2vec: Scalable Representation Learning for Heterogeneous Networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.
45. Aditya, G., and Leskovec, J. (2017, January 13–17). Node2vec: Scalable Feature Learning for Networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.
46. Meng, C., Ma, X., Xu, M., and Wang, C. (2019). Heterogeneous Information Network Embedding with Meta-Path Based Graph Attention Networks, Springer.
47. Ming, J., Sun, Y., Danilevsky, M., Han, J., and Gao, J. (2010, January 20–24). Graph Regularized Transductive Classification on Heterogeneous Information Networks. Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases: Part I, Barcelona, Spain.
48. Ziqi, L., Chen, C., Yang, X., Zhou, J., Li, X., and Song, L. (2018, January 22–26). Heterogeneous Graph Neural Networks for Malicious Account Detection. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, New York, NY, USA.
49. Kingma, D., and Ba, J. (2015). Adam: A Method for Stochastic Optimization. arXiv.
50. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
51. Statistical comparisons of classifiers over multiple data sets