Heterogeneous Graph Purification Network: Purifying Noisy Heterogeneity without Metapaths
Source abstract
Abstract
Heterogeneous graph neural networks (HGNNs) deliver the powerful capability to model many complex systems in real-world scenarios by embedding rich structural and semantic information of a heterogeneous graph into low-dimensional representations. However, existing HGNNs encounter great difficulty in balancing the ability to avoid artificial metapaths with resisting structural and informational noise in a heterogeneous graph. In this paper, we propose a novel framework called Heterogeneous Graph Purification Network (HGPN) which aims to solve such dilemma by adaptively purifying the noisy heterogeneity. Specifically, instead of relying on artificial metapaths, HGPN models heterogeneity by subgraph decomposition and adopts inter-subgraph and intra-subgraph aggregation methods. HGPN can learn to purify noisy edges based on semantic information with a parallel heterogeneous structure purification mechanism. Besides, we design a neighborhood-related dynamic residual update method, a type-specific normalization module and cluster-aware loss to help all types of node achieve high-quality representations and maintain feature distribution while preventing feature over-mixing problems. Extensive experiments are conducted on four common heterogeneous graph datasets, and results show that our approach outperforms all existing methods and achieves state-of-the-art performances consistently among all the datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Recommender Systems and Techniques · Graph Theory and Algorithms
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
Key Laboratory of Network Information System Technology (NIST), Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
University of Chinese Academy of Sciences, Beijing 100190, China
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China
References (44)
Show source-provided references
1. Mining heterogeneous information networks: A structural analysis approach10.1145/2481244.2481248 ↗
2. A heterogeneous graph embedding framework for location-based social network analysis in smart cities
3. He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. (2020, January 25–30). LIGHTGCN: Simplifying and powering graph convolution network for recommendation. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Online.10.1145/3397271.3401063 ↗
4. Welling, M., and Kipf, T.N. (2017, January 24–26). Semi-supervised classification with graph convolutional networks. Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, France.
5. Inductive representation learning on large graphs
6. Graph attention networks
7. Fu, X., Zhang, J., Meng, Z., and King, I. (2020, January 20–24). MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding. Proceedings of the Web Conference 2020, Taipei, Taiwan.10.1145/3366423.3380297 ↗
8. Hu, Z., Dong, Y., Wang, K., and Sun, Y. (2020, January 20–24). Heterogeneous graph transformer. Proceedings of the Web Conference 2020, Taipei, Taiwan.10.1145/3366423.3380027 ↗
9. Zhu, S., Zhou, C., Pan, S., Zhu, X., and Wang, B. (2019, January 8–11). Relation structure-aware heterogeneous graph neural network. Proceedings of the 2019 IEEE International Conference on Data Mining (ICDM), Beijing, China.10.1109/ICDM.2019.00203 ↗
10. Open graph benchmark: Datasets for machine learning on graphs
11. Lv, Q., Ding, M., Liu, Q., Chen, Y., Feng, W., He, S., Zhou, C., Jiang, J., Dong, Y., and Tang, J. (2021, January 14–18). Are we really making much progress? Revisiting, benchmarking and refining heterogeneous graph neural networks. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Online.10.1145/3447548.3467350 ↗
12. Nonlinear dimensionality reduction by locally linear embedding10.1126/science.290.5500.2323 ↗
13. Laplacian eigenmaps and spectral techniques for embedding and clustering
14. Distributed representations of words and phrases and their compositionality
15. Perozzi, B., Al-Rfou, R., and Skiena, S. (2014, January 24–27). Deepwalk: Online learning of social representations. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/2623330.2623732 ↗
16. Grover, A., and Leskovec, J. (2016, January 13–17). node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939754 ↗
17. Dong, Y., Chawla, N.V., and Swami, A. (2017, January 13–17). metapath2vec: Scalable representation learning for heterogeneous networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.10.1145/3097983.3098036 ↗
18. Abu-El-Haija, S., Kapoor, A., Perozzi, B., and Lee, J. (2020, January 3–6). N-GCN: Multi-scale graph convolution for semi-supervised node classification. Proceedings of the Uncertainty in Artificial Intelligence, PMLR, 2020, Online.
19. Zhang, M., Cui, Z., Neumann, M., and Chen, Y. (2018, January 2–7). An end-to-end deep learning architecture for graph classification. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, New Orleans, LA, USA.10.1609/aaai.v32i1.11782 ↗
20. Graph neural network for traffic forecasting: A survey10.1016/j.eswa.2022.117921 ↗
21. Graph neural networks in recommender systems: A survey10.1145/3342103 ↗
22. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., and Welling, M. (2018, January 3–7). Modeling relational data with graph convolutional networks. Proceedings of the European Semantic Web Conference, Heraklion, Greece.10.1007/978-3-319-93417-4_38 ↗
23. Zhang, C., Song, D., Huang, C., Swami, A., and Chawla, N.V. (2019, January 4–8). Heterogeneous graph neural network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA.10.1145/3292500.3330961 ↗
24. Hong, H., Guo, H., Lin, Y., Yang, X., Li, Z., and Ye, J. (2020, January 7–12). An attention-based graph neural network for heterogeneous structural learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, New York, NY, USA.
25. Attention is all you need
26. Alon, U., and Yahav, E. (2020). On the bottleneck of graph neural networks and its practical implications. arXiv.
27. Translating embeddings for modeling multi-relational data
28. Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv.
29. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
30. Li, G., Müller, M., Ghanem, B., and Koltun, V. (2021, January 18–24). Training graph neural networks with 1000 layers. Proceedings of the International Conference on Machine Learning, PMLR, 2021, Online.
31. Van den Oord, A., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv.
32. Wang, H., and Leskovec, J. (2020). Unifying graph convolutional neural networks and label propagation. arXiv.
33. Shi, Y., Huang, Z., Feng, S., Zhong, H., Wang, W., and Sun, Y. (2020). Masked label prediction: Unified message passing model for semi-supervised classification. arXiv.10.24963/ijcai.2021/214 ↗
34. Sun, K., Lin, Z., and Zhu, Z. (2020, January 7–12). Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, New York, NY, USA.
35. Sun, C., Gu, H., and Hu, J. (2021). Scalable and adaptive graph neural networks with self-label-enhanced training. arXiv.
36. Yu, L., Shen, J., Li, J., and Lerer, A. (2020). Scalable graph neural networks for heterogeneous graphs. arXiv.
37. Zhang, W., Yin, Z., Sheng, Z., Li, Y., Ouyang, W., Li, X., Tao, Y., Yang, Z., and Cui, B. (2022). Graph attention multi-layer perceptron. arXiv.10.1145/3534678.3539121 ↗
38. Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv.
39. Pytorch: An imperative style, high-performance deep learning library
40. Wang, M.Y. (2019, January 6–9). Deep graph library: Towards efficient and scalable deep learning on graphs. Proceedings of the ICLR Workshop on Representation Learning on Graphs and Manifolds, New Orleans, LA, USA.
41. Frasca, F., Rossi, E., Eynard, D., Chamberlain, B., Bronstein, M., and Monti, F. (2020). Sign: Scalable inception graph neural networks. arXiv.
42. Ioffe, S., and Szegedy, C. (2015, January 7–9). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, PMLR, 2015, Lille, France.
43. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
44. Rong, Y., Huang, W., Xu, T., and Huang, J. (2019). Dropedge: Towards deep graph convolutional networks on node classification. arXiv.