One-Shot Distributed Generalized Eigenvalue Problem (DGEP): Concept, Algorithm and Experiments
Source abstract
Abstract
This paper focuses on the design of a distributed algorithm for generalized eigenvalue problems (GEPs) in one-shot communication. Since existing distributed methods for eigenvalue decomposition cannot be applied to GEP, a general one-shot distributed GEP framework is proposed. The theoretical analysis of the approximation error reveals its relation to the divergence of the data covariance, the eigenvalues of the empirical data covariance, and the number of local servers. If the symmetric data covariance has repeated eigenvalues in GEP, e.g., in canonical component analysis, we further modify the method for better convergence and prove the necessity experimentally. Numerical experiments validate the effectiveness of the proposed algorithms both on synthetic and real-world datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote-Sensing Image Classification · Soil Geostatistics and Mapping · Geochemistry and Geologic Mapping
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai 200240, China
Quality and Business Operation Department, SAIC Motor Corporation Limited, 489 Wei Hai Road, Shanghai 200041, China
Funding
National Key R&D Program of China · National Science Foundation, China · Committee of Scienceand Technology, Shanghai, China
References (35)
Show source-provided references
1. Zhou, J., Li, X., Zhao, P., Chen, C., Li, L., Yang, X., Cui, Q., Yu, J., Chen, X., and Ding, Y. (2017, January 13–17). Kunpeng: Parameter server based distributed learning systems and its applications in alibaba and ant financial. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Leipzig, Germany.10.1145/3097983.3098029 ↗
2. Accounting for data variability in multi-institutional distributed deep learning for medical imaging10.1093/jamia/ocaa017 ↗
3. Distributed learning from multiple EHR databases: Contextual embedding models for medical events10.1016/j.jbi.2019.103138 ↗
4. Distributed target classification and tracking in sensor networks10.1109/JPROC.2003.814923 ↗
5. Distributed Classification of Multiple Observation Sets by Consensus10.1109/TSP.2010.2086450 ↗
6. de Cock, M., Dowsley, R., Nascimento, A.C., and Newman, S.C. (2015, January 16). Fast, privacy preserving linear regression over distributed datasets based on pre-distributed data. Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security, Denver, CO, USA.10.1145/2808769.2808774 ↗
7. Privacy Preserving Linear Regression on Distributed Databases
8. Privacy-Preserving Distributed Linear Regression on High-Dimensional Data
9. Wang, S., Gittens, A., and Mahoney, M.W. (2017, January 6–11). Sketched ridge regression: Optimization perspective, statistical perspective, and model averaging. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
10. The singular value decomposition: Its computation and some applications10.1109/TAC.1980.1102314 ↗
11. Principal component analysis10.1002/wics.101 ↗
12. Relations Between Two Sets of Variates10.1093/biomet/28.3-4.321 ↗
13. On Minimizing Quadratically Constrained Ratio of Two Quadratic Functions
14. Distributed LCMV Beamforming in a Wireless Sensor Network With Single-Channel Per-Node Signal Transmission10.1109/TSP.2013.2259486 ↗
15. Distributed Node-Specific LCMV Beamforming in Wireless Sensor Networks10.1109/TSP.2011.2169409 ↗
16. Grammenos, A., Mendoza-Smith, R., Mascolo, C., and Crowcroft, J. (2019). Federated PCA with adaptive rank estimation. arXiv.
17. Distributed Estimation of Principal Eigenspaces
18. Improved distributed principal component analysis
19. Tron, R., and Vidal, R. (2011, January 20–25). Distributed computer vision algorithms through distributed averaging. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995654 ↗
20. Ge, J., Wang, Z., Wang, M., and Liu, H. (2018, January 9–11). Minimax-optimal privacy-preserving sparse pca in distributed systems. Proceedings of the International Conference on Artificial Intelligence and Statistics, Lanzarote, Canary Islands.
21. Li, X., Wang, S., Chen, K., and Zhang, Z. (2021, January 18–24). Communication-efficient distributed SVD via local power iterations. Proceedings of the International Conference on Machine Learning, Virtual.
22. Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data10.1109/TSP.2015.2472372 ↗
23. Numerical Analysis of Plasma KdV Equation: Time-Fractional Approach10.1007/s40819-017-0420-7 ↗
24. On the existence of solutions of non-linear 2D Volterra integral equations in a Banach Space10.1007/s13398-022-01246-0 ↗
25. The use of multiple measurements in taxonomic problems10.1111/j.1469-1809.1936.tb02137.x ↗
26. User-Friendly Tail Bounds for Sums of Random Matrices10.1007/s10208-011-9099-z ↗
27. Sparse generalized eigenvalue problem: Optimal statistical rates via truncated Rayleigh flow10.1111/rssb.12291 ↗
28. Distributed Canonical Correlation Analysis in Wireless Sensor Networks With Application to Distributed Blind Source Separation10.1109/TSP.2015.2443729 ↗
29. A Distributed Canonical Correlation Analysis-Based Fault Detection Method for Plant-Wide Process Monitoring10.1109/TII.2019.2893125 ↗
30. Golub, G.H., and Van Loan, C.F. (2013). Matrix Computations, JHU Press.10.56021/9781421407944 ↗
31. Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis
32. PLS regression methods10.1002/cem.1180020306 ↗
33. Perturbation bounds in connection with singular value decomposition10.1007/BF01932678 ↗
34. Canonical Correlation Analysis: An Overview with Application to Learning Methods10.1162/0899766042321814 ↗
35. BagBoosting for tumor classification with gene expression data10.1093/bioinformatics/bth447 ↗