Deep Subspace Clustering with Block Diagonal Constraint
Source abstract
Abstract
The deep subspace clustering method, which adopts deep neural networks to learn a representation matrix for subspace clustering, has shown good performance. However, this representation matrix ignores the structural constraint when it is applied to subspace clustering. It is known that samples from different classes can be taken as embedding in independent subspaces. Thus, the representation matrix should have a block diagonal structure. This paper presents the Deep Subspace Clustering with Block Diagonal Constraint (DSC-BDC), a model which constrains the representation matrix with block diagonal structure and gives a block diagonal regularizer for learning a suitable representation. Furthermore, to enhance the representation capacity, DSC-BDC reforms the block-diagonal structure constraint by performing a separation strategy on the representation matrix. Specifically, the separation strategy ensures that the most compact samples are selected to the represent data. An alternative optimization algorithm is designed for our model. Extensive experiments on four public and real-world databases demonstrate the effectiveness and superiority of our proposed model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face and Expression Recognition · Remote-Sensing Image Classification · Blind Source Separation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
Funding
National Natural Science Foundation of China under Grant
References (54)
Show source-provided references
1. Multiscale Hybrid Linear Models for Lossy Image Representation10.1109/TIP.2006.882016 ↗
2. Robust Recovery of Subspace Structures by Low-Rank Representation10.1109/TPAMI.2012.88 ↗
3. Robust Recovery of Signals From a Structured Union of Subspaces10.1109/TIT.2009.2030471 ↗
4. Cao, X., Zhang, C., Fu, H., Liu, S., and Zhang, H. (2015, January 7–12). Diversity-induced Multi-view Subspace Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298657 ↗
5. Zhang, C., Fu, H., Liu, S., Liu, G., and Cao, X. (2015). Low-Rank Tensor Constrained Multiview Subspace Clustering. IEEE Int. Conf. Comput. Vis., 1582–1590.10.1109/ICCV.2015.185 ↗
6. Pham, D., Saha, B., Phung, D.Q., and Venkatesh, S. (2012, January 16–21). Improved Subspace Clustering via Exploitation of Spatial Constraints. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.
7. Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories10.1109/TPAMI.2009.191 ↗
8. Nearness to Local Subspace Algorithm for Subspace and Motion Segmentation10.1109/LSP.2012.2214211 ↗
9. Fuzzy C-means Clustering with Prior Biological Knowledge10.1016/j.jbi.2008.05.009 ↗
10. A Fully Automatic 2D Segmentation Method for Uterine Fibroid in MRgFUS Treatment Evaluation10.1016/j.compbiomed.2015.04.030 ↗
11. Subspace Clustering for High Dimensional Data: A Review10.1145/1007730.1007731 ↗
12. Agrawal, R., Gehrke, J., Gunopulos, D., and Raghavan, P. (1998, January 2–4). Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications. Proceedings of the International Conference on Management of Data, Seattle, WA, USA.10.1145/276304.276314 ↗
13. Sparse Subspace Clustering: Algorithm, Theory, and Applications10.1109/TPAMI.2013.57 ↗
14. Spectral Curvature Clustering (SCC)10.1007/s11263-008-0178-9 ↗
15. Liu, G., Lin, Z., and Yu, Y. (2010, January 21–24). Robust Subspace Segmentation by Low-Rank Representation. Proceedings of the International Conference on Machine Learning, Haifa, Israel.
16. Lu, C., Min, H., Zhao, Z., Zhu, L., Huang, D., and Yan, S. (2012, January 7–13). Robust and Efficient Subspace Segmentation via Least Squares Regression. Proceedings of the European Conference on Computer Vision, Florence, Italy.10.1007/978-3-642-33786-4_26 ↗
17. Elhamifar, E., and Vidal, R. (2009, January 20–25). Sparse Subspace Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPRW.2009.5206547 ↗
18. Low Rank Subspace Clustering (LRSC)10.1016/j.patrec.2013.08.006 ↗
19. Feng, J., Lin, Z., Xu, H., and Yan, S. (2014, January 23–28). Robust Subspace Segmentation with Block-diagonal Prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.482 ↗
20. Subspace Clustering by Block Diagonal Representation10.1109/TPAMI.2018.2794348 ↗
21. Multi-View Subspace Clustering With Block Diagonal Representation10.1109/ACCESS.2019.2923614 ↗
22. Wang, Y., Xu, H., and Leng, C. (2013, January 3–7). Provable Subspace Clustering: When LRR Meets SSC. Proceedings of the Neural Information Processing Systems, Daegu, Korea.
23. Ji, P., Salzmann, M., and Li, H. (2015, January 7–13). Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.532 ↗
24. Split Multiplicative Multi-View Subspace Clustering10.1109/TIP.2019.2913096 ↗
25. Chen, G., Atev, S., and Lerman, G. (October, January 27). Kernel Spectral Curvature Clustering (KSCC). Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan.
26. Patel, V.M., Van Nguyen, H., and Vidal, R. (2013, January 1–8). Latent Space Sparse Subspace Clustering. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.35 ↗
27. Patel, V.M., and Vidal, R. (2014, January 27–30). Kernel Sparse Subspace Clustering. Proceedings of the IEEE International Conference on Image Processing, Paris, France.10.1109/ICIP.2014.7025576 ↗
28. Yin, M., Guo, Y., Gao, J., He, Z., and Xie, S. (2016, January 27–30). Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.557 ↗
29. Peng, X., Xiao, S., Feng, J., Yau, W., and Yi, Z. (2016, January 9–15). Deep Subspace Clustering with Sparsity Prior. Proceedings of the International Joint Conference on Artificial Intelligence, New York, NY, USA.
30. Ji, P., Zhang, T., Li, H., Salzmann, M., and Reid, I. (2017, January 4–9). Deep Subspace Clustering Networks. Proceedings of the Conference and Workshop on Neural Information Processing Systems, Long Beach, CA, USA.
31. Structured AutoEncoders for Subspace Clustering10.1109/TIP.2018.2848470 ↗
32. Zhang, J., Li, C., You, C., Qi, X., Zhang, H., Guo, J., and Lin, Z. (2019, January 16–20). Self-Supervised Convolutional Subspace Clustering Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00562 ↗
33. Zhang, T., Ji, P., Harandi, M., Huang, W., and Li, H. (2019, January 10–15). Neural Collaborative Subspace Clustering. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
34. Zhou, P., Hou, Y., and Feng, J. (2018, January 18–23). Deep Adversarial Subspace Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00172 ↗
35. Yang, S., Zhu, W., and Zhu, Y. (2020, January 25–28). Residual Encoder-Decoder Network for Deep Subspace Clustering. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, UAE.10.1109/ICIP40778.2020.9191335 ↗
36. Deep Clustering With Sample-Assignment Invariance Prior10.1109/TNNLS.2019.2958324 ↗
37. Effect of the Latent Structure on Clustering with GANs10.1109/LSP.2020.2996935 ↗
38. Mukherjee, S., Asnani, H., Lin, E., and Kannan, S. (2019). ClusterGAN: Latent Space Clustering in Generative Adversarial Networks. arXiv.10.1609/aaai.v33i01.33014610 ↗
39. Zhao, S., Song, J., and Ermon, S. (2017). InfoVAE: Information Maximizing Variational Autoencoders. arXiv.
40. Han, C., Murao, K., Noguchi, T., Kawata, Y., Uchiyama, F., Rundo, L., Nakayama, H., and Satoh, S. (2019, January 3–7). Learning More with Less: Conditional PGGAN-based Data Augmentation for Brain Metastases Detection Using Highly-Rough Annotation on MR Images. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.10.1145/3357384.3357890 ↗
41. Kingma, D.P., and Welling, M. (2014). Auto-Encoding Variational Bayes. arXiv.
42. Generative Adversarial Networks for Noise Reduction in Low-Dose CT10.1109/TMI.2017.2708987 ↗
43. Masci, J., Meier, U., Ciresan, D.C., and Schmidhuber, J. (2011). Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. International Conference on Artificial Neural Networks, Springer.10.1007/978-3-642-21735-7_7 ↗
44. Chung, F.R., and Graham, F.C. (1997). Spectral Graph Theory, American Mathematical Society. Number 92.
45. Kheirandishfard, M., Zohrizadeh, F., and Kamangar, F. (2020, January 1–5). Multi-Level Representation Learning for Deep Subspace Clustering. Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093277 ↗
46. Lin, Z., Liu, R., and Su, Z. (2011). Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation. Adv. Neural Inf. Process. Syst., 612–620.
47. Ji, P., Salzmann, M., and Li, H. (2014, January 24–26). Efficient Dense Subspace Clustering. Proceedings of the Workshop on Applications of Computer Vision, Steamboat Springs, CO, USA.
48. Ji, P., Reid, I., Garg, R., Li, H., and Salzmann, M. (2017). Low-Rank Kernel Subspace Clustering. arXiv.
49. Convolutional Subspace Clustering Network with Block Diagonal Prior10.1109/ACCESS.2019.2963279 ↗
50. From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose10.1109/34.927464 ↗
51. Samaria, F.S., and Harter, A. (1994, January 5–7). Parameterisation of a Stochastic Model for Human Face Identification. Proceedings of the 1994 IEEE Workshop on Applications of Computer Vision, Sarasota, FL, USA.
52. Homomorphic Filtering Based Illumination Normalization Method for Face Recognition10.1016/j.patrec.2011.03.023 ↗
53. Sim, T., Baker, S., and Bsat, M. (2002, January 20–21). The CMU Pose, Illumination, and Expression (PIE) Database. Proceedings of the Fifth IEEE International Conference on Automatic Face Gesture Recognition, Washington, DC, USA.
54. Nene, S.A., Nayar, S.K., and Murase, H. (1996). Columbia Object Image Library (COIL-20), Department of Computer Science, Columbia University.