Automatic Identification of Local Features Representing Image Content with the Use of Convolutional Neural Networks
Source abstract
Abstract
Image analysis has many practical applications and proper representation of image content is its crucial element. In this work, a novel type of representation is proposed where an image is reduced to a set of highly sparse matrices. Equivalently, it can be viewed as a set of local features of different types, as precise coordinates of detected keypoints are given. Additionally, every keypoint has a value expressing feature intensity at a given location. These features are extracted from a dedicated convolutional neural network autoencoder. This kind of representation has many advantages. First of all, local features are not manually designed but are automatically trained for a given class of images. Second, as they are trained in a network that restores its input on the output, they may be expected to minimize information loss. Consequently, they can be used to solve similar tasks replacing original images; such an ability was illustrated with image classification task. Third, the generated features, although automatically synthesized, are relatively easy to interpret. Taking a decoder part of our network, one can easily generate a visual building block connected with a specific feature. As the proposed method is entirely new, a detailed analysis of its properties for a relatively simple data set was conducted and is described in this work. Moreover, to present the quality of trained features, it is compared with results of convolutional neural networks having a similar working principle (sparse coding).
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image and Video Retrieval Techniques · Advanced Vision and Imaging · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Information Technology, Lodz University of Technology, ul. Wolczanska 215, 90-924 Lodz, Poland
Funding
Narodowe Centrum Nauki
References (40)
Show source-provided references
1. Marr, D., Ullman, S., and Poggio, T. (2010). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, The MIT Press.10.7551/mitpress/9780262514620.001.0001 ↗
2. Arbib, M.A. (1995). Convolutional Networks for Images, Speech and Time Series. The Handbook of Brain Theory and Neural Networks, The MIT Press.
3. Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014). Visualizing and Understanding Convolutional Networks. Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part I, Springer International Publishing.10.1007/978-3-319-10578-9 ↗
4. Do Semantic Parts Emerge in Convolutional Neural Networks?10.1007/s11263-017-1048-0 ↗
5. Part Detector Discovery in Deep Convolutional Neural Networks
6. Zhang, Q., Wang, X., Cao, R., Wu, Y.N., Shi, F., and Zhu, S. (2018). Explanatory Graphs for CNNs. CoRR, abs/1812.07997.
7. LeCun, Y., and Cortes, C. MNIST handwritten digit database.
8. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods10.1109/TPAMI.2012.120 ↗
9. LSD: A Line Segment Detector10.5201/ipol.2012.gjmr-lsd ↗
10. Distinctive Image Features from Scale-Invariant Keypoints10.1023/B:VISI.0000029664.99615.94 ↗
11. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
12. Geometric Deep Learning: Going beyond Euclidean Data10.1109/MSP.2017.2693418 ↗
13. Monti, F., Boscaini, D., Masci, J., Rodolà, E., Svoboda, J., and Bronstein, M.M. (2017, January 21–26). Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.10.1109/CVPR.2017.576 ↗
14. Tomczyk, A., and Szczepaniak, P.S. (2019). Ear Detection Using Convolutional Neural Network on Graphs with Filter Rotation. Sensors, 19.10.3390/s19245510 ↗
15. Kwaśnicka, H., and Jain, L.C. (2018). Active Partitions in Localization of Semantically Important Image Structures. Bridging the Semantic Gap in Image and Video Analysis, Springer International Publishing. Intelligent Systems Reference Library.10.1007/978-3-319-73891-8 ↗
16. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning Deep Features for Discriminative Localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.319 ↗
17. Liu, B., Wang, M., Foroosh, H., Tappen, M., and Penksy, M. (2015, January 7–12). Sparse Convolutional Neural Networks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
18. Gu, S., Zuo, W., Xie, Q., Meng, D., Feng, X., and Zhang, L. (2015, January 5–13). Convolutional Sparse Coding for Image Super-Resolution. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.212 ↗
19. Fast convolutional sparse coding using matrix inversion lemma10.1016/j.dsp.2016.04.012 ↗
20. Heide, F., Heidrich, W., and Wetzstein, G. (2015, January 7–12). Fast and flexible convolutional sparse coding. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7299149 ↗
21. Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification10.1007/s11263-014-0722-8 ↗
22. Yang, J., Yu, K., and Huang, T. (2010, January 13–18). Supervised translation-invariant sparse coding. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539958 ↗
23. Chen, B., Li, J., Ma, B., and Wei, G. (2016, January 25–28). Convolutional sparse coding classification model for image classification. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP).10.1109/ICIP.2016.7532692 ↗
24. Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation10.1109/TSP.2016.2550016 ↗
25. Wang, Z., Yang, J., Nasrabadi, N., and Huang, T. (2013, January 1–8). A Max-Margin Perspective on Sparse Representation-Based Classification. Proceedings of the 2013 IEEE International Conference on Computer Vision.10.1109/ICCV.2013.154 ↗
26. Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition10.1007/s11263-014-0788-3 ↗
27. Role of Temporal Integration and Fluctuation Detection in the Highly Irregular Firing of a Leaky Integrator Neuron Model with Partial Reset10.1162/neco.1997.9.5.985 ↗
28. Hunsberger, E., and Eliasmith, C. (2015). Spiking Deep Networks with LIF Neurons. CoRR, abs/1510.08829.
29. Tavanaei, A., and Maida, A.S. (2016). Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning. CoRR, abs/1611.03000.
30. Skew variation, a rejoinder
31. Diehl, P.U., Neil, D., Binas, J., Cook, M., Liu, S.C., and Pfeiffer, M. (2015, January 12–17). Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. Proceedings of the 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.10.1109/IJCNN.2015.7280696 ↗
32. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3–7). Caffe: Convolutional Architecture for Fast Feature Embedding. Proceedings of the 22Nd ACM International Conference on Multimedia, MM ’14, Orlando, FL, USA.10.1145/2647868.2654889 ↗
33. Receptive Fields and Functional Architecture in Two Nonstriate Visual Areas (18 and 19) of the Cat10.1152/jn.1965.28.2.229 ↗
34. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
35. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7–13). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV ’15, Santiago, Chile.10.1109/ICCV.2015.123 ↗
36. Wan, L., Zeiler, M., Zhang, S., LeCun, Y., and Fergus, R. (2013, January 17–19). Regularization of Neural Networks Using Dropconnect. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28. JMLR.org, 2013, ICML’13, Atlanta, GE, USA.
37. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the IEEE Computer Vision and Pattern Recognition (CVPR) 2009, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
38. The Pascal Visual Object Classes (VOC) Challenge10.1007/s11263-009-0275-4 ↗
39. Laptev, D., Savinov, N., Buhmann, J.M., and Pollefeys, M. (2016). TI-POOLING: Transformation-invariant pooling for feature learning in Convolutional Neural Networks. CoRR, abs/1604.06318.10.1109/CVPR.2016.38 ↗
40. Geometric Transformations Embedded into Convolutional Neural Networks