Time Classification Algorithm Based on Windowed-Color Histogram Matching
Source abstract
Abstract
A web-based search system recommends and gives results such as customized image or video contents using information such as user interests, search time, and place. Time information extracted from images can be used as a important metadata in the web search system. We present an efficient algorithm to classify time period into day, dawn, and night when the input is a single image with a sky region. We employ the Mask R-CNN to extract a sky region. Based on the extracted sky region, reference color histograms are generated, which can be considered as the ground-truth. To compare the histograms effectively, we design the windowed-color histograms (for RGB bands) to compare each time period from the sky region of the reference data with one of the input images. Also, we use a weighting approach to reflect a more separable feature on the windowed-color histogram. With the proposed windowed-color histogram, we verify about 91% of the recognition accuracy in the test data. Compared with the existing deep neural network models, we verify that the proposed algorithm achieves better performance in the test dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Video Analysis and Summarization · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of IT Engineering, Sookmyung Women’s University, 100 Chungpa-ro 47 gil, Yongsna-gu, Seoul 04310, Korea
References (40)
Show source-provided references
1. Shiyamala, M., and Kalaiarasi, G. (2014, January 27–28). Contextual image search with keyword and image input. Proceedings of the International Conference on Information Communication and Embedded Systems (ICICES2014), Chennai, India.10.1109/ICICES.2014.7033765 ↗
2. Krapac, J., Allan, M., Verbeek, J., and Juried, F. (2010, January 13–18). Improving web image search results using query-relative classifiers. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5540092 ↗
3. The characteristics of voice search: Comparing spoken with typed-in mobile web search queries10.1145/3182163 ↗
4. Guy, I. (2016, January 17–21). Searching by talking: Analysis of voice queries on mobile web search. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, Pisa, Italy.10.1145/2911451.2911525 ↗
5. Eom, W., Lee, S., De Neve, W., and Ro, Y.M. (2011, January 11–14). Improving image tag recommendation using favorite image context. Proceedings of the 2011 18th IEEE International Conference on Image Processing, Brussels, Belgium.10.1109/ICIP.2011.6116154 ↗
6. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
7. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
8. Jeong, D., Kim, B.G., and Dong, S.Y. (2020). Deep joint spatiotemporal network (DJSTN) for efficient facial expression recognition. Sensors, 20.10.3390/s20071936 ↗
9. Efficient facial expression recognition algorithm based on hierarchical deep neural network structure10.1109/ACCESS.2019.2907327 ↗
10. deepGesture: Deep learning-based gesture recognition scheme using motion sensors10.1016/j.displa.2018.08.001 ↗
11. Yang, Z., Leng, L., and Kim, B.G. (2019). StoolNet for color classification of stool medical images. Electronics, 8.10.3390/electronics8121464 ↗
12. Detection of Dangerous Situations using Deep Learning Model with Relational Inference10.33851/JMIS.2020.7.3.205 ↗
13. Exploiting contextual information for image re-ranking and rank aggregation10.1007/s13735-012-0002-8 ↗
14. Robust image recognition by fusion of contextual information10.1016/S1566-2535(02)00092-1 ↗
15. de Souza Gazolli, K.A., and Salles, E.O.T. (2012). A contextual image descriptor for scene classification. Online Proc. Trends Innov. Comput., 66–71.
16. Pazzani, M.J., and Billsus, D. (2007). Content-based recommendation systems. The Adaptive Web, Springer.10.1007/978-3-540-72079-9_10 ↗
17. Lops, P., De Gemmis, M., and Semeraro, G. (2011). Content-based recommender systems: State of the art and trends. Recommender Systems Handbook, Springer.10.1007/978-0-387-85820-3_3 ↗
18. Popescul, A., Ungar, L.H., Pennock, D.M., and Lawrence, S. (2013). Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. arXiv.
19. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
20. Saha, B., Davies, D., and Raghavan, A. (2016). Day Night Classification of Images Using Thresholding on HSV Histogram. (9,530,056), U.S. Patent.
21. Day/night detector for vehicle tracking in traffic monitoring systems
22. Park, K.H., and Lee, Y.S. (2018, January 29–31). Classification of Daytime and Night Based on Intensity and Chromaticity in RGB Color Image. Proceedings of the 2018 International Conference on Platform Technology and Service (PlatCon), Jeju, Korea.10.1109/PlatCon.2018.8472764 ↗
23. BBAS: Towards large scale effective ensemble adversarial attacks against deep neural network learning10.1016/j.ins.2020.11.026 ↗
24. Adaptive and robust partition learning for person retrieval with policy gradient10.1109/TMM.2020.3023272 ↗
25. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
26. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
27. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4–9). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11231 ↗
28. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–22). Learning transferable architectures for scalable image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
29. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning. PMLR, Long Beach, CA, USA.
30. Volokitin, A., Timofte, R., and Van Gool, L. (July, January 25). Deep features or not: Temperature and time prediction in outdoor scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.
31. Ibrahim, M.R., Haworth, J., and Cheng, T. (2019). WeatherNet: Recognising weather and visual conditions from street-level images using deep residual learning. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8120549 ↗
32. Sky region detection in a single image for autonomous ground robot navigation10.5772/56884 ↗
33. A novel sky region detection algorithm based on border points
34. Wang, C.W., Ding, J.J., and Chen, P.J. (2015, January 16–19). An efficient sky detection algorithm based on hybrid probability model. Proceedings of the 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Hong Kong.10.1109/APSIPA.2015.7415405 ↗
35. An efficient approach for sky detection
36. La Place, C., Urooj, A., and Borji, A. (2019, January 7–11). Segmenting sky pixels in images: Analysis and comparison. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA.10.1109/WACV.2019.00189 ↗
37. Sharma, P., Schoemaker, M., and Pan, D. (2021, November 30). Automated Image Timestamp Inference Using Convolutional Neural Networks. Available online: https://www.semanticscholar.org/paper/Automated-Image-Timestamp-Inference-Using-Neural-Sharma/63f4e243aaefdea5cc14d453c12625242feeb2f1?sort=relevance&citationIntent=background.
38. Hart, P.E., Stork, D.G., and Duda, R.O. (2000). Pattern Classification, Wiley.
39. Fukunaga, K. (2013). Introduction to Statistical Pattern Recognition, Elsevier.
40. Greenwood, P.E., and Nikulin, M.S. (1996). A Guide to Chi-Squared Testing, John Wiley & Sons.