DRNet: Towards fast, accurate and practical dish recognition
Source abstract
Abstract
Existing algorithms of dish recognition mainly focus on accuracy with predefined classes, thus limiting their application scope. In this paper, we propose a practical two-stage dish recognition framework (DRNet) that yields a tradeoff between speed and accuracy while adapting to the variation in class numbers. In the first stage, we build an arbitrary-oriented dish detector (AODD) to localize dish position, which can effectively alleviate the impact of background noise and pose variations. In the second stage, we propose a dish reidentifier (DReID) to recognize the registered dishes to handle uncertain categories. To further improve the accuracy of DRNet, we design an attribute recognition (AR) module to predict the attributes of dishes. The attributes are used as auxiliary information to enhance the discriminative ability of DRNet. Moreover, pruning and quantization are processed on our model to be deployed in embedded environments. Finally, to facilitate the study of dish recognition, a well-annotated dataset is established. Our AODD, DReID, AR, and DRNet run at about 14, 25, 16, and 5 fps on the hardware RKNN 3399 pro, respectively.
Source: expertdata · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Currency Recognition and Detection · Handwritten Text Recognition Techniques · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Nag N, Pandey V, Jain R C. Health multimedia: Lifestyle recommendations based on diverse observations. In: Proceedings of the ACM on International Conference on Multimedia Retrieval. Bucharest, 2017. 99–10610.1145/3078971.3080545 ↗
2. Aguilar E, Remeseiro B, Bolanos M, et al. Grab, pay, and eat: Semantic food detection for smart restaurants. IEEE Trans Multimedia, 2018, 20: 3266–327510.1109/TMM.2018.2831627 ↗
3. Min W, Jiang S, Liu L, et al. A survey on food computing. ACM Comput Surv, 2019, 52: 1–3610.1145/3329168 ↗
4. Jiang S, Min W, Liu L, et al. Multi-scale multi-view deep feature aggregation for food recognition. IEEE Trans Image Process, 2020, 29: 265–27610.1109/TIP.2019.2929447 ↗
5. Jiang S, Min W, Lyu Y, et al. Few-shot food recognition via multiview representation learning. ACM Trans Multimedia Comput Commun Appl, 2020, 16: 1–20
6. Sandler M, Howard A G, Zhu M L, et al. MobileNet V2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 4510–452010.1109/CVPR.2018.00474 ↗
7. Xin L, Xian N. Biological object recognition approach using space variant resolution and pigeon-inspired optimization for UAV. Sci China Tech Sci, 2017, 60: 1577–158410.1007/s11431-016-0641-4 ↗
8. Xu S, Ding Y. Real-time recognition of human lower-limb locomotion based on exponential coordinates of relative rotations. Sci China Tech Sci, 2021, 64: 1423–143510.1007/s11431-020-1802-2 ↗
9. Chen M, Dhingra K, Wu W, et al. PFID: Pittsburgh fast-food image dataset. In: Proceedings of the 16th IEEE International Conference on Image Processing. Cairo, 2009. 289–29210.1109/ICIP.2009.5413511 ↗
10. Bossard L, Guillaumin M, Gool L V. Food-101—Mining discriminative components with random forests. In: Proceedings of the European Conference on Computer Vision. Zurich, 2014. 446–46110.1007/978-3-319-10599-4_29 ↗
11. Chen J J, Ngo C W. Deep-based ingredient recognition for cooking recipe retrieval. In: Proceedings of the ACM Multimedia Conference. New York, 2016. 32–4110.1145/2964284.2964315 ↗
12. Ciocca G, Napoletano P, Schettini R. Food recognition: A new dataset, experiments, and results. IEEE J Biomed Health Inform, 2017, 21: 588–59810.1109/JBHI.2016.2636441 ↗
13. Min W, Liu L, Wang Z, et al. ISIA food-500: A dataset for large-scale food recognition via stacked global-local attention network. In: Proceedings of the 28th ACM International Conference on Multimedia. Seattle, 2020. 393–40110.1145/3394171.3414031 ↗
14. Kagaya H, Aizawa K, Ogawa M. Food detection and recognition using convolutional neural network. In: Proceedings of the 22nd ACM international conference on Multimedia. Orlando Florida, 2014. 1085–108810.1145/2647868.2654970 ↗
15. Xu R H, Herranz L, Jiang S Q, et al. Geolocalized modeling for dish recognition. IEEE Trans Multimedia, 2015, 17: 1187–119910.1109/TMM.2015.2438717 ↗
16. Zhou F, Lin Y Q. Fine-grained image classification by exploring bipartite-graph labels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016. 1124–113310.1109/CVPR.2016.127 ↗
17. Fan Y M, Ding M, Cao Y F. Vision algorithms for fixed-wing unmanned aerial vehicle landing system. Sci China Tech Sci, 2017, 60: 434–44310.1007/s11431-016-0618-3 ↗
18. Ren S Q, He K M, Girshick R B, et al. Faster R-CNN: Towards realtime object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 1137–114910.1109/TPAMI.2016.2577031 ↗
19. Lin T Y, Dollár P, Girshick R B, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, 2017. 936–94410.1109/CVPR.2017.106 ↗
20. Liu W, Anguelov D, Erhan D, et al. SSD: Single shot multibox detector. In: Proceedings of the European Conference on Computer Vision. Amsterdam, 2016. 21–3710.1007/978-3-319-46448-0_2 ↗
21. Redmon J, Divvala S K, Girshick R B, et al. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016. 779–78810.1109/CVPR.2016.91 ↗
22. Tian Z, Shen C H, Chen H, et al. FCOS: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, 2019. 9626–963510.1109/ICCV.2019.00972 ↗
23. Duan K, Bai S, Xie L, et al. Centernet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, 2019. 6568–657710.1109/ICCV.2019.00667 ↗
24. Zhou Q Q, Zhong B N, Lan X Y, et al. Fine-grained spatial alignment model for person re-identification with focal triplet loss. IEEE Trans Image Process, 2020, 29: 7578–758910.1109/TIP.2020.3004267 ↗
25. Sun Y F, Zheng L, Yang Y, et al. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision. Germany, 2018. 501–51810.1007/978-3-030-01225-0_30 ↗
26. Wang G S, Yuan Y F, Chen X, et al. Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM International Conference on Multimedia. New York, 2018. 274–28210.1145/3240508.3240552 ↗
27. Luo H, Gu Y, Liao X, et al. Bag of tricks and a strong baseline for deep person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, 2019. 1487–149510.1109/CVPRW.2019.00190 ↗
28. Lin Y, Zheng L, Zheng Z, et al. Improving person re-identification by attribute and identity learning. Pattern Recogn, 2019, 95: 151–16110.1016/j.patcog.2019.06.006 ↗
29. Zhong B N, Bai B, Li J, et al. Hierarchical tracking by reinforcement learning-based searching and coarse-to-fine verifying. IEEE Trans Image Process, 2019, 28: 2331–234110.1109/TIP.2018.2885238 ↗
30. Chen Z D, Zhong B N, Li G R, et al. Siamese box adaptive network for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, 2020. 6667–667610.1109/CVPR42600.2020.00670 ↗
31. Zhong B N, Yao H X, Chen S, et al. Visual tracking via weakly supervised learning from multiple imperfect oracles. Pattern Recogn, 2014, 47: 1395–141010.1016/j.patcog.2013.10.002 ↗
32. Han S, Liu X, Mao H, et al. EIE: Efficient inference engine on compressed deep neural network. In: Proceedings of the 43rd Annual International Symposium on Computer Architecture. Seoul, 2016. 243–25410.1145/3007787.3001163 ↗
33. Gao X, Zhao Y, Dudziak L, et al. Dynamic channel pruning: Feature boosting and suppression. In: Proceedings of the International Conference on Learning Representations. New Orleans, 2019. 1–14
34. Banner R, Nahshan Y, Soudry D. Post training 4-bit quantization of convolutional networks for rapid-deployment. In: Proceedings of the Advances in Neural Information Processing Systems. Vancouver, 2019. 7950–7958
35. Jacob B, Kligys S, Chen B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 2704–271310.1109/CVPR.2018.00286 ↗
36. Chen H T, Wang Y H, Xu C J, et al. Addernet: Do we really need multiplications in deep learning? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, 2020. 1465–147410.1109/CVPR42600.2020.00154 ↗
37. Mezgec S, Koroušić S B. NutriNet: A deep learning food and drink image recognition system for dietary assessment. Nutrients, 2017, 9: 65710.3390/nu9070657 ↗