An Improved Knowledge Distillation Algorithm and Its Application to Object Detection
Source abstract
Abstract
Knowledge distillation (KD) is an emerging model compression technique for learning compact object detector models. Previous KD often focused solely on distilling from the logits layer or the feature intermediate layers, whic... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced Algorithms and Applications · Advanced Measurement and Detection Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (27)
Show source-provided references
1. Mask R-CNN
2. Faster R-CNN: towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
3. Relation networks for object detection
4. Focal loss for dense object detection10.1109/TPAMI.2018.2858826 ↗
5. SSD: single shot multibox detector
6. Redmon J, Farhadi A. YOLOv3: an incremental improvement. arXiv:1804.02767v1. 2018.
7. Fully cascade consistency learning for one-stage object detection10.1109/TCSVT.2023.3263557 ↗
8. Cascade R-CNN: delving into high quality object detection
9. A dense-aware cross-splitNet for object detection and recognition10.1109/TCSVT.2022.3221658 ↗
10. FCOS: fully convolutional one-stage object detection
11. CrossDet++: growing crossline representation for object detection10.1109/TCSVT.2022.3211734 ↗
12. Feature alignment in anchor-free object detection10.1109/TCSVT.2023.3241993 ↗
13. KRRNet: keypoint relational regression network for bottom-up anchor-free object detection10.1109/TCSVT.2023.3305289 ↗
14. Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. arXiv:1503.02531v1. 2015.
15. Zhou H, Song L, Chen J, Zhou Y, Wang G, Yuan J, et al. Rethinking soft labels for knowledge distillation: a bias-variance tradeoff perspective. arxiv:2102.00650v1. 2021.
16. Adriana R, Nicolas B, Ebrahimi KS, Chassang A, Gatta C, Bengio Y. FitNets: hints for thin deep nets. arXiv:1412.6550. 2014.
17. A comprehensive overhaul of feature distillation
18. Distilling knowledge via knowledge review
19. Chen · 31st Annual Conference on Neural Information Processing Systems (NIPS 2017) · 2017 Dec 3–9 · 2017 · 742
20. Focal and global knowledge distillation for detectors
21. Improve object detection with feature-based knowledge distillation: towards accurate and efficient detectors
22. CrossKD: cross-head knowledge distillation for object detection
23. Cosine similarity-guided knowledge distillation for robust object detectors10.1038/s41598-024-69813-6 ↗
24. Masked generative distillation
25. CBAM: convolutional block attention module
26. Decoupled knowledge distillation
27. Microsoft coco: common objects in context