Learning to Segment Blob-like Objects by Image-Level Counting
Source abstract
Abstract
There is a high demand for manually annotated data in many of the segmentation tasks based on neural networks. Selecting objects pixel by pixel not only takes much time, but it can also lead to inattentiveness and to inconsistencies due to changing annotators for different datasets and monotonous work. This is especially, but not exclusively, the case with sensor data such as microscopy imaging, where many blob-like objects need to be annotated. In addressing these problems, we present a weakly supervised training method that uses object counts at the image level to learn a segmentation implicitly instead of relying on a pixelwise annotation. Our method uses a given segmentation network and extends it with a counting head to enable training by counting. As part of the method, we introduce two specialized losses, contrast loss and morphological loss, which allow for a blob-like output with high contrast to be extracted from the last convolutional layer of the network before the actual counting. We show that similar high F1-scores can be achieved with weakly supervised learning methods as with strongly supervised training; in addition, we address the limitations of the presented method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Cell Image Analysis Techniques · Digital Imaging for Blood Diseases · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, TU Dortmund University, 44227 Dortmund, Germany
References (40)
Show source-provided references
1. Vădineanu, C., Pelt, D.M., Dzyubachyk, O., and Batenburg, K.J. (2022, January 6–8). An Analysis of the Impact of Annotation Errors on the Accuracy of Deep Learning for Cell Segmentation. Proceedings of the 5th International Conference on Medical Imaging with Deep Learning, Zurich, Switzerland.
2. Weakly Supervised Object Localization and Detection: A Survey10.1109/TPAMI.2020.3046647 ↗
3. Revisiting Crowd Counting: State-of-the-Art, Trends, and Future Perspectives10.1016/j.imavis.2022.104597 ↗
4. Pandey, G., and Dukkipati, A. (2019, January 7–11). Learning to Segment with Image-Level Supervision. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA.10.1109/WACV.2019.00202 ↗
5. Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., and Torralba, A. (2016, January 7–12). Learning Deep Features for Discriminative Localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2016.319 ↗
6. Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images10.1002/cyto.a.23863 ↗
7. Deep Learning-Based Image Processing in Optical Microscopy10.1007/s12551-022-00949-3 ↗
8. Recent Advances in Surface Plasmon Resonance Imaging and Biological Applications10.1016/j.talanta.2022.124213 ↗
9. Hergenröder, R., Weichert, F., Wüstefeld, K., and Shpacovitch, V. (2023). Volume 3 Applications, De Gruyter.
10. Libuschewski, P. (2017). Exploration of Cyber-Physical Systems for GPGPU Computer Vision-Based Detection of Biological Viruses. [Ph.D. Thesis, TU Dortmund].
11. Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2015, January 7–12). Is Object Localization for Free?—Weakly-Supervised Learning with Convolutional Neural Networks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298668 ↗
12. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
13. Gao, G., Gao, J., Liu, Q., Wang, Q., and Wang, Y. (2020). CNN-based Density Estimation and Crowd Counting: A Survey. arXiv.
14. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv.
15. Li, Y., Zhang, X., and Chen, D. (2018, January 18–23). CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00120 ↗
16. Transcrowd: Weakly-Supervised Crowd Counting With Transformers10.1007/s11432-021-3445-y ↗
17. Ronneberger, O., Fischer, P., and Brox, T. (2015). Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October 2015, Springer. Proceedings Part III 18.
18. U-Net-Based Medical Image Segmentation
19. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Proceedings of the 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018, Springer. Proceedings 4.
20. Attention Is All You Need
21. Gohel, P., Singh, P., and Mohanty, M. (2021). Explainable AI: Current Status and Future Directions. arXiv.
22. Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., and Batra, D. (2017, January 22–29). Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
23. Chattopadhyay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2018, January 12–15). Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00097 ↗
24. Full-Gradient Representation for Neural Network Visualization
25. Draelos, R.L., and Carin, L. (2020). Use HiResCAM Instead of Grad-CAM for Faithful Explanations of Convolutional Neural Networks. arXiv.
26. LayerCAM: Exploring Hierarchical Class Activation Maps for Localization10.1109/TIP.2021.3089943 ↗
27. Muhammad, M.B., and Yeasin, M. (2020, January 29–24). Eigen-CAM: Class Activation Map Using Principal Components. Proceedings of the 2020 International Joint Conference on Neural Networks, Glasgow, UK.10.1109/IJCNN48605.2020.9206626 ↗
28. Desai, S., and Ramaswamy, H.G. (2020, January 1–5). Ablation-CAM: Visual Explanations for Deep Convolutional Network via Gradient-free Localization. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093360 ↗
29. Wang, H., Du, M., Yang, F., and Zhang, Z. (2020, January 13–19). Score-CAM: Improved Visual Explanations via Score-Weighted Class Activation Mapping. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.
30. Zheng, H., Yang, Z., Liu, W., Liang, J., and Li, Y. (2015, January 12–17). Improving Deep Neural Networks Using Softplus Units. Proceedings of the 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.
31. Wüstefeld, K., and Weichert, F. (2020, January 25–28). An Automated Rapid Test for Viral Nanoparticles Based on Spatiotemporal Deep Learning. Proceedings of the 2020 IEEE Sensors Conference, Virtual.10.1109/SENSORS47125.2020.9278935 ↗
32. Image Segmentation Using Multilevel Thresholding: A Research Review10.1007/s40998-019-00251-1 ↗
33. PyTorch: An Imperative Style, High-Performance Deep Learning Library
34. Riba, E., Mishkin, D., Ponsa, D., Rublee, E., and Bradski, G.R. (2020, January 1–5). Kornia: An Open Source Differentiable Computer Vision Library for PyTorch. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA.10.1109/WACV45572.2020.9093363 ↗
35. Azad, R., Aghdam, E.K., Rauland, A., Jia, Y., Avval, A.H., Bozorgpour, A., Karimijafarbigloo, S., Cohen, J.P., Adeli, E., and Merhof, D. (2022). Medical Image Segmentation Review: The Success of U-Net. arXiv.
36. Chang, M., Li, Q., Feng, H., and Xu, Z. (2020). Spatial-Adaptive Network for Single Image Denoising. arXiv.10.1007/978-3-030-58577-8_11 ↗
37. Wu, Y., and He, K. (2018, January 8–14). Group Normalization. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01261-8_1 ↗
38. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
39. The Surface-Plasmon-Resonance Effect of Nanogold/Silver and Its Analytical Applications10.1016/j.trac.2012.03.015 ↗
40. Roth, A., Wüstefeld, K., and Weichert, F. (2021). A Data-Centric Augmentation Approach for Disturbed Sensor Image Segmentation. J. Imaging, 7.10.3390/jimaging7100206 ↗