journal article·2018-06-01Open access

Image Captioning with Word Gate and Adaptive Self-Critical Learning

Xinxin Zhu, Lixiang Li, Jing Liu, Longteng Guo, Zhiwei Fang, Haipeng Peng, Xinxin Niu

Applied Sciences· Volume 8· Issue 6

Abstract

Although the policy-gradient methods for reinforcement learning have shown significant improvement in image captioning, how to achieve high performance during the reinforcement optimizing process is still not a simple task. There are at least two difficulties: (1) The large size of vocabulary leads to a large action space, which makes it difficult for the model to accurately predict the current word. (2) The large variance of gradient estimation in reinforcement learning usually causes severe instabilities in the training process. In this paper, we propose two innovations to boost the performance of self-critical sequence training (SCST). First, we modify the standard long short-term memory (LSTM)based decoder by introducing a gate function to reduce the search scope of the vocabulary for any given image, which is termed the word gate decoder. Second, instead of only considering current maximum actions greedily, we propose a stabilized gradient estimation method whose gradient variance is controlled by the difference between the sampling reward from the current model and the expectation of the historical reward. We conducted extensive experiments, and results showed that our method could accelerate the training process and increase the prediction accuracy. Our method was validated on MS COCO datasets and yielded state-of-the-art performance.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Multimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning · Human Pose and Action Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Information Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China

National Engineering Laboratory for Disaster Backup and Recovery, Beijing University of Posts and Telecommunications, Beijing 100876, China

National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

Funding

the National Key Research and Development Program of China · the National Natural Science Foundation of China

References (27)

Show source-provided references
1. Ranzato, M., Chopra, S., Auli, M., and Zaremba, W. (2015). Sequence level training with recurrent neural networks. arXiv.
2. Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. (2015). Scheduled sampling for sequence prediction with recurrent Neural networks. arXiv.
3. Liu, S., Zhu, Z., Ye, N., Guadarrama, S., and Murphy, K. (2017, January 22–29). Improved Image Captioning via Policy Gradient optimization of SPIDEr. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.100 ↗
4. Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., and Goel, V. (2016). Self-critical Sequence Training for Image Captioning. arXiv.10.1109/CVPR.2017.131 ↗
5. Vedantam, R., Zitnick, C.L., and Parikh, D. (2015). CIDEr: Consensus-based image description evaluation. arXiv.10.1109/CVPR.2015.7299087 ↗
6. Mnih, A., and Gregor, K. (2014). Neural Variational Inference and Learning in Belief Networks. arXiv.
7. Bahdanau, D., Brakel, P., Xu, K., Goyal, A., Lowe, R., Pineau, J., Courville, A., and Bengio, Y. (2016). An actor-critic algorithm for sequence prediction. arXiv.
8. Lin, T., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, C.L. (2014). Microsoft COCO: Common Objects in Context. arXiv.10.1007/978-3-319-10602-1_48 ↗
9. Show and tell: Lessons learned from the 2015 mscoco image captioning challenge10.1109/TPAMI.2016.2587640 ↗
10. Xu, K., Ba, J.L., Kiros, R., Cho, K., Courville, A.C., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. arXiv.
11. Wu, Q., Shen, C., Liu, L., Dick, A.R., and Den Hengel, A.V. (2016). What Value Do Explicit High Level Concepts Have in Vision to Language Problems. arXiv.10.1109/CVPR.2016.29 ↗
12. Yao, T., Pan, Y., Li, Y., Qiu, Z., and Mei, T. (2016). Boosting Image Captioning with Attributes. arXiv.10.1109/ICCV.2017.524 ↗
13. Simple statistical gradient-following algorithms for connectionist reinforcement learning10.1007/BF00992696 ↗
14. Zhang, L., Sung, F., Liu, F., Xiang, T., Gong, S., Yang, Y., and Hospedales, T.M. (2017). Actor-Critic Sequence Training for Image Captioning. arXiv.
15. Zaremba, W., and Sutskever, I. (2015). Reinforcement Learning Neural Turing Machines-Revised. arXiv.
16. Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.10.1109/TNN.1998.712192 ↗
17. Karpathy, A., and Feifei, L. (2015, January 7–12). Deep visual-semantic alignments for generating image descriptions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298932 ↗
18. Xie, S., Girshick, R.B., Dollar, P., Tu, Z., and He, K. (2016). Aggregated Residual Transformations for Deep Neural Networks. arXiv.10.1109/CVPR.2017.634 ↗
19. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
20. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to Sequence Learning with Neural Networks. arXiv.
21. Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
22. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7–12). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
23. Denkowski, M., and Lavie, A. (2014, January 26–27). Meteor Universal: Language Specific Translation Evaluation for Any Target Language. Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, MA, USA.10.3115/v1/W14-3348 ↗
24. Jia, X., Gavves, E., Fernando, B., and Tuytelaars, T. (2015). Guiding the Long-Short Term Memory Model for Image Caption Generation. arXiv.10.1109/ICCV.2015.277 ↗
25. Zhou, L., Xu, C., Koch, P., and Corso, J.J. (2016). Image caption generation with text-conditional semantic attention. arXiv.
26. Yang, Z., Yuan, Y., Wu, Y., Salakhutdinov, R., and Cohen, W.W. (2016). Review Networks for Caption Generation. arXiv.
27. You, Q., Jin, H., Wang, Z., Fang, C., and Luo, J. (2016). Image Captioning with Semantic Attention. arXiv.10.1109/CVPR.2016.503 ↗