PCATNet: Position-Class Awareness Transformer for Image Captioning
Source abstract
Abstract
Existing image captioning models usually build the relation between visual information and words to generate captions, which lack spatial information and object classes. To address the issue, we propose a novel Position-Class... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Multimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (42)
Show source-provided references
1. Baby talk: Understanding and generating simple image descriptions
2. Meta captioning: A meta learning based remote sensing image captioning framework10.1016/j.isprsjprs.2022.02.001 ↗
3. Visual cluster grounding for image captioning
4. ImageNet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
5. Bottleneck Transformers for visual recognition
6. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai et al., “An image is worth 16 × 16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2010.
7. Faster R-CNN: Towards real-time object detection with region proposal networks
8. End-to-end object detection with transformers
9. Sequence to sequence learning with neural networks
10. K. Cho, B. V. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares et al., “Learning phrase representations using RNN Encoder-Decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014.10.3115/v1/D14-1179 ↗
11. Recurrent fusion network for image captioning
12. Attention is all you need
13. J. Devlin, M. W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
14. Bottom-up and top-down attention for image captioning and visual question answering
15. Knowing when to look: Adaptive attention via a visual sentinel for image captioning
16. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning
17. Context-aware visual policy network for fine-grained image captioning10.1109/TPAMI.2019.2909864 ↗
18. VinVL: Revisiting visual representations in vision-language models
19. RSTNet: Captioning with adaptive attention on visual and non-visual words
20. X-Linear attention networks for image captioning
21. Effective multimodal encoding for image paragraph captioning
22. Meshed-memory Transformer for image captioning
23. Attention-guided image captioning through word information
24. Rethinking and improving relative position encoding for vision Transformer
25. Oscar: Object-semantics aligned pre-training for vision-language tasks
26. Show and tell: A neural image caption generator
27. Convolutional sequence to sequence learning
28. P. Shaw, J. Uszkoreit and A. Vaswani, “Self-attention with relative position representations,” arXiv preprint arXiv:1803.02155, 2018.10.18653/v1/N18-2074 ↗
29. Self-critical sequence training for image captioning
30. CIDEr: Consensus-based image description evaluation
31. Common objects in context
32. Deep visual-semantic alignments for generating image descriptions
33. BLEU: A method for automatic evaluation of machine translation
34. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments
35. SPICE: Semantic propositional image caption evaluation
36. ROUGE: A package for automatic evaluation of summaries
37. Objects365: A large-scale, high-quality dataset for object detection
38. The open images dataset v410.1007/s11263-020-01316-z ↗
39. Visual Genome: Connecting language and vision using crowdsourced dense image annotations10.1007/s11263-016-0981-7 ↗
40. Attention on attention for image captioning
41. Position-aware image captioning with spatial relation10.1016/j.neucom.2022.05.003 ↗
42. End-to-end Transformer based model for image captioning