Multimodal Network for Object Detection Using Channel Adjustment and Multi-Scale Attention

Object detection benefits greatly from multimodal image fusion, which integrates complementary data from different modalities like RGB and thermal images. However, existing methods struggle with effective inter-modal fusion, particularly in capturing spatial and contextual information across diverse regions and scales. To address these limitations, we propose the dynamic channel adjustment and multi-scale activated attention mechanism network (MNCM). Our approach incorporates dynamic channel adjustment for precise feature fusion across modalities and a multi-scale attention mechanism to capture both local and global contexts. This design improves robustness while balancing computational efficiency. The model’s scalability is enhanced through its ability to adaptively process multi-scale information without being constrained by fixed-scale designs. To validate our method, we used two multimodal datasets from traffic and industrial scenarios, which consisted of paired thermal infrared and visible light images. The results first demonstrate strong performance in multimodal fusion and then show state-of-the-art results in object detection, proving its effectiveness for real-world applications.