DMSCNet: A Dilated Multi-Scale Contrastive Attention Network for Sensor-Based Human Activity Recognition
Source abstract
Abstract
Wearable-sensor human activity recognition (HAR) plays a key role in health monitoring, elderly care, and human–computer interaction. Deep learning dominates the field, but two limitations remain. CNNs with fixed kernels cannot capture cross-scale temporal events such as gait cycles and postural transitions in a single layer, and softmax attention on small sensor datasets is often diluted by common-mode background responses across the sequence. We propose DMSCNet, an end-to-end framework with two modules. The Dilated Multi-Scale Branch Block (DMSB) combines a shared bottleneck, parallel dilated convolutions, a pooling bypass, and SE-based channel recalibration to widen the temporal receptive field under a controlled parameter budget. The Contrastive Temporal Attention (CTA) module adopts a dual-path differential design, in which the two paths learn overlapping but non-identical attention patterns and their subtraction suppresses shared low-level responses while preserving the discriminative positions each path locks onto, encoded with opposite signs. DMSB and CTA are cascaded into a DMSC Block and stacked residually. On UCI-HAR, USC-HAD, and RealWorld, DMSCNet reaches F1-scores of 97.65%, 91.80%, and 99.05%, outperforming nine baselines. Ablations confirm that SE acts along the channel axis and CTA along the temporal axis, and visualization reveals a dynamic–static dichotomy together with a signed bipolar encoding pattern produced by the dual-path subtraction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Context-Aware Activity Recognition Systems · Human Pose and Action Recognition · Gait Recognition and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Shandong Province of China
References (43)
Show source-provided references
1. A Survey on Human Activity Recognition Using Wearable Sensors10.1109/SURV.2012.110112.00192 ↗
2. A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors10.1145/2499621 ↗
3. Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2012, January 3–5). Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine. Proceedings of the International Workshop on Ambient Assisted Living (IWAAL), Vitoria-Gasteiz, Spain.10.1007/978-3-642-35395-6_30 ↗
4. Human Activity Recognition with Smartphone Sensors Using Deep Learning Neural Networks10.1016/j.eswa.2016.04.032 ↗
5. Ordóñez, F.J., and Roggen, D. (2016). Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors, 16.10.3390/s16010115 ↗
6. Wang, Z., Yan, W., and Oates, T. (2017, January 14–19). Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966039 ↗
7. InceptionTime: Finding AlexNet for Time Series Classification10.1007/s10618-020-00710-y ↗
8. InnoHAR: A Deep Neural Network for Complex Human Activity Recognition10.1109/ACCESS.2018.2890675 ↗
9. Bai, S., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv.
10. TCN-Inception: Temporal Convolutional Network and Inception Modules for Sensor-Based Human Activity Recognition10.1016/j.future.2024.06.016 ↗
11. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
12. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 14–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
13. Yuan, Y., Xun, G., Ma, F., Wang, Y., Du, N., Jia, K., Su, L., and Zhang, A. (2018, January 17–20). MuVAN: A Multi-View Attention Network for Multivariate Temporal Data. Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore.10.1109/ICDM.2018.00087 ↗
14. Two-Stream Convolution Augmented Transformer for Human Activity Recognition
15. Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., and Alshurafa, N. (2022). Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances. Sensors, 22.10.3390/s22041476 ↗
16. Toward Human Activity Recognition: A Survey10.1007/s00521-022-07937-4 ↗
17. A Comprehensive Survey on Deep Learning Methods in Human Activity Recognition10.3390/make6020040 ↗
18. A Survey on Deep Learning for Human Activity Recognition10.1145/3472290 ↗
19. Yang, J., Nguyen, M.N., San, P.P., Li, X., and Krishnaswamy, S. (2015, January 25–31). Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), Buenos Aires, Argentina.
20. Yao, S., Hu, S., Zhao, Y., Zhang, A., and Abdelzaher, T. (2017, January 3–7). DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. Proceedings of the 26th International Conference on World Wide Web (WWW), Perth, Australia.10.1145/3038912.3052577 ↗
21. Attention Is All You Need
22. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
23. Mahmud, S., Tonmoy, M.T.H., Bhaumik, K.K., Rahman, A.K.M.M., Amin, M.A., Shoyaib, M., Khan, M.A.H., and Ali, A.A. (September, January 29). Human Activity Recognition from Wearable Sensor Data Using Self-Attention. Proceedings of the 24th European Conference on Artificial Intelligence (ECAI), Santiago de Compostela, Spain.
24. Dirgová Luptáková, I., Kubovčík, M., and Pospíchal, J. (2022). Wearable Sensor-Based Human Activity Recognition with Transformer Model. Sensors, 22.10.20944/preprints202202.0111.v1 ↗
25. TASKED: Transformer-Based Adversarial Learning for Human Activity Recognition Using Wearable Sensors via Self-Knowledge Distillation10.1016/j.knosys.2022.110143 ↗
26. A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition10.1109/JSEN.2023.3242603 ↗
27. Mekruksavanich, S., and Jitpattanakul, A. (2024). Device Position-Independent Human Activity Recognition with Wearable Sensors Using Deep Neural Networks. Appl. Sci., 14.10.3390/app14052107 ↗
28. A Novel CNN-based Bi-LSTM Parallel Model with Attention Mechanism for Human Activity Recognition with Noisy Data10.1038/s41598-022-11880-8 ↗
29. A Hybrid LSTM-CNN Model with Efficient Channel Attention for Enhanced Human Activity Recognition Using Wearable Sensors10.1007/s42452-025-07896-0 ↗
30. Child, R., Gray, S., Radford, A., and Sutskever, I. (2019). Generating Long Sequences with Sparse Transformers. arXiv.
31. Katharopoulos, A., Vyas, A., Pappas, N., and Fleuret, F. (2020, January 12–18). Transformers Are RNNs: Fast Autoregressive Transformers with Linear Attention. Proceedings of the 37th International Conference on Machine Learning (ICML), PMLR, Vienna, Austria.
32. Ye, T., Dong, L., Xia, Y., Sun, Y., Zhu, Y., Huang, G., and Wei, F. (2025, January 24–28). Differential Transformer. Proceedings of the International Conference on Learning Representations (ICLR), Singapore.
33. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
34. Lin, M., Chen, Q., and Yan, S. (2014, January 14–16). Network in Network. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada.
35. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer Normalization. arXiv.
36. Yu, F., and Koltun, V. (2016, January 2–4). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, Puerto Rico.
37. Zhang, B., and Sennrich, R. (2019, January 8–14). Root Mean Square Layer Normalization. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada.
38. Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., and Liu, T. (2020, January 12–18). On Layer Normalization in the Transformer Architecture. Proceedings of the 37th International Conference on Machine Learning (ICML), PMLR, Vienna, Austria.
39. Hendrycks, D., and Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). arXiv.
40. Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24–26). A Public Domain Dataset for Human Activity Recognition Using Smartphones. Proceedings of the 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium.
41. Zhang, M., and Sawchuk, A.A. (2012, January 5–8). USC-HAD: A Daily Activity Dataset for Ubiquitous Activity Recognition Using Wearable Sensors. Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp), Pittsburgh, PA, USA.10.1145/2370216.2370438 ↗
42. Sztyler, T., and Stuckenschmidt, H. (2016, January 14–19). On-Body Localization of Wearable Devices: An Investigation of Position-Aware Activity Recognition. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, Australia.10.1109/PERCOM.2016.7456521 ↗
43. Transformer-based models to deal with heterogeneous environments in Human Activity Recognition10.1007/s00779-023-01776-3 ↗