journal article·2026-05-07Open access

MambaTech: A Hybrid Mamba–Transformer Architecture with Adaptive Gating for Science and Technology Innovation Time Series Analysis and Pattern Recognition

Yiran Song, Junru Si, Yulin Yang, Zhen Tian

Applied Sciences· Volume 16· Issue 10

Abstract

Quantitative forecasting and lifecycle pattern recognition of science and technology (S&T) innovation activities are essential for evidence-based technology management, yet modeling long-range dependencies in multivariate S&T time series while capturing cross-variate interactions remains a fundamental challenge. Existing Transformer-based forecasters suffer from quadratic complexity that limits effective lookback windows, while state space models typically process each variate independently, ignoring inter-variate coupling among technology fields. We propose MambaTech, a problem-specific hybrid architecture for patent innovation time series that explicitly decomposes intra-domain temporal dynamics from inter-domain technology interactions. The architecture fuses a Mamba-based selective state space branch for O(n)-complexity temporal modeling with a Transformer cross-attention branch for inter-variate association modeling, unified through a learnable domain-conditioned adaptive gating mechanism. An auxiliary classification head regularizes the shared representation by performing lifecycle stage recognition across technology domains. Evaluated on patent filing time series spanning eight CPC sections over 180 months (Harvard USPTO Patent Dataset, 4.5M records) and a 632-variate subclass setting, with additional cross-domain assessment on public benchmarks (Electricity, Traffic, Weather), MambaTech achieves one-step MAPEs of 3.71% (coarse) and 6.24% (fine), reducing errors by 31.4% over Transformer and 52.2% over ARIMA baselines (averaged over five random seeds). At a 12-step horizon, the degradation rate is only +0.48%/step versus +0.89% for the best competing method. The gating fusion yields a 12.8% improvement over naïve concatenation, and the lifecycle classification head attains a macro-F1 of 0.915 across 632 technology domains, with statistically significant improvements under paired bootstrap and Diebold–Mariano tests. Ablation studies confirm the complementary branch contributions, and the learned cross-attention weights reveal technology association patterns consistent with known S&T innovation spillover theory and are validated against external co-classification networks. The originality of MambaTech lies in its task-specific hybridization and analytical framework for patent innovation time series, rather than in proposing a fundamentally new sequence modeling primitive.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Time Series Analysis and Forecasting · Forecasting Techniques and Applications · Intellectual Property and Patents

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

University of Chinese Academy of Sciences, Beijing 100049, China

Institutes of Science and Development, Chinese Academy of Sciences, Beijing 100190, China

Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, UK

References (47)

Show source-provided references
1. United States Patent and Trademark Office (2026, March 31). Patent Technology Monitoring Team Report, Available online: https://www.uspto.gov/web/offices/ac/ido/oeip/taf/reports.htm.
2. Patent Indicators for the Technology Life Cycle Development10.1016/j.respol.2006.12.004 ↗
3. Patent Portfolios for Strategic R&D Planning10.1016/S0923-4748(98)00018-6 ↗
4. Foster, R.N. (1986). Innovation: The Attacker’s Advantage, Summit Books.
5. Exploring the Limits of the Technology S-curve10.1111/j.1937-5956.1992.tb00001.x ↗
6. Technology Life Cycle Analysis Method Based on Patent Documents10.1016/j.techfore.2012.10.003 ↗
7. Identifying Core Technologies Based on Technological Cross-Impacts: An Association Rule Mining and Analytic Network Process Approach10.1016/j.eswa.2011.04.042 ↗
8. Patent Trends as a Technological Forecasting Tool10.1016/0172-2190(83)90134-5 ↗
9. Forecasting Emerging Technologies: Use of Bibliometrics and Patent Analysis10.1016/j.techfore.2006.04.004 ↗
10. Forecasting Emerging Technologies with the Aid of Science and Technology Databases10.1016/j.techfore.2005.09.001 ↗
11. Application of S-shaped Curves10.1016/j.proeng.2011.03.142 ↗
12. Forecasting Emerging Technologies: A Supervised Learning Approach through Patent Analysis10.1016/j.techfore.2017.08.002 ↗
13. A Novel Technology Life Cycle Analysis Method Based on LSTM and CRF10.1007/s11192-024-04946-z ↗
14. Patent Indicators for Monitoring Technology and Knowledge Flows: Examples from NFF and ICT10.1016/j.techfore.2010.06.021 ↗
15. Trajectory Patterns of Technology Fusion: Trend Analysis and Taxonomical Grouping in Nanobiotechnology10.1016/j.techfore.2009.06.006 ↗
16. Gu, A., and Dao, T. (2024, January 7–11). Mamba: Linear-time Sequence Modeling with Selective State Spaces. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
17. Dao, T., and Gu, A. (2024). Transformers Are SSMs: Generalized Models and Efficient Algorithms through Structured State Space Duality. arXiv.
18. Lahoti, A., Li, K.Y., Chen, B., Wang, C., Bick, A., Kolter, J.Z., Dao, T., and Gu, A. (2026, January 23–25). Mamba-3: Improved Sequence Modeling Using State Space Principles. Proceedings of the International Conference on Learning Representations (ICLR), Rio de Janeiro, Brazil.
19. Cai, X., Zhu, Y., Wang, X., and Yao, Y. (2025, January 24–28). MambaTS: Improved Selective State Space Models for Long-Term Time Series Forecasting. Proceedings of the International Conference on Learning Representations (ICLR), Singapore.
20. Xu, X., Chen, C., Liang, Y., Huang, B., Bai, G., Zhao, L., and Shu, K. (2025). SST: Multi-scale Hybrid Mamba-Transformer Experts for Time Series Forecasting. Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM), ACM.10.1145/3746252.3761394 ↗
21. Gu, A., Goel, K., and Ré, C. (2022, January 25–29). Efficiently Modeling Long Sequences with Structured State Spaces. Proceedings of the International Conference on Learning Representations (ICLR), Virtual.
22. Smith, J.T.H., Warrington, A., and Linderman, S.W. (2023, January 1–5). Simplified State Space Layers for Sequence Modeling. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
23. Fu, D.Y., Dao, T., Saab, K.K., Thomas, A.W., Rudra, A., and Ré, C. (2023, January 1–5). Hungry Hungry Hippos: Towards Language Modeling with State Space Models. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
24. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. (2024, January 7–11). iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
25. Are Transformers Effective for Time Series Forecasting?10.1609/aaai.v37i9.26317 ↗
26. Schmookler, J. (1966). Invention and Economic Growth, Harvard University Press.10.4159/harvard.9780674432833 ↗
27. Forecasting Emerging Technologies Using Data Augmentation and Deep Learning
28. Suzgun, M., Melas-Kyriazi, L., Sarkar, S.K., Kominers, S.D., and Shieber, S.M. (2023, January 10–16). The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applications. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, New Orleans, LA, USA.
29. The Artificial Intelligence Patent Dataset (AIPD) 2023 Update10.1007/s10961-025-10189-8 ↗
30. Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., and Xu, C. (2022). CMT: Convolutional Neural Networks Meet Vision Transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR52688.2022.01186 ↗
31. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
32. Liang, W., Shen, J., Zhang, G., Dong, N., Zettlemoyer, L., and Yu, L. (2025, January 24–28). Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity. Proceedings of the International Conference on Learning Representations (ICLR), Singapore.
33. Wei, X., Zhang, T., Li, Y., Zhang, Y., and Wu, F. (2020). Multi-Modality Cross Attention Network for Image and Sentence Matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR42600.2020.01095 ↗
34. Loh, J.Q., Luo, X., Ding, F., Tew, H.H., Loo, J.Y., Ding, Z.Y., Susilawati, S., and Tan, C.P. (2024). Cross-Domain Transfer Learning Using Attention Latent Features for Multi-Agent Trajectory Prediction. arXiv.10.1109/SMC54092.2024.10831990 ↗
35. Gu, A., Dao, T., Ermon, S., Rudra, A., and Ré, C. (2020, January 6–12). HiPPO: Recurrent Memory with Optimal Polynomial Projections. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual.
36. Blelloch, G.E. (1990). Prefix Sums and Their Applications, Carnegie Mellon University. Technical Report CMU-CS-90-190.
37. Godahewa, R., Bergmeir, C., Webb, G.I., Hyndman, R.J., and Montero-Manso, P. (2021). Monash Time Series Forecasting Archive. arXiv.
38. Box, G.E.P., and Jenkins, G.M. (1976). Time Series Analysis: Forecasting and Control, Holden-Day. [Revised ed.].
39. Automatic Time Series Forecasting: The Forecast Package for R10.18637/jss.v027.i03 ↗
40. Macroeconomics and Reality10.2307/1912017 ↗
41. Forecasting at Scale10.1080/00031305.2017.1380080 ↗
42. Random Forests10.1023/A:1010933404324 ↗
43. Chen, T., and Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM.10.1145/2939672.2939785 ↗
44. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
45. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA.
46. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting10.1609/aaai.v35i12.17325 ↗
47. Nie, Y., Nguyen, N.H., Sinthong, P., and Kalagnanam, J. (2023, January 1–5). A Time Series Is Worth 64 Words: Long-term Forecasting with Transformers. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.