journal article·2026-05-12Open access

A Public-Data-Based Multimodal Framework for Plant Growth State Analysis Toward Future Filter-Free Aquaponic Validation

Yina Jeong, Surak Son

Applied Sciences· Volume 16· Issue 10

Abstract

This study proposes the Hydroponic Plant Growth Analysis System (HPGAS), a public-data-based preliminary framework for multimodal plant growth state analysis toward future filter-free aquaponic validation. The HPGAS integrates plant images, water quality signals, and environmental signals to estimate an image-centered growth index, growth stage, and proxy abnormal state probability. Because no public dataset jointly provides plant images, direct growth labels, fish metabolic variables, suspended solids, and nitrification-related measurements from a real filter-free aquaponic system, this study is not a direct operational validation. A two-stage evaluation was conducted using the Autonomous Greenhouse Challenge (AGC), HydroGrowNet, and two aquaponic Internet of Things (IoT) water quality datasets. Stage 1 implemented dataset loaders, image–sensor alignment, proxy label generation, and unimodal and fusion baselines. Stage 2 expanded handcrafted image and sensor-context features and adopted month-wise hold-out evaluation. The image-only model achieved the best growth index regression performance, with a root mean square error (RMSE) of 0.0492 ± 0.0187, whereas the fusion model showed a RMSE of 0.0837 ± 0.0196. Conversely, the fusion model achieved the best proxy abnormal state classification performance, with a F1 score of 0.9695 ± 0.0057 under the clean condition, decreasing to 0.9232 ± 0.0263 under sensor dropout and 0.9132 ± 0.0169 under image noise. Under sensor dropout, the fusion model was more stable than the sensor-only model, whereas under image noise it degraded more than the image-only model. These results indicate that multimodal fusion is most useful for proxy abnormal state classification and robust state interpretation, rather than universally superior scalar growth regression. The HPGAS provides a reproducible baseline for future real filter-free aquaponic experiments, while its operational validity remains to be tested using real filter-free aquaponic data.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Innovations in Aquaponics and Hydroponics Systems · Water Quality Monitoring Technologies · Coastal Management and Development

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Software, College of Engineering, Catholic Kwandong University, Gangneung 25601, Republic of Korea

Funding

Ministry of Education (MOE) and the Gangwon State

References (32)

Show source-provided references
1. FAO (2024). The State of World Fisheries and Aquaculture 2024: Blue Transformation in Action, Food and Agriculture Organization of the United Nations.
2. FAO (2021). AQUASTAT—Water Use, Food and Agriculture Organization of the United Nations. Available online: https://www.fao.org/aquastat/en/overview/methodology/water-use/.
3. Hemming, S., de Zwart, F., Elings, A., Bijlaard, M., van Marrewijk, B., and Petropoulou, A. (2021). 3rd Autonomous Greenhouse Challenge: Online Challenge Lettuce Images, 4TU.ResearchData.
4. Petropoulou, A.S., van Marrewijk, B., de Zwart, F., Elings, A., Bijlaard, M., van Daalen, T., Jansen, G., and Hemming, S. (2023). Lettuce production in intelligent greenhouses—3D imaging and computer vision for plant spacing decisions. Sensors, 23.10.3390/s23062929 ↗
5. Shalash, O., Hassan, N., Métwalli, A., and Elhefny, A. (2026, March 01). HydroGrowNet of Batavia Dataset, Version 5; Mendeley Data: 2025. Available online: https://data.mendeley.com/datasets/g6cm3v3wdp/5.
6. Siswanto, B., Dani, Y., Morika, D., and Mardiyana, B. (2023). A simple dataset of water quality on aquaponic fish ponds based on an internet of things measurement device. Data Brief, 48.10.1016/j.dib.2023.109248 ↗
7. Siswanto, B. (2026, March 01). A Simple Dataset of Aquaponic Fish Pond Water Quality Measurement using Internet of Things Devices, Version 2; Mendeley Data: 2023. Available online: https://data.mendeley.com/datasets/yd36bx6f8f/2.
8. Ardika, I.K. (2026, March 01). Water Quality Parameters for Lettuce Aquaponic System Using IoT Sensors, Version 1; Mendeley Data: 2024. Available online: https://data.mendeley.com/datasets/447dppnf9h/1.
9. Applications, technologies, and evaluation methods in smart aquaponics: A systematic literature review10.1007/s10462-024-11003-x ↗
10. Love, D.C., Fry, J.P., Genello, L., Hill, E.S., Frederick, J.A., Li, X., and Semmens, K. (2014). An International Survey of Aquaponics Prac-titioners. PLoS ONE, 9.10.1371/journal.pone.0102662 ↗
11. Nitrogen Transformations in Aquaponic Systems: A Review10.1016/j.aquaeng.2017.01.004 ↗
12. Hydroponic Systems and Water Management in Aquaponics: A Review10.4081/ija.2017.1012 ↗
13. Nozzi, V., Graber, A., Schmautz, Z., Mathis, A., and Junge, R. (2018). Nutrient Management in Aquaponics: Comparison of Three Ap-proaches for Cultivating Lettuce, Mint and Mushroom Herb. Agronomy, 8.10.3390/agronomy8030027 ↗
14. Goddek, S., Joyce, A., Kotzen, B., and Burnell, G.M. (2019). Nutrient Cycling in Aquaponics Systems. Aquaponics Food Production Systems, Springer.10.1007/978-3-030-15943-6 ↗
15. Aquaponic Trends and Challenges—A Review10.1016/j.jclepro.2019.04.290 ↗
16. Goddek, S., Joyce, A., Kotzen, B., and Burnell, G.M. (2019). Decoupled Aquaponics Systems. Aqua-Ponics Food Production Systems, Springer.10.1007/978-3-030-15943-6 ↗
17. Aquaponic Systems: Biological and Technological Parameters10.1080/13102818.2022.2074892 ↗
18. Lights, Camera, Action: High-Throughput Plant Phenotyping Is Ready for a Close-Up10.1016/j.pbi.2015.02.006 ↗
19. Plant Phenomics, From Sensors to Knowledge10.1016/j.cub.2017.05.055 ↗
20. Ubbens, J.R., and Stavness, I. (2017). Deep Plant Phenomics: A Deep Learning Platform for Complex Plant Phenotyping Tasks. Front. Plant Sci., 8.10.3389/fpls.2017.01190 ↗
21. Digital Image-Based Plant Phenotyping: A Review10.7744/kjoas.2020004 ↗
22. Gang, M.-S., Kim, H.-J., and Kim, D.-W. (2022). Estimation of Greenhouse Lettuce Growth Indices Based on a Two-Stage CNN Using RGB-D Images. Sensors, 22.10.3390/s22155499 ↗
23. Lin, Z., Fu, R., Ren, G., Zhong, R., Ying, Y., and Lin, T. (2022). Automatic Monitoring of Lettuce Fresh Weight by Multi-Modal Fusion Based Deep Learning. Front. Plant Sci., 13.10.3389/fpls.2022.980581 ↗
24. Hou, L., Zhu, Y., Wei, N., Liu, Z., You, J., Zhou, J., and Zhang, J. (2024). Study on Utilizing Mask R-CNN for Phenotypic Estimation of Lettuce’s Growth Status and Optimal Harvest Timing. Agronomy, 14.10.3390/agronomy14061271 ↗
25. 3D Time-Series Phenotyping of Lettuce in Greenhouses10.1016/j.biosystemseng.2025.01.004 ↗
26. Barbedo, J.G.A. (2022). Data Fusion in Agriculture: Resolving Ambiguities and Closing Data Gaps. Sensors, 22.10.3390/s22062285 ↗
27. Maimaitijiang, M., Sagan, V., Sidike, P., Hartling, S., Esposito, F., and Fritschi, F.B. (2020). Soybean Yield Prediction from UAV Using Multimodal Data Fusion and Deep Learning. Remote Sens. Environ., 237.10.1016/j.rse.2019.111599 ↗
28. SMIL: Multimodal Learning with Severely Missing Modality
29. Robust Multimodal Learning With Missing Modalities via Parameter-Efficient Adaptation10.1109/TPAMI.2024.3476487 ↗
30. Yang, Z.-X., Li, Y., Wang, R.-F., Hu, P., and Su, W.-H. (2025). Deep Learning in Multimodal Fusion for Sustainable Plant Care: A Com-prehensive Review. Sustainability, 17.10.3390/su17125255 ↗
31. PhenoNet: A two-stage lightweight deep learning framework for real-time wheat phenophase classification10.1016/j.isprsjprs.2024.01.006 ↗
32. Albahli, S. (2025). AgriFusionNet: A Lightweight Deep Learning Model for Multisource Plant Disease Diagnosis. Agriculture, 15.10.3390/agriculture15141523 ↗