A novel meta-optimization (ALA) CatBoost residual modeling scheme for error compensation in the visible light band of multi-channel spectral sensors

In practical applications, spectral sensors are affected by dark current drift, near-infrared (NIR) crosstalk, and non-ideal optical filter characteristics. This study proposes a two-stage error compensation method. First, errors caused by dark current and NIR crosstalk are compensated. Then, the compensated data is calibrated using a multivariate linear matrix to calibrate the spectral sensor’s eight-channel visible light counts to the spectral irradiance of each wavelength from 380 to 780 nm. Finally, to address the crosstalk problem caused by the non-ideal transmittance of the optical filter, an error compensation model based on PCA and a meta-heuristic ALA-CatBoost algorithm is constructed. First, the principal components of the calibration spectra are extracted and combined with sensor data as model features, and the CatBoost hyperparameters are optimized by the Artificial Lemming Algorithm (ALA). Experiments show that the proposed ALA-CatBoost model performs excellently: RMSE decreases from \break3.12×10−2mw∗m−2∗nm−1 to 6.63×10−3mw∗m−2∗nm−1 , MAE from 1.78×10−2mw∗m−2∗nm−1 to 1.15×10−3mw∗m−2∗nm−1, and R² reaches 0.9468. Compared with ALA-XGBoost, ALA-LightGBM, ALA-GPR, and ALA-SVR, the ALA-CatBoost model achieves the strongest spectral residual suppression. In addition, two high-performance meta-heuristic algorithms, the Sparrow Search Algorithm and the Seagull Optimization Algorithm, are used for further comparison, confirming the effectiveness of the proposed method.