Optimized Deep Feature Learning with Hybrid Ensemble Soft Voting for Early Breast Cancer Histopathological Image Classification

Breast cancer is among the leading causes of cancer mortality globally, and its diagnosis through histopathological image analysis is often prone to inter-observer variability and misclassification. Existing machine learning (ML) methods struggle with intra-class heterogeneity and inter-class similarity, necessitating more robust classification models. This study presents an ML classifier ensemble hybrid model for deep feature extraction with deep learning (DL) and Bat Swarm Optimization (BSO) hyperparameter optimization to improve breast cancer histopathology (BCH) image classification. A dataset of 804 Hematoxylin and Eosin (H&E) stained images classified as Benign, in situ, Invasive, and Normal categories (ICIAR2018_BACH_Challenge) has been utilized. ResNet50 was utilized for feature extraction, while Support Vector Machines (SVM), Random Forests (RF), XGBoosts (XGB), Decision Trees (DT), and AdaBoosts (ADB) were utilized for classification. BSO was utilized for hyperparameter optimization in a soft voting ensemble approach. Accuracy, precision, recall, specificity, F1-score, Receiver Operating Characteristic (ROC), and Precision-Recall (PR) were utilized for model performance metrics. The model using an ensemble outperformed individual classifiers in terms of having greater accuracy (~90.0%), precision (~86.4%), recall (~86.3%), and specificity (~96.6%). The robustness of the model was verified by both ROC and PR curves, which showed AUC values of 1.00, 0.99, and 0.98 for Benign, Invasive, and in situ instances, respectively. This ensemble model delivers a strong and clinically valid methodology for breast cancer classification that enhances precision and minimizes diagnostic errors. Future work should focus on explainable AI, multi-modal fusion, few-shot learning, and edge computing for real-world deployment.