Comparison of Image Normalization Methods for Multi-Site Deep Learning

In this study, we evaluate the influence of normalization on the performance of deep learning networks for tumor segmentation and the prediction of the pathological response of locally advanced rectal cancer to neoadjuvant chemoradiotherapy. The techniques were applied to a multicenter and multimodal magnet resonance imaging data set consisting of 201 patients recorded at six centers. We implemented and investigated six different normalization methods (setting the mean and standard deviation, histogram matching, percentiles, combining percentiles and histogram matching, fixed window and an auto-encoder with adversarial loss using the imaging parameters) and evaluated their impact on four deep learning tasks: tumor segmentation, prediction of treatment outcome, and prediction of sex and age. The latter two tasks were implemented as a reference test. We trained a modified U-Net with different normalization methods in multiple configurations: on all images, images from all centers except one, and images from a single center. Our results show that normalization only plays a minor role in segmentation, with a difference in Dice of less than 0.02 between the best and worst performing networks. For the prediction of sex and treatment outcomes, the percentile method combined with histogram matching works best for all scenarios. The biggest difference in performance, depending on the normalization method, occurs for classification. In conclusion, normalization is especially important for small data sets or for generalizing to different data distributions. The deep learning method was superior to the classical methods only in a minority of cases, probably due to the limited amount of training data.