Mispronunciation Detection and Diagnosis for Non-Native Korean Learners Using Iterative Pseudo-Label Refinement Based on Self-Supervised Learning

Accurate Mispronunciation Detection and Diagnosis (MDD) for non-native Korean learners is critical for effective pronunciation feedback, but it is hindered by the lack of training labels that reflect learners’ actual pronunciations. This paper presents a pseudo-label generation framework that fine-tunes Whisper to output pronunciation-oriented sequences, supported by data-quality management and iterative label refinement. We convert orthographic transcripts into pronunciation targets using an existing Grapheme-to-Phoneme (G2P) tool to reduce reliance on standard written forms, and apply multi-stage refinement with cross-model agreement validation under progressively adjusted thresholds to filter unreliable pseudo-labels. We further improve robustness by incorporating larger and more diverse non-native speech corpora and by applying dataset-specific preprocessing, including noise removal, duration-based selection, and duplicate control. Evaluation on a manually annotated test set of actual learner pronunciations shows that models trained with refined pseudo-labels achieved a lower Phoneme Error Rate (PER) and performed better than the baseline model on MDD. Overall, the proposed framework enables practical MDD for non-native Korean speech without requiring large-scale manual phoneme annotation.