Fit for What Purpose? NER Certification of Automatic Captions in English and Spanish

As human and fully automatic live captioning methods coexist and compete against one another, quality analyses and certification become essential. A case in point is LiRICS, the Live Respeaking International Certification Standard created by the Galician Observatory for Media Accessibility (GALMA) to help maintain high international standards in the live captioning profession. Until now, this certification had only been used to assess human captioners. In this paper, it is applied for the first time to automatic captioning (more specifically to Lexi, the automatic software used by the leading captioning company AI-Media) in order to ascertain whether automatic captions have reached an accuracy level that can match that of human captions. After presenting the materials and the methods (NER model), the paper reports on the results of the analysis of Lexi’s English and Spanish automatic captions. With average accuracy rates of 98.56% in English and 98.26% in Spanish, these captions often manage to reach human levels of quality, except when applied to colloquial content featuring several speakers. A final discussion is devoted to a reflection on how automatic and human live captions can coexist as long as the different purposes they serve are considered, namely the access in bulk provided by automatic captions and the curated access offered by human captions.