Multilingual Text Summarization in Healthcare Using Pre-Trained Transformer-Based Language Models
Source abstract
Abstract
We analyze the suitability of existing pre-trained transformer-based language models (PLMs) for abstractive text summarization on German technical healthcare texts. The study focuses on the multilingual capabilities of these ... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Computational and Text Analysis Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (45)
Show source-provided references
1. Armengol-Estapé J, de Gibert Bonet O, Melero M. On the multilingual capabilities of very large-scale English language models. arXiv:2108.13349. 2021.
2. Extraction and analysis of text patterns from NSFW adult content in Reddit10.1016/j.datak.2022.101979 ↗
3. Zhang X, Li S, Hauer B, Shi N, Kondrak G. Don’t trust ChatGPT when your question is not in English: a study of multilingual abilities and types of LLMs. arXiv:2305.16339. 2023.10.18653/v1/2023.emnlp-main.491 ↗
4. BC4LLM: a perspective of trusted artificial intelligence when blockchain meets large language models10.1016/j.neucom.2024.128089 ↗
5. Jiao J, Afroogh S, Xu Y, Phillips C. Navigating LLM ethics: advancements, challenges, and future directions. arXiv:2406.18841. 2024.10.1007/s43681-025-00814-5 ↗
6. Min B, Ross H, Sulem E, Ben Veyseh AP, Nguyen TH, Sainz O, et al. Recent advances in natural language processing via large pre-trained language models: a survey. arXiv:2111.01243. 2021.
7. Pretraining-based natural language generation for text summarization
8. Survey on automatic text summarization and transformer models applicability
9. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. arXiv:1706.03762. 2017.
10. Zaczynska K, Feldhus N, Schwarzenberg R, Gabryszak A, Möller S. Evaluating German transformer language models with syntactic agreement tests. arXiv:2007.03765. 2020.
11. BERT: pre-training of deep bidirectional transformers for language understanding
12. How multilingual is multilingual BERT?
13. Abstract text summarization: a low resource challenge
14. Scialom T, Dray P-A, Lamprier S, Piwowarski B, Staiano J. MLSUM: the multilingual summarization corpus. arXiv:2004.14900. 2020.10.18653/v1/2020.emnlp-main.647 ↗
15. Summarization corpora of wikipedia articles
16. Recent automatic text summarization techniques: a survey10.1007/s10462-016-9475-9 ↗
17. Lin · Text summarization branches out · 2004 · 74
18. The challenging task of summary evaluation: an overview10.1007/s10579-017-9399-2 ↗
19. SUMMAC: a text summarization evaluation10.1017/S1351324901002741 ↗
20. Evaluating information content by factoid analysis: human annotation and stability
21. Evaluating content selection in summarization: the pyramid method
22. An exploratory study of automatic text summarization in biomedical and healthcare domain10.1016/j.health.2022.100058 ↗
23. Domain-aware abstractive text summarization for medical documents
24. Towards an efficient approach for automatic medical document summarization
25. Moradi M, Samwald M. Clustering of deep contextualized representations for summarization of biomedical texts. arXiv:1908.02286. 2019.
26. German medical natural language processing—a data-centric survey
27. Adapting large language models for automated summarisation of electronic medical records in clinical coding
28. Pre-trained language models in medicine: a survey10.1016/j.artmed.2024.102904 ↗
29. Schubiger R. German summarization with large language models [master’s thesis]. Zurich, Switzerland: ETH Zurich; 2024.
30. Zhang H, Yu PS, Zhang J. A systematic survey of text summarization: from statistical methods to large language models. arXiv:2406.11289. 2024.10.1145/3731445 ↗
31. Liddy ED. Natural language processing [Internet]. 2001 [cited 2025 Mar 3]. Available from: https://surface.syr.edu/cgi/viewcontent.cgi?article=1043&context=istpub.
32. Comparative study of text summarization methods
33. Liu Y. Fine-tune BERT for extractive summarization. arXiv:1903.10318. 2019.
34. A survey on abstractive text summarization
35. Bird S, Klein E, Loper E. Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media [Internet]. 2009 [cited 2025 Mar 3]. Available from: https://www.nltk.org/book/.
36. Subramanyam Kalyan K, Rajasekharan A, Sangeetha S. AMMUS : a survey of transformer-based pretrained models in natural language processing [Internet]. 2001 [cited 2025 Mar 3]. Available from: https://mr-nlp.github.io.
37. Transformer-XL: attentive language models beyond a fixed-length context
38. Exploring the limits of transfer learning with a unified text-to-text transformer
39. aphasie suisse | Home. Retrieved January 31, 2023. [Internet]. [cited 2025 Mar 3]. Available from: https://aphasie.org/.
40. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics
41. Revisiting readability: a unified framework for predicting text quality
42. OpenAI API. Retrieved 2023 Jan 20. [Internet]. [cited 2025 Mar 3]. Available from: https://openai.com/api/.
43. Hugging Face—The AI community building the future. Retrieved January 20, 2023. [Internet]. [cited 2025 Mar 3]. Available from: https://huggingface.co/.
44. Cachola I, Lo K, Cohan A, Weld DS. TLDR: extreme summarization of scientific documents [Internet]. 2020 [cited 2025 Mar 3]. Available from: https://github.com/allenai/scitldr.10.18653/v1/2020.findings-emnlp.428 ↗
45. Schmitter P, Kirecci I, Gatziu Grivas S, Hanne T, Beck C. Transformation Compass für nicht-medizinische Supportprozesse (DE) [Internet]. 2003 [cited 2025 Mar 3]. Available from: https://digitalcollection.zhaw.ch/items/4cf03e1e-6ad0-45b4-8477-86fdf681a416.