Can LLMs Generate Coherent Summaries? Leveraging LLM Summarization for Spanish-Language News Articles
Source abstract
Abstract
Automatic summarization is essential for processing the vast quantity of news articles. However, existing methods struggle with factual consistency, hallucinations, and English-centric evaluations. This paper investigates whether Large Language Models can generate coherent and factually grounded summaries of Spanish-language news articles, using the DACSA dataset as a benchmark. Several strategies are evaluated, including zero-shot prompting, one-shot prompting, fine-tuning of seq2seq models mBART and mT5, and a novel bottleneck prompting method that integrates attention-based salience scoring with Named Entity Recognition. Our results show that modern instruction-tuned language models can achieve competitive performance in zero- and one-shot settings, often approaching the performance of fine-tuned baselines. Our proposed bottleneck method enhances factual accuracy and content selection, leading to measurable improvements in ROUGE and BERTScore, especially for larger models such as LLaMA-3.1-70B and Gemma-2-9B. These results suggest that structured prompting can complement conventional approaches, offering an effective and cost-efficient alternative to full supervision. The results indicate that LLMs guided by entity-anchored bottlenecks provide a promising approach to multilingual summarization in domains with limited resources.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Biomedical Text Mining and Ontologies · Text Readability and Simplification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Departamento de Informática y Sistemas, Universidad de Murcia, Campus de Espinardo, 30100 Murcia, Spain
Tecnológico Nacional de México/I.T.S. Teziutlán, Fracción I y II, Teziutlán 73960, Mexico
Funding
Secretary of Science, Humanities, Technology, and Innovation · University of Murcia
References (31)
Show source-provided references
1. Automatic summarization of scientific articles: A survey10.1016/j.jksuci.2020.04.020 ↗
2. Abstractive summarization: An overview of the state of the art10.1016/j.eswa.2018.12.011 ↗
3. Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (2020, January 5–10). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.
4. Exploring the limits of transfer learning with a unified text-to-text transformer
5. Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (2021, January 6–11). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.
6. Zhang, J., Zhao, Y., Saleh, M., and Liu, P.J. (2020, January 13–18). PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. Proceedings of the 37th International Conference on Machine Learning, ICML’20, Virtual.
7. Chen, D., Bolton, J., and Manning, C.D. (2016, January 7–12). A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.10.18653/v1/P16-1223 ↗
8. Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (November, January 31). Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.
9. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The llama 3 herd of models. arXiv.
10. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv.
11. Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., and Rivière, M. (2025). Gemma 3 technical report. arXiv.
12. Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (2020, January 5–10). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.
13. Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (2021, January 6–11). Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.
14. Cohan, A., Feigenblat, G., Freitag, D., Ghosal, T., Herrmannova, D., Knoth, P., Lo, K., Mayr, P., Shmueli-Scheuer, M., and de Waard, A. (2022, January 12–17). Named Entity Inclusion in Abstractive Text Summarization. Proceedings of the Third Workshop on Scholarly Document Processing, Gyeongju, Republic of Korea.
15. Carpuat, M., de Marneffe, M.C., and Meza Ruiz, I.V. (2022, January 10–15). DACSA: A large-scale Dataset for Automatic summarization of Catalan and Spanish newspaper Articles. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, WA, USA.
16. Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv.
17. Jiang, A.Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., Casas, D.d.l., Hanna, E.B., and Bressand, F. (2024). Mixtral of experts. arXiv.
18. Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R.J., Javaheripi, M., and Kauffmann, P. (2024). Phi-4 technical report. arXiv.
19. Wang, Y., Zhang, Z., and Wang, R. (2023, January 9–14). Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-long.482 ↗
20. Benchmarking Large Language Models for News Summarization10.1162/tacl_a_00632 ↗
21. García-Díaz, J.A., Pan, R., and Valencia-García, R. (2023). Leveraging zero and few-shot learning for enhanced model generality in hate speech detection in Spanish and English. Mathematics, 11.10.3390/math11245004 ↗
22. Bernal-Beltrán, T., Paredes-Valverde, M.A., Salas-Zárate, M.d.P., García-Díaz, J.A., and Valencia-García, R. (2025). Sentiment Analysis in Mexican Spanish: A Comparison Between Fine-Tuning and In-Context Learning with Large Language Models. Future Internet, 17.10.3390/fi17100445 ↗
23. Askari, H., Chhabra, A., Chen, M., and Mohapatra, P. (May, January 29). Assessing LLMs for Zero-shot Abstractive Summarization Through the Lens of Relevance Paraphrasing. Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, NM, USA.10.18653/v1/2025.findings-naacl.116 ↗
24. Chiruzzo, L., Ritter, A., and Wang, L. (May, January 29). On Positional Bias of Faithfulness for Long-form Summarization. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, NM, USA.
25. BOE-XSUM: Extreme Summarization in Clear Language of Spanish Legal Decrees and Notifications
26. NoticIA: A Clickbait Article Summarization Dataset in Spanish
27. Cañete, J., Chaperon, G., Fuentes, R., Ho, J.H., Kang, H., and Pérez, J. (2020). Spanish Pre-Trained BERT Model and Evaluation Data. Practical ML for Developing Countries Workshop@ICLR 2020, Springer.
28. Performance comparison: Cloud-based vs. open-source NER for English and Polish10.1177/1088467X241305521 ↗
29. García-Díaz, J.A., Vivancos-Vicente, P.J., Almela, Á., and Valencia-García, R. (2022, January 20–25). UMUTextStats: A linguistic feature extraction tool for Spanish. Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France.
30. B’echard, P., and Ayala, O.M. (2024, January 16–21). Reducing hallucination in structured outputs via Retrieval-Augmented Generation. Proceedings of the North American Chapter of the Association for Computational Linguistics, Mexico City, Mexico.
31. Spanish MTLHateCorpus 2023: Multi-task learning for hate speech detection to identify speech type, target, target group and intensity10.1016/j.csi.2025.103990 ↗