Automated Detection of Deceptive Design Patterns on University Websites: A Comparative Analysis of Browser-Based Tools and LLM-Based Approaches

Although recent years have seen an emergence of tools for automated identification of deceptive design patterns on websites, their scope and reliability remain understudied. Institutional websites are a particularly interesting research domain. They have an extensive information structure and shape the conditions of user interaction. The purpose of the article is to empirically evaluate signs of deceptive design patterns on Polish universities’ websites and analyse how they are identified using automated analytical tools. The study covers all public universities in Poland (N = 65). The analysis involved automated tools representing different methodological underpinnings, including web browser extensions and GPT language model-based analytical procedures. The study pinpoints significant differences in the paradigms behind the results provided by the two methods. Browser extensions yielded only qualitative suggestions of potential problems. They did not generate complete and comparable quantitative results for the entire population of the investigated websites. Results from the heuristic Real-Time Deceptive Pattern Auditor (RTDPA) were highly concentrated (mean 90.03, median 90, SD = 1.55, and interval 80–95), which may suggest limited discriminatory power for this website collection. In contrast, the rule-based Structural Interface Risk Screening (SIRS) revealed a much greater differentiation of results (mean 89.23, median 90, SD = 11.50, and interval 50–100). The association between the results from the two procedures was very weak (r = 0.089, p ≈ 0.48), which indicates their limited quantitative comparability. These findings indicate that the current capabilities of automated tools offer merely fragmented detection of selected deceptive design patterns, instead of a complete systemic diagnosis of the problem. Although they are measurement tools by declaration, the solutions can offer only preliminary screening, flagging potential risk areas. Not differentiating between risk signalling and actual measurement may lead to the illusion of automated precision.