Having a Smarter City through Digital Urban Interfaces: An Evaluation Method
Source abstract
Abstract
This paper appraises a tool developed to evaluate user experiences of urban digital interfaces. The authors propose an evaluation method that uses 14 guidelines to analyze questions pertaining to efficiency, assistance and instructions, content structure, resemblance to reality, feedback interface, visual design, cognitive processes, internationalization, and perceptive access. The proposed tool serves to identify obstacles that once identified can then be tackled and resolved in the design phase. Addressing obstacles in the design phase serves to prevent the creation of inefficient interfaces that would lead to poor user experiences, or, likewise, the rejection of these interfaces by users. To verify the effectiveness of the proposed guidelines in a real-world environment a field study has been conducted in which eight urban interfaces located in different cities and countries were observed. The study reveals the issues typically encountered by users that prevent them from having satisfactory or enjoyable experiences when using digital urban interfaces. The paper concludes by identifying and discussing areas of opportunity for further research and improvements to the proposed guidelines.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Smart Cities and Technologies · Consumer Retail Behavior Studies · Innovative Human-Technology Interaction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Universidad de Monterrey, Nuevo Leon 66238, Mexico
Department of Technics and Projects in Engineering and Architecture, Universidad La Laguna, 38071 Tenerife, Spain
References (76)
Show source-provided references
1. We Are Social Ltd. (2019, March 30). Global Digital Report 2019. Available online: https://wearesocial.com/global-digital-report-2019.
2. Technological Change and Economic Growth10.1016/j.sbspro.2015.06.174 ↗
3. Andrea Vesco Ferrero, F. (2019, June 12). Handbook of Research on Social, Economic, and Environmental Sustainability in the Development of Smart Cities. Available online: https://searchworks.stanford.edu/view/11474955.
4. Developing Smart Cities: An Integrated Framework10.1016/j.procs.2016.07.258 ↗
5. Characteristics of Smart Sustainable City Development: Implications for Project Management10.3390/smartcities1010005 ↗
6. Garcia, R., and Dacko, S. (2015). Design Thinking for Sustainability. Design Thinking, John Wiley & Sons, Inc.10.1002/9781119154273.ch25 ↗
7. Smart Cities: Definitions, Dimensions, Performance, and Initiatives10.1080/10630732.2014.942092 ↗
8. From Multi-Channel Retailing to Omni-Channel Retailing: Introduction to the Special Issue on Multi-Channel Retailing10.1016/j.jretai.2015.02.005 ↗
9. Citizen Apps to Solve Complex Urban Problems10.1080/10630732.2012.673056 ↗
10. Roblek, V., Meško, M., and Krapež, A. (2016). A Complex View of Industry 4.0. SAGE Open, 1–11.10.1177/2158244016653987 ↗
11. Caprotti, F. (2018). Spaces of visibility in the smart city: Flagship urban spaces and the smart urban imaginary. Urban Stud.10.1177/0042098018798597 ↗
12. Smart cities with big data: Reference models, challenges, and considerations10.1016/j.cities.2018.04.011 ↗
13. The Smart City Innovation Ecosystem: A Practical Approach
14. Ballesteros, L.G.M., Alvarez, O., and Markendahl, J. (2015, January 25–28). Quality of Experience (QoE) in the smart cities context: An initial analysis. Proceedings of the 2015 IEEE First International Smart Cities Conference (ISC2), Guadalajara, Mexico.
15. Abbas, M. (2014). Challenges in Implementation of TVM (Ticket Vending Machine) in Developing Countries for Mass Transport System: A Study of Human Behavior while Interacting with Ticket Vending Machine-TVM, Springer.10.1007/978-3-319-07635-5_24 ↗
16. Urban Interfaces: The Cartographies of Screen-Based Installations10.1177/1527476416667818 ↗
17. CrowdSC: Building Smart Cities with Large-Scale Citizen Participation10.1109/MIC.2013.88 ↗
18. Wac, K., Ickin, S., Hong, J.H., Janowski, L., and Fiedler, M. (2011, January 19). Studying the experience of mobile applications used in different contexts of daily life. Proceedings of the First ACM SIGCOMM Workshop on Measurements Up the stack, Toronto, ON, Canada.10.1145/2018602.2018605 ↗
19. Othman, M.K., Sulaiman, M.N., and Aman, S. (2018). Heuristic Evaluation: Comparing Generic and Specific Usability Heuristics for Identification of Usability Problems in a Living Museum Mobile Guide App. Adv. Hum. Comput. Interact., 1–13.10.1155/2018/1518682 ↗
20. Heuristic evaluation: Comparing ways of finding and reporting usability problems10.1016/j.intcom.2006.10.001 ↗
21. Heuristics Considering UX and Quality Criteria for Heuristics
22. Nielsen, J., and Molich, R. (1990, January 1–5). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People—CHI ’90, Seattle, WA, USA.10.1145/97243.97281 ↗
23. Bruce, T. (2019, June 12). Bruce Tognazzini s Tog on Interface—Developing User Interfaces for Microsoft Windows. Available online: http://flylib.com/books/en/2.847.1.19/1/.
24. Cognitive engineering principles for enhancing human-computer performance10.1080/10447319609526147 ↗
25. Connell, I. (2019, June 12). Full Principles Set. Available online: http://www0.cs.ucl.ac.uk/staff/i.connell/DocsPDF/PrinciplesSet.pdf.
26. Sanchez, J. (2019, June 12). Psychological Usability Heuristics|UX Magazine. Available online: http://uxmag.com/articles/psychological-usability-heuristics.
27. Johnson, J. (2014). Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Guidelines, Elsevier.
28. Web Accesibility Initiative (2019, March 01). WAI Guidelines and Techniques|Web Accessibility Initiative (WAI)|W3C. Available online: https://www.w3.org/WAI/standards-guidelines/.
29. Insitituto de Biomecanica de Valencia (2019, June 12). Guía de Recomendaciones Para el Diseño de Mobiliario Ergonómico. Available online: http://www.ibv.org/publicaciones/catalogo-de-publicaciones/ergonomia-y-mueble-guia-de-recomendaciones-para-el-diseno-de-mobiliario-ergonomico.
30. Russo, P., and Boor, S. (1993, January 24–29). How fluent is your interface?: Designing for international users. Proceedings of the INTERCHI ’93 Conference on Human Factors in Computing Systems, Amsterdam, The Netherlands.10.1145/169059.169274 ↗
31. Weinschenk, S. (2019, June 12). The Psychologist’s View of UX Design|UX Magazine. Available online: https://uxmag.com/articles/the-psychologists-view-of-ux-design.
32. Weinschenk, S. (2015). 100 MORE Things Every Designer Needs to Know About People, New Riders.
33. Law, E.L.C., and Hvannberg, E.T. (2004, January 23–27). Analysis of Strategies for Improving and Estimating the Effectiveness of Heuristic Evaluation. Proceedings of the Third Nordic Conference on Human-computer Interaction (NordiCHI ’04), Tampere, Finland.10.1145/1028014.1028051 ↗
34. Martins, A.I., Queirós, A., and Rocha, N.P. (2019). Validation of a usability assessment instrument according to the evaluators’ perspective about the users’ performance. Univers. Access Inf. Soc., 1–11.10.1007/s10209-019-00659-w ↗
35. Sharon, T. (2016). Validating Product Ideas, Rosenfeld Media.
36. Marsh, S. (2018). User Research: A Practical Guide to Designing Better Products and Services, Kogan Page.
37. Rohrer, C. (2019, June 16). When to Use Which User-Experience Research Methods. Available online: https://www.nngroup.com/articles/which-ux-research-methods/.
38. Laufer, D., Burnette, A., Costa, T., and Hogan, A. (2019, June 12). The Digital Customer Experience Improvement Playbook for 2019. Available online: https://www.forrester.com/playbook/The+Digital+Customer+Experience+Improvement+Playbook+For+2019/-/E-PLA130.
39. Rodden, K., Hutchinson, H., and Fu, X. (2010, January 10–15). Measuring the User Experience on a Large Scale: User-centered Metrics for Web Applications. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10), Atlanta, GA, USA.10.1145/1753326.1753687 ↗
40. RITE+Krug: A Combination of Usability Test Methods for Agile Design
41. Adapting Usability Investigations for Agile User-centered Design
42. Sheppard, B., Sarrazin, H., Kouyoumjian, G., and Dore, F. (2019, June 12). The Business Value of Design. Available online: https://www.mckinsey.com/business-functions/mckinsey-design/our-insights/the-business-value-of-design.
43. Persson, M., Grundstrom, C., and Väyrynen, K. (2018, January 17–20). A case for participatory practices in the digital transformation of insurance. Proceedings of the 31st Bled Econference: Digital Transformation: Meeting the Challenges, Bled, Slovenia.10.18690/978-961-286-170-4.29 ↗
44. MacDonald, C.M. (2019, January 23–28). User Experience (UX) Capacity-Building: A Conceptual Model and Research Agenda. Proceedings of the 2019 on Designing Interactive Systems Conference (DIS ’19), San Diego, CA, USA.10.1145/3322276.3322346 ↗
45. Ruud, O. (2017, January 4–5). Successful digital transformation projects in public sector with focus on municipalities (research in progress). Proceedings of the Central and Eastern European e|Dem and e|Gov Days 2017, Budapest, Hungary.
46. A four-step blueprint for digital reinvention10.1108/SL-06-2016-0042 ↗
47. Bennett, D., Pérez-Bustamante, D., and Medrano, M.L. (2017). Challenges for Smart Cities in the UK. Sustainable Smart Cities: Creating Spaces for Technological, Social and Business Development, Springer.10.1007/978-3-319-40895-8_1 ↗
48. Oliveira, A., and Campolargo, M. (2015, January 5–8). From Smart Cities to Human Smart Cities. Proceedings of the 48th Hawaii International Conference on System Sciences, Kauai, HI, USA.10.1109/HICSS.2015.281 ↗
49. Cognitive Walkthroughs: A Method for Theory-based Evaluation of User Interfaces10.1016/0020-7373(92)90039-N ↗
50. Farkas, D., and Nunnally, B. (2016). UX Research, O’Reilly Media.
51. Optimal Workshop (2019, June 30). Reframer. Available online: https://www.optimalworkshop.com/reframer.
52. Micheaux, A. (2018). Customer Journey Mapping as a New Way to Teach Data-Driven Marketing as a Service. J. Mark. Educ.10.1177/0273475318812551 ↗
53. Time and Cognitive Load in Working Memory10.1037/0278-7393.33.3.570 ↗
54. Sandnes, F. (2010). User Interface Design for Public Kiosks: An Evaluation of the Taiwan High Speed Rail Ticket Vending Machine. J. Inf. Sci. Eng., 307–321. Available online: https://www.researchgate.net/publication/220587882_User_Interface_Design_for_Public_Kiosks_An_Evaluation_of_the_Taiwan_High_Speed_Rail_Ticket_Vending_Machine.
55. Hernández Sampieri, R., Fernández Collado, C., and Baptista Lucio, P. (2014). Metodología de la Investigación. Quinta Edicion, McGraw-Hill.
56. Nielsen, J. (2018, December 03). Why You Only Need to Test with 5 Users. Available online: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.
57. Turner, C.W., Lewis, J.R., and Nielsen, J. (2006). Determining Usability Test Sample Size. International Encyclopedia of Ergonomics and Human Factors, CRC Press.
58. Personas and user-centered design: How can personas benefit product design processes?10.1016/j.destud.2011.03.003 ↗
59. Nielsen, L. (2019). Personas—User Focused Design (Human–Computer Interaction Series), Springer International Publishing.10.1007/978-1-4471-7427-1 ↗
60. Ethnographic Research 2.010.1108/JOE-01-2012-0008 ↗
61. Reeves, S., Kuper, A., and Hodges, B.D. (2008). Qualitative research methodologies: Ethnography. BMJ, 337.10.1136/bmj.a1020 ↗
62. Spool, J.M., and Schroeder, W. (April, January 31). Testing web sites: Five users is nowhere near enough. Proceedings of the CHI ’01 Extended Abstracts on Human Factors in Computing Systems, Seattle, WA, USA.
63. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing10.3758/BF03195514 ↗
64. Conducting Iterative Usability Testing on a Web Site: Challenges and Benefits
65. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?10.1177/001872089203400407 ↗
66. Nielsen, J. (2019, June 15). Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier. Available online: https://www.nngroup.com/articles/guerrilla-hci/.
67. Estimating Sample Size for Usability Testing
68. Guest, G.S., Namey, E.M., and Mitchell, M.L. (2012). Collecting Qualitative Data: A Field Manual for Applied Research, SAGE Publications, Inc.10.4135/9781506374680 ↗
69. Clustering Methods with Qualitative Data: A Mixed Methods Approach for Prevention Research with Small Samples10.1007/s11121-015-0561-z ↗
70. Watkins, D.C. (2017). Rapid and Rigorous Qualitative Data Analysis: The “RADaR” Technique for Applied Research. Int. J. Qual. Methods, 16.10.1177/1609406917712131 ↗
71. The struggle for digital inclusion: Phones, healthcare, and marginalisation in rural India10.1016/j.worlddev.2017.12.023 ↗
72. Ghosal, A., and Halder, S. (2018). Building Intelligent Systems for Smart Cities: Issues, Challenges and Approaches. Smart Cities, Springer.10.1007/978-3-319-76669-0_5 ↗
73. The Keystroke-Level Model for User Performance Time with Interactive Systems10.1145/358886.358895 ↗
74. Card, S.K., Moran, T.P., and Newell, A. (1986). The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates.
75. Fox, J.E. (2015, January 1–3). The Science of Usability Testing. Proceedings of the 2015 Federal Committee on Statistical Methodology (FCSM) Research Conference, Washington, DC, USA.
76. Examination of the Use of Nielsen’s 10 Usability Heuristics & Outlooks for the Future10.1177/1541931213601853 ↗