To support older adults in the context of autonomous public transportation, we designed an intuitive voice assistant and brought the human-centered design process to life by testing and validating HCI prototypes through iterative, scientifically grounded evaluation methods.
Our project started with a competitive analysis of existing solutions in the field of autonomous buses, with a special focus on senior citizens who have limited technical experience. It became evident early on that our concept occupied a novel space, as only very few AI-supported systems existed that targeted this specific user group. After the research phase, we created a low-fidelity paper prototype for AIVA and tested it in a formative user study with seven participants. Using a scenario-based cognitive walkthrough and the think-aloud method, we were able to observe user interactions in real time and collect qualitative data about expectations and mental models.The tests revealed weaknesses in the dashboard’s information architecture and icon design, which we refined in the next iteration. Additionally, the sample demonstrated a generally low level of trust in speech-based interaction with AI, a factor that is particularly critical for this target group.
Based on the first round of testing and user feedback, we developed a high-fidelity prototype and carried out a second usability study with 14 participants. The tests included four core tasks that allowed us to directly compare speech interaction with manual input via the on-screen keyboard.
For the summative evaluation, we used standardized questionnaires:
The collected data was split into two age groups (>60 and <=60) and analyzed in Microsoft Excel using the official metrics. For visualization, I used R Studio, which allowed me to expand my skills with the tool and get a better sense of its possibilities. Before this project, I had primarily worked with JMP and python.
Our results showed SUS scores indicated usability that was close to excellent.
The NASA-TLX results revealed that the task “You are unsure how to use the system – what do you do?” caused a high cognitive load (40.1), especially among older users. As the main reason users verbally commented their confusion between the “Help” icon and the “Voice Assistant” icon.
Manual destination entry showed a lower workload (9.0) compared to voice control (22.0). The UEQ results indicated an overall positive evaluation (0.9), with pragmatic quality (1.12) scoring higher than hedonic quality (0.66). Older participants had difficulty understanding the concept of a prototype, which points to the need for clearer briefing protocols.
Our data-driven approach provided valuable insights into user experience and the challenges involved in interacting with speech-based conversational agents, especially in autonomous vehicles. Close collaboration within the team, combined with research, visual design, and testing, allowed us to develop a prototype that directly addresses the needs of our target group.
