Designing and usability evaluation of multimodal input modalities for visually impaired persons on mobile platforms

Designing and usability evaluation of multimodal input modalities for visually impaired persons on mobile platforms

연구 분야: Analysis

학회: Universal Access in the Information Society

Visually impaired persons use smartphones daily to perform basic actions such as interacting with others through calling, texting, and using social media applications. They require interactions that ease the use of these applications. This paper presents the design of multimodal gestures for visually impaired smartphone users to support them in performing basic tasks and compares their usability with Android accessibility gestures (unimodal design). The paper presents a set of multimodal interactions combining surface, speech, and motion gestures, modelled using the User-Centred Design approach. The design process involved participatory sessions with ten visually impaired smartphone users. The proposed design is developed to replace surface gestures (the current unimodal design) that provide vocal feedback via Talkback functionality. User testing was conducted to evaluate the designed multimodal gestures and to compare their performance with Android accessibility gestures from the point of view of 12 visually impaired smartphone users. Performance metrics (task completion time and the number of errors) and satisfaction metrics (using the System Usability Scale questionnaire) were collected. The usability testing results showed that visually impaired smartphone users perform tasks 3.5 times faster when using the proposed multimodal interactions than unimodal. An average of 87% of the interaction errors made by the users were reduced using the proposed multimodal design compared with the unimodal interactions. In addition, the results showed that 71% of the participants reported that they like to use the designed multimodal interactions compared to the existing Android accessibility gesture, and 66.7% of the participants found it easier and more confident to use than existing unimodal interactions. Hypothesis testing proves a significant difference between unimodal and the proposed multimodal gestures in task completion time, errors rate and satisfaction rate. The study provides insight for future work extending multimodal interactions into various smartphone functions to ease its use for visually impaired users.