A rule-based multimodal computer-access system for upper-limb accessibility
Design and preliminary usability evaluation of the Turkish voice-command module
DOI:
https://doi.org/10.17411/jacces.v16i2.727Abstract
Computer access can be difficult for people who cannot use conventional mouse-and-keyboard input effectively. This study documents the design of a rule-based multimodal computer-access prototype comprising a Turkish voice-command module and a face/head-motion cursor-control module, and reports a preliminary usability evaluation limited to the voice-command component. Before development, formative consultations were conducted with two individuals with upper-limb impairments, one individual with a visual impairment, and six non-disabled stakeholders. After development, 25 non-disabled proxy participants completed a predefined scenario of 19 common computer tasks using the Turkish voice-command module. Task completion, completion time, facilitator assistance, and descriptive post-scenario feedback were recorded. All participants reached the predefined outcome for every task within an assisted protocol. Facilitator assistance was required in 28 task-participant instances across six tasks, most frequently when copying selected text, selecting a search-result link, and creating a new file. Mean task-completion times ranged from 9 to 240 seconds. The observations point to command operations and workflow transitions that may warrant further refinement, including clearer feedback, more visible error recovery, simplified multi-step interactions, and greater personalisation. However, the evaluation did not include participants with upper-limb impairments, did not test the face/head-motion module, and did not use standardised usability, workload, or cognitive-load measures. The results therefore offer preliminary proxy-user observations about the Turkish voice-command module rather than validation of the complete multimodal system or its accessibility for the intended user population.
References
Accot, J., & Zhai, S. (1997). Beyond Fitts’ law: Models for trajectory-based HCI tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’97) (pp. 295–302). Association for Computing Machinery. https://doi.org/10.1145/258549.
258760
Center for Universal Design. (1997). The principles of universal design (Version 2.0). North Carolina State University. https://design.ncsu.edu/wp-content/uploads/2022/11/
principles-of-universal-design.pdf
Hawley, M. S. (2002). Speech recognition as an input to electronic assistive technology. British Journal of Occupational Therapy, 65(1), 15–20. https://doi.org/10.1177/
030802260206500104
International Organization for Standardization. (2025). Ergonomics of human-system interaction—Part 171: Software accessibility (ISO Standard No. 9241-171:2025). https://www.iso.org/standard/86308.html
Kabir, M. R., Mahmud, H., & Hasan, M. K. (2023). Acceptability of a head-mounted assistive mouse controller for people with upper limb disability: An empirical study using the technology acceptance model. PLOS ONE, 18(10), e0293608. https://doi.org/10.1371/journal.pone.
0293608
Keleş, M. S. (2024). El veya kollarında engeli bulunan bireylerin bilgisayar kullanabilmelerini sağlayabilecek bilgisayar sistemi (Tez No. 874956) [Yüksek lisans tezi, İzmir Kâtip Çelebi Üniversitesi]. Ulusal Tez Merkezi
Kodandaram, S. R., Uckun, U., Bi, X., Ramakrishnan, I. V., & Ashok, V. (2024). Enabling uniform computer interaction experience for blind users through large language models. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’24) (Article 73, pp. 1–14). Association for Computing Machinery. https://doi.org/10.1145/3663548.3675605
Mandak, K., Light, J., & Brittlebank-Douglas, S. (2022). Exploration of multimodal alternative access for individuals with severe motor impairments: Proof of concept. Assistive Technology, 34(6), 674–683. https://doi.org/10.1080/10400435.2021.1910375
Mankoff, J., Hayes, G. R., & Kasnitz, D. (2010). Disability studies as a source of critical inquiry for the field of assistive technology. In Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’10) (pp. 3–10). Association for Computing Machinery. https://doi.org/10.1145/1878803.1878807
Oliver, M. (1990). The politics of disablement. Macmillan Education. https://doi.org/10.1007/
978-1-349-20895-1
Oviatt, S. (2000). Taming recognition errors with a multimodal interface. Communications of the ACM, 43(9), 45–51. https://doi.org/10.1145/348941.348979
Paas, F., & van Merriënboer, J. J. G. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394–398. https://doi.org/10.1177/0963721420922183
Pan, L., Yu, C., Li, J. H., Huang, T., Bi, X., & Shi, Y. (2022). Automatically generating and improving voice command interface from operation sequences on smartphones. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) (Article 208, pp. 1–21). Association for Computing Machinery. https://doi.org/10.1145/3491102.
3517459
Soukoreff, R. W., & MacKenzie, I. S. (2004). Towards a standard for pointing device evaluation: Perspectives on 27 years of Fitts’ law research in HCI. International Journal of Human-Computer Studies, 61(6), 751–789. https://doi.org/10.1016/j.ijhcs.2004.09.001
United Nations. (2006). Convention on the Rights of Persons with Disabilities. https://www.un.org/disabilities/documents/convention/convoptprot-e.pdf
Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. (2011). Ability-based design: Concept, principles and examples. ACM Transactions on Accessible Computing, 3(3), Article 9, 1–27. https://doi.org/10.1145/1952383.1952384
World Health Organization. (2023, March 7). Disability and health. https://www.who.int/news-room/fact-sheets/detail/disability-and-health
World Wide Web Consortium. (2023). Web Content Accessibility Guidelines (WCAG) 2.2. https://www.w3.org/TR/WCAG22
World Wide Web Consortium. (2024, March 14). Essential components of web accessibility. https://www.w3.org/WAI/fundamentals/components
World Wide Web Consortium. (2025). Guidance on applying WCAG 2 to non-web information and communications technologies (WCAG2ICT). https://www.w3.org/TR/wcag2ict
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Keleş M.S., Alaybeyoğlu A.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share or adapt the work with an acknowledgment of the work's authorship and initial publication in this journal. Use of the work for commercial purposes are not allowed.
- Authors are able to publish the journal's published version of the work in other media (e.g., post it to an institutional repository or publish it in a book), as far as they inform the Journal of Accessibility and Design for All of that fact. When publishing their work in other sources, authors must mention the name of the Journal of Accessibility and Design for All, its ISSN, the number and issue in which the article was published and a link to the main page of the Journal of Accessibility and Design for All. Optionally, they can also include a link to the article published in the Journal of Accessibility and Design for All.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website), as it can lead to productive exchanges, as well as earlier and greater citation of published work.
