Prediction of System Usability through Neural NetworksApplied to Conversational User Interfaces
DOI:
https://doi.org/10.19153/cleiej.29.1.8Keywords:
CUI, Usability, Deep Neural Networks, ScalesAbstract
This study investigates the use of Deep Neural Networks (DNN) to predict perceived usability in conversational user interfaces (CUIs), focusing on two systems (ChatGPT and Gemini) and two interaction modalities (text and voice). A structured methodology
involving data segmentation, regularization techniques, and crossvalidation was applied to develop predictive models that incorporate demographic variables and user experience. Results highlight the superior performance of the Gemini voice model (RMSE: 0.18, R2: 0.80), followed by ChatGPT in text mode (RMSE: 0.2, R2: 0.78). Text-based interaction with Gemini showed lower predictive accuracy, suggesting underfitting and the need for architectural adjustments. In general, voice-based models demonstrated greater consistency and predictive power, possibly due to a more intuitive user experience. These findings confirm the potential of DNNs to model complex user perceptions and support user-centered design. The proposed approach offers a valuable tool for anticipating usability outcomes and enhancing the personalization of CUI experiences.
References
L. C. et al., “What makes a good conversation? challenges in designing truly conversational agents,” in
Conference on Human Factors in Computing Systems - Proceedings, vol. 12, Jan 2019.
C. Murad and C. Munteanu, “’i don’t know what you’re talking about, halexa’: The case for voice user
interface guidelines,” in ACM International Conference Proceeding Series, Aug 2019.
B. R. C. C. Murad, C. Munteanu and L. Clark, “Finding a new voice: Transitioning designers from gui
to vui design,” in ACM International Conference Proceeding Series, Jul 2021.
L. K. G. Samboni, “Revisión de heurísticas y escalas de usabilidad y ux para tecnologías basadas en el
habla,” 2025.
J. Brooke, “Sus - a quick and dirty usability scale,” available:
https://www.researchgate.net/publication/228593520SUSAquickanddirtyusabilityscale.
A. M. L. Fulfagar, A. Gupta and A. Shrivastava, “Development and evaluation of usability heuristics for
voice user interfaces,” in Smart Innovation, Systems and Technologies, 2021, vol. 221, pp. 375–385.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Lady K. Gómez, Huizilopoztli Luna-García, Cesar A. Collazos
This work is licensed under a Creative Commons Attribution 4.0 International License.
CLEIej is supported by its home institution, CLEI, and by the contribution of the Latin American and international researchers community, and it does not apply any author charges whatsoever for submitting and publishing. Since its creation in 1998, all contents are made publicly accesibly. The current license being applied is a (CC)-BY license (effective October 2015; between 2011 and 2015 a (CC)-BY-NC license was used).
