Project Description

BioVisualSpeech is a research project that aims to research natural and multimodal interaction mechanisms for providing bio-feedback in speech therapy through the use of serious (computer) games. This project proposes a platform for speech therapy that has two main goals: the first goal is to keep the patients engaged in the exercises in order to have more fruitful sessions. For this, the platform includes a gamified interactive environment that uses visual stimuli, several serious games, and a reward system that motivates children to keep exercising. The environment provides instantaneous bio-feedback, replacing other tools such as a mirror, and assists the therapist during the session with audio-visual recordings and annotations of the session. The second goal aims at providing the therapist with a toolset to plan the course of the ongoing therapy session. For this, the platform will include an application for post-session analysis where a patient’s audio-visual recordings can be examined through browsing and searching, and a future session can be planned. The novelty of this platform is the combination of visual bio-feedback and the gamification of the speech therapy. The gamified interactive environment is proposed as a key instrument to keep the patient engaged and reactive to the instantaneous bio-feedback provided by the platform.

One of the games with the mirror feature enabled.