AI reduces ‘communication gap’ for nonverbal people by as much as half


17-06-2020
  Speech bubble  Credit: Photo by Volodymyr Hryshchenko on Unsplash

Researchers have used artificial intelligence to reduce the ‘communication gap’ for nonverbal people with motor disabilities who rely on computers to converse with others.

The team, from the University of Cambridge and the University of Dundee, developed a new context-aware method that reduces this communication gap by eliminating between 50% and 96% of the keystrokes the person has to type to communicate.

The system is specifically tailed for nonverbal people and uses a range of context ‘clues’ – such as the user’s location, the time of day or the identity of the user’s speaking partner – to assist in suggesting sentences that are the most relevant for the user.

Nonverbal people with motor disabilities often use a computer with speech output to communicate with others. However, even without a physical disability that affects the typing process, these communication aids are too slow and error-prone for meaningful conversation: typical typing rates are between five and 20 words per minute, while a typical speaking rate is in the range of 100 to 140 words per minute.

“This difference in communication rates is referred to as the communication gap,” said Professor Per Ola Kristensson from Cambridge’s Department of Engineering, the study’s lead author. “The gap is typically between 80 and 135 words per minute and affects the quality of everyday interactions for people who rely on computers to communicate.”

The method developed by Kristensson and his colleagues uses artificial intelligence to allow a user to quickly retrieve sentences they have typed in the past. Prior research has shown that people who rely on speech synthesis, just like everyone else, tend to reuse many of the same phrases and sentences in everyday conversation. However, retrieving these phrases and sentences is a time-consuming process for users of existing speech synthesis technologies, further slowing down the flow of conversation.

In the new system, as the person is typing, the system uses information retrieval algorithms to automatically retrieve the most relevant previous sentences based on the text typed and the context the conversation the person is involved in. Context includes information about the conversation such as the location, time of day, and automatic identification of the speaking partner’s face. The other speaker is identified using a computer vision algorithm trained to recognise human faces from a front-mounted camera.

Read the full story

Image: Speech bubble

Credit: Photo by Volodymyr Hryshchenko on Unsplash

Reproduced courtesy of the University of Cambridge

 

The University of Cambridge is acknowledged as one of the world's leading higher education and research institutions. The University was instrumental in the formation of the Cambridge Network and its Vice- Chancellor, Professor Stephen Toope, is also the President of the Cambridge Network.

University of Cambridge (cam.ac.uk)