TEHRAN- Researchers in Amirkabir University of Technology have proposed a new method for automating lip-reading process to aid people with hearing difficulty. 

The study investigates a model that can use hybrid visual features for optimizing lip-reading.

Lip reading, also known as speech-reading, is a technique of understanding speech by visually interpreting the movements of the lips, face and tongue when normal sound is not available. 

Experiments over many years have revealed that speech intelligibility increases if visual facial information becomes available.

The research was carried out by Fatemeh Vakhshiteh with the supervision of professors Farshad Almasgan and Ahmad Nickabadi.

In an interview with ISNA, Vakhshiteh said using a variety of sources for extracting information substantially helps the lip-reading process.

According to Vakhshiteh, this model was inspired by the function of the brain because the human brain also processes several sources of information in production and reception of speech. 

In this model, deep neural networks are used to make the recognition of lip-reading as well as phone recognition easier, she said. 

“The neural networks were specially used for situations that audial and visual features must be processed simultaneously.”

“This is especially helpful in noisy environments where the audial data produced by speakers might become less clear or incomprehensible.”

“This would also help the people with speech difficulty because they can use their visual data to compensate for the interruption in the speech signal they receive,” she added. 

The research results demonstrated that the proposed method outperforms the conventional Hidden Markov Model (HMM) and competes well with the state-of-the-art visual speech recognition works.

SJ/MQ/MG