Automatic Assessment of Dysarthria Using Audio-visual Vowel Graph Attention Network

no code implementations • 6 May 2024 • Xiaokang Liu, Xiaoxia Du, Juan Liu, Rongfeng Su, Manwa Lawrence Ng, Yumei Zhang, Yudong Yang, Shaofeng Zhao, Lan Wang, Nan Yan

Currently, research on the automatic assessment of dysarthria primarily focuses on two approaches: one that utilizes expert features combined with machine learning, and the other that employs data-driven deep learning methods to extract representations.

Graph Attention

An Audio-textual Diffusion Model For Converting Speech Signals Into Ultrasound Tongue Imaging Data

no code implementations • 9 Mar 2024 • Yudong Yang, Rongfeng Su, Xiaokang Liu, Nan Yan, Lan Wang

In this model, the inherent acoustic characteristics of individuals related to the tongue motion details are encoded by using wav2vec 2. 0, while the ASR transcriptions related to the universality of tongue motions are encoded by using BERT.