Gloss-free Argentinian Sign Language Translation with pose-based deep learning models

The main challenge of automatic Sign Language Translation (SLT) is obtaining data to train models. For Argentinian Sign Language (LSA), the only dataset available for SLT is LSA-T, which contains extracts of a news channel in LSA and the corresponding Spanish subtitles provided by the authors. LSA-T...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Dal Bianco, Pedro Alejandro, Ríos, Gastón Gustavo, Hasperué, Waldo, Stanchi, Oscar Agustín, Ronchetti, Franco, Quiroga, Facundo Manuel
Formato: Objeto de conferencia
Lenguaje:Inglés
Publicado: 2024
Materias:
Ciencias Informáticas
Sign Language Translation
Pose Estimation
Sign Language Datasets
Deep Learning
Gloss-free
Acceso en línea:http://sedici.unlp.edu.ar/handle/10915/176192
Aporte de:
SEDICI (UNLP) de Universidad Nacional de La Plata
Descripción
Sumario:The main challenge of automatic Sign Language Translation (SLT) is obtaining data to train models. For Argentinian Sign Language (LSA), the only dataset available for SLT is LSA-T, which contains extracts of a news channel in LSA and the corresponding Spanish subtitles provided by the authors. LSA-T contains a wide variety of signers, scenarios, and lightnings that could bias a model trained on it. We propose a model for Argentinian gloss-free SLT, since LSA-T does not contain gloss representations of the signs. The model is also pose-based to improve performance on low resource devices. Different versions of the model are also tested in two other well-known datasets to compare the results: GSL and RWTH Phoenix Weather 2014T. Our model stablished the new SoTA over LSA-T, which proved to be the most challenging due to the variety of topics covered that result in a vast vocabulary with many words appearing few times.

Ejemplares similares