CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos
Action recognition in videos is currently a topic of interest in the area of computer vision, due to potential applications such as: multimedia indexing, surveillance in public spaces, among others. Attention mechanisms have become a very important concept within deep learning approach, their operat...
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Artículo publishedVersion |
| Lenguaje: | Español |
| Publicado: |
FIUBA
2021
|
| Materias: | |
| Acceso en línea: | https://elektron.fi.uba.ar/elektron/article/view/130 https://repositoriouba.sisbi.uba.ar/gsdl/cgi-bin/library.cgi?a=d&c=elektron&d=130_oai |
| Aporte de: |
| id |
I28-R145-130_oai |
|---|---|
| record_format |
dspace |
| spelling |
I28-R145-130_oai2026-02-11 Orozco, Carlos Ismael Buemi, María Elena Jacobo Berlles, Julio 2021-06-15 Action recognition in videos is currently a topic of interest in the area of computer vision, due to potential applications such as: multimedia indexing, surveillance in public spaces, among others. Attention mechanisms have become a very important concept within deep learning approach, their operation tries to imitate the visual capacity of people that allows them to focus their attention on relevant parts of a scene to extract important information. In this paper we propose a soft attention mechanism adapted to a base CNN–LSTM architecture. First, a VGG16 convolutional neural network extracts the features from the input video. Then an LSTM classifies the video into a particular class. To carry out the training and testing phases, we used the HMDB-51 and UCF-101 datasets. We evaluate the performance of our system using accuracy as an evaluation metric, obtaining 40,7 % (base approach), 51,2 % (with attention) for HMDB-51 and 75,8 % (base approach), 87,2 % (with attention) for UCF-101. El reconocimiento de acciones en videos es actualmente un tema de interés en el área de la visión por computador, debido a potenciales aplicaciones como: indexación multimedia, vigilancia en espacios públicos, entre otras. Los mecanismos de atención se han convertido en un concepto muy importante dentro del enfoque de aprendizaje profundo, su operación intenta imitar la capacidad visual de las personas que les permite enfocar su atención en partes relevantes de una escena para extraer información importante. En este artículo proponemos un mecanismo de atención suave adaptado para degradar la arquitectura CNN–LSTM. Primero, una red neuronal convolucional VGG16 extrae las características del video de entrada. Para llevar a cabo las fases de entrenamiento y prueba, usamos los conjuntos de datos HMDB-51 y UCF-101. Evaluamos el desempeño de nuestro sistema usando la precisión como métrica de evaluación, obteniendo 40,7 % (enfoque base), 51,2 % (con atención) para HMDB-51 y 75,8 % (enfoque base), 87,2 % (con atención) para UCF-101. application/pdf text/html https://elektron.fi.uba.ar/elektron/article/view/130 10.37537/rev.elektron.5.1.130.2021 spa FIUBA https://elektron.fi.uba.ar/elektron/article/view/130/246 https://elektron.fi.uba.ar/elektron/article/view/130/247 Derechos de autor 2021 Carlos Ismael Orozco, María Elena Buemi, Julio Jacobo Berlles Elektron Journal; Vol. 5 No. 1 (2021); 37-44 Revista Elektron; Vol. 5 Núm. 1 (2021); 37-44 Revista Elektron; v. 5 n. 1 (2021); 37-44 2525-0159 2525-0159 action recognition convolutional neural network long short-term memory attention mechanism reconocimiento de acciones redes neuronales convolucionales redes neuronales lstm mecanismo de atención CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos CNN–LSTM con mecanismo de atención suave para el reconocimiento de acciones humanas en videos info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion https://repositoriouba.sisbi.uba.ar/gsdl/cgi-bin/library.cgi?a=d&c=elektron&d=130_oai |
| institution |
Universidad de Buenos Aires |
| institution_str |
I-28 |
| repository_str |
R-145 |
| collection |
Repositorio Digital de la Universidad de Buenos Aires (UBA) |
| language |
Español |
| orig_language_str_mv |
spa |
| topic |
action recognition convolutional neural network long short-term memory attention mechanism reconocimiento de acciones redes neuronales convolucionales redes neuronales lstm mecanismo de atención |
| spellingShingle |
action recognition convolutional neural network long short-term memory attention mechanism reconocimiento de acciones redes neuronales convolucionales redes neuronales lstm mecanismo de atención Orozco, Carlos Ismael Buemi, María Elena Jacobo Berlles, Julio CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| topic_facet |
action recognition convolutional neural network long short-term memory attention mechanism reconocimiento de acciones redes neuronales convolucionales redes neuronales lstm mecanismo de atención |
| description |
Action recognition in videos is currently a topic of interest in the area of computer vision, due to potential applications such as: multimedia indexing, surveillance in public spaces, among others. Attention mechanisms have become a very important concept within deep learning approach, their operation tries to imitate the visual capacity of people that allows them to focus their attention on relevant parts of a scene to extract important information. In this paper we propose a soft attention mechanism adapted to a base CNN–LSTM architecture. First, a VGG16 convolutional neural network extracts the features from the input video. Then an LSTM classifies the video into a particular class. To carry out the training and testing phases, we used the HMDB-51 and UCF-101 datasets. We evaluate the performance of our system using accuracy as an evaluation metric, obtaining 40,7 % (base approach), 51,2 % (with attention) for HMDB-51 and 75,8 % (base approach), 87,2 % (with attention) for UCF-101. |
| format |
Artículo publishedVersion |
| author |
Orozco, Carlos Ismael Buemi, María Elena Jacobo Berlles, Julio |
| author_facet |
Orozco, Carlos Ismael Buemi, María Elena Jacobo Berlles, Julio |
| author_sort |
Orozco, Carlos Ismael |
| title |
CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| title_short |
CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| title_full |
CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| title_fullStr |
CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| title_full_unstemmed |
CNN–LSTM with Soft Attention Mechanism for Human Action Recognition in Videos |
| title_sort |
cnn–lstm with soft attention mechanism for human action recognition in videos |
| publisher |
FIUBA |
| publishDate |
2021 |
| url |
https://elektron.fi.uba.ar/elektron/article/view/130 https://repositoriouba.sisbi.uba.ar/gsdl/cgi-bin/library.cgi?a=d&c=elektron&d=130_oai |
| work_keys_str_mv |
AT orozcocarlosismael cnnlstmwithsoftattentionmechanismforhumanactionrecognitioninvideos AT buemimariaelena cnnlstmwithsoftattentionmechanismforhumanactionrecognitioninvideos AT jacoboberllesjulio cnnlstmwithsoftattentionmechanismforhumanactionrecognitioninvideos AT orozcocarlosismael cnnlstmconmecanismodeatencionsuaveparaelreconocimientodeaccioneshumanasenvideos AT buemimariaelena cnnlstmconmecanismodeatencionsuaveparaelreconocimientodeaccioneshumanasenvideos AT jacoboberllesjulio cnnlstmconmecanismodeatencionsuaveparaelreconocimientodeaccioneshumanasenvideos |
| _version_ |
1859522257978654720 |