Fuzzing class specifications

Expressing class specifications via executable constraints is important for various software engineering tasks such as test generation, bug finding and automated debugging, but developers rarely write them. Techniques that infer specifications from code exist to fill this gap, but they are designed...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Molina, Facundo, d’Amorim, Marcelo, Aguirre, Nazareno Matías
Formato: Objeto de conferencia Resumen
Lenguaje:Español
Publicado: 2022
Materias:
Ciencias Informáticas
Assertion languages
SpecFuzzer
Acceso en línea:http://sedici.unlp.edu.ar/handle/10915/151641
https://publicaciones.sadio.org.ar/index.php/JAIIO/article/download/304/253
Aporte de:
SEDICI (UNLP) de Universidad Nacional de La Plata
id I19-R120-10915-151641
record_format dspace
spelling I19-R120-10915-1516412023-05-03T20:04:19Z http://sedici.unlp.edu.ar/handle/10915/151641 https://publicaciones.sadio.org.ar/index.php/JAIIO/article/download/304/253 issn:2451-7496 Fuzzing class specifications Molina, Facundo d’Amorim, Marcelo Aguirre, Nazareno Matías 2022-10 2022 2023-04-18T15:21:16Z es Ciencias Informáticas Assertion languages SpecFuzzer Expressing class specifications via executable constraints is important for various software engineering tasks such as test generation, bug finding and automated debugging, but developers rarely write them. Techniques that infer specifications from code exist to fill this gap, but they are designed to support specific kinds of assertions and are difficult to adapt to support different assertion languages, e.g., to add support for quantification, or additional comparison operators, such as membership or containment.To address the above issue, we propose SpecFuzzer, a novel technique that combines grammar-based fuzzing, dynamic invariant detection, and mutation analysis, to automatically produce class specifications. SpecFuzzer uses: (i) a fuzzer as a generator of candidate assertions derived from a grammar that is automatically obtained from the class definition; (ii) a dynamic invariant detector –Daikon– to filter out assertions invalidated by a test suite; and (iii) a mutationbased mechanism to cluster and rank assertions, so that similar constraints are grouped and then the stronger prioritized. Fuzzing, traditionally used to efficiently produce structured random data for testing, has two key advantages in this context: (1) it eliminates the need of developers to manually define candidate assertions and (2) it enables developers to straightforwardly adapt the language of assertions by manipulating the fuzzing grammar, e.g., to include additional operators.We evaluated our technique on a benchmark of 43 Java methods employed in the evaluation of the state-of-the-art techniques GAssert and EvoSpex. In our evaluation, we used the same benchmarks from the evaluation of GAssert and EvoSpex, carefully studied the subjects, and manually produced corresponding “ground truth” assertions capturing the intended behavior of the subjects. We then used this ground truth to accurately assess precision and recall of SpecFuzzer, GAssert, and EvoSpex. Our results show that SpecFuzzer can easily support a more expressive assertion language, over which is more effective than GAssert and EvoSpex in inferring specifications, according to standard performance metrics. More precisely, SpecFuzzer was able to express ∼45% more assertions in the ground truth than these tools. Also, SpecFuzzer was able to detect 75% of all assertions in the ground truth, showing a better overall performance compared to previous techniques. The results we obtained provide initial, yet strong evidence that SpecFuzzer is effective. This work was published at the 44th International Conference on Software Engineering (ICSE 2022) held in Pittsburgh, PA, USA on 22-27 May 2022. Sociedad Argentina de Informática e Investigación Operativa Objeto de conferencia Resumen http://creativecommons.org/licenses/by-nc-sa/4.0/ Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) application/pdf 72-72
institution Universidad Nacional de La Plata
institution_str I-19
repository_str R-120
collection SEDICI (UNLP)
language Español
topic Ciencias Informáticas
Assertion languages
SpecFuzzer
spellingShingle Ciencias Informáticas
Assertion languages
SpecFuzzer
Molina, Facundo
d’Amorim, Marcelo
Aguirre, Nazareno Matías
Fuzzing class specifications
topic_facet Ciencias Informáticas
Assertion languages
SpecFuzzer
description Expressing class specifications via executable constraints is important for various software engineering tasks such as test generation, bug finding and automated debugging, but developers rarely write them. Techniques that infer specifications from code exist to fill this gap, but they are designed to support specific kinds of assertions and are difficult to adapt to support different assertion languages, e.g., to add support for quantification, or additional comparison operators, such as membership or containment.To address the above issue, we propose SpecFuzzer, a novel technique that combines grammar-based fuzzing, dynamic invariant detection, and mutation analysis, to automatically produce class specifications. SpecFuzzer uses: (i) a fuzzer as a generator of candidate assertions derived from a grammar that is automatically obtained from the class definition; (ii) a dynamic invariant detector –Daikon– to filter out assertions invalidated by a test suite; and (iii) a mutationbased mechanism to cluster and rank assertions, so that similar constraints are grouped and then the stronger prioritized. Fuzzing, traditionally used to efficiently produce structured random data for testing, has two key advantages in this context: (1) it eliminates the need of developers to manually define candidate assertions and (2) it enables developers to straightforwardly adapt the language of assertions by manipulating the fuzzing grammar, e.g., to include additional operators.We evaluated our technique on a benchmark of 43 Java methods employed in the evaluation of the state-of-the-art techniques GAssert and EvoSpex. In our evaluation, we used the same benchmarks from the evaluation of GAssert and EvoSpex, carefully studied the subjects, and manually produced corresponding “ground truth” assertions capturing the intended behavior of the subjects. We then used this ground truth to accurately assess precision and recall of SpecFuzzer, GAssert, and EvoSpex. Our results show that SpecFuzzer can easily support a more expressive assertion language, over which is more effective than GAssert and EvoSpex in inferring specifications, according to standard performance metrics. More precisely, SpecFuzzer was able to express ∼45% more assertions in the ground truth than these tools. Also, SpecFuzzer was able to detect 75% of all assertions in the ground truth, showing a better overall performance compared to previous techniques. The results we obtained provide initial, yet strong evidence that SpecFuzzer is effective. This work was published at the 44th International Conference on Software Engineering (ICSE 2022) held in Pittsburgh, PA, USA on 22-27 May 2022.
format Objeto de conferencia
Resumen
author Molina, Facundo
d’Amorim, Marcelo
Aguirre, Nazareno Matías
author_facet Molina, Facundo
d’Amorim, Marcelo
Aguirre, Nazareno Matías
author_sort Molina, Facundo
title Fuzzing class specifications
title_short Fuzzing class specifications
title_full Fuzzing class specifications
title_fullStr Fuzzing class specifications
title_full_unstemmed Fuzzing class specifications
title_sort fuzzing class specifications
publishDate 2022
url http://sedici.unlp.edu.ar/handle/10915/151641
https://publicaciones.sadio.org.ar/index.php/JAIIO/article/download/304/253
work_keys_str_mv AT molinafacundo fuzzingclassspecifications
AT damorimmarcelo fuzzingclassspecifications
AT aguirrenazarenomatias fuzzingclassspecifications
_version_ 1765659994559086592

Ejemplares similares