Automatic Generation of Test Cases from UML Models
DOI:
https://doi.org/10.19153/cleiej.21.1.3Keywords:
Model-based testing, Abstract test cases, UML models, MBTAbstract
[Context] The growing demand for high-quality software has caused the industry to incorporate processes to enable them to comply with these standards, but increasing the cost of development. A strategy to reduce this cost is to incorporate quality evaluations from early stages of software development. A technique that facilitates this evaluation is the model-based testing, which allows to generate test cases at early phases using as input the conceptual models of the system.
[Objective] In this paper, we introduce TCGen, a tool that enables the automatic generation of abstract test cases starting from UML conceptual models.
[Method] The design and implementation of TCGen, a technique that applies different testing criteria to class diagrams and state transition diagrams to generates test cases, is presented as a model-based testing approach. To do that, TCGen uses UML models, which are widely used at industry and a set of algorithms that recognize the concepts in the models in order to generate abstract test cases.
[Results] An exploratory experimental evaluation has been performed to compare the TCGen tool with traditional testing.
[Conclusions] Even though the exploratory evaluation shows promising results, it is necessary to perform more empirical evaluations in order to generalize the results.
Abstract (in Spanish):
[Contexto] La creciente demanda de software de alta calidad ha provocado que la industria incorpore procesos para permitirles cumplir con estos estándares, pero aumentando el costo del desarrollo. Una estrategia para reducir este costo es incorporar evaluaciones de calidad desde las primeras etapas del desarrollo del software. Una técnica que facilita esta evaluación es la prueba basada en modelos, que permite generar casos de prueba en fases tempranas utilizando como entrada los modelos conceptuales del sistema.
[Objetivo] En este artículo, presentamos TCGen, una herramienta que permite la generación automática de casos de pruebas abstractas a partir de modelos conceptuales UML.
[Método] El diseño e implementación de TCGen, una técnica que aplica diferentes criterios de prueba a los diagramas de clases y diagramas de transición de estados para generar casos de prueba, se presenta como un enfoque de prueba basado en modelos. Para hacer eso, TCGen utiliza modelos UML, que son ampliamente utilizados en la industria y un conjunto de algoritmos que reconocen los conceptos en los modelos para generar casos de prueba abstractos.
[Resultados] Se realizó una evaluación experimental exploratoria para comparar la herramienta TCGen con las pruebas tradicionales.
[Conclusiones] Aunque la evaluación exploratoria muestra resultados prometedores, es necesario realizar más evaluaciones empíricas para generalizar los resultados.
References
[2] M. Botteck and T. Deiß, "Introduction of TTCN-3 into the product development process: considerations from an electronic devices developer point of view," International Journal on Software Tools for Technology Transfer, vol. 10, pp. 285-289, 2008. Available: https://doi.org/10.1007/s10009-008-0077-y
[3] S. A. Slaughter, D. E. Harter, and M. S. Krishnan, "Evaluating the cost of software quality," Communications of the ACM vol. 41, pp. 67-73, 1998. Available: http://dx.doi.org/10.1145/280324.280335
[4] F. Lindlar, "Search-Based Functional Testing of Embedded Software Systems," Doctoral Symposium in conjunction with IEEE International Conference on Software Testing, Verification and Validation (ICST 2009), Denver, USA, 2009.
[5] P. Reales Mateo, M. Polo, J. L. Fernández Alemán, A. Toval, and M. Piattini, " Mutation Testing," IEEE Software, vol. 31, pp. 30-35, 2014. Available: http://dx.doi.org/10.1109/MS.2014.68
[6] M. Utting, A. Pretschner, and B. Legeard, "A taxonomy of model-based testing approaches," Software Testing, Verification and Reliability, vol. 22, pp. 297-312 2012. Available: http://dx.doi.org/10.1002/stvr.456
[7] R. H. Thayer, J. B. Slaughter, B. W. Boehm, J. A. Clapp, J. H. Manley, and J. H. Burrows, "The high cost of software: causes and corrections," Proceedings of the national computer conference and exposition AFIPS '74 1974, pp. 1009-1009 Available: http://dx.doi.org/10.1145/1500175.1500373
[8] P. Iyenghar, E. Pulvermueller, and C. Westerkamp, "Towards Model-Based Test automation for embedded systems using UML and UTP," IEEE 16th Conference on Emerging Technologies & Factory Automation (ETFA), 2011, pp. 1-9. Available: http://dx.doi.org/10.1109/ETFA.2011.6058982
[9] Selenium. (last visited on January 2017). Available: http://docs.seleniumhq.org/
[10] Watir. (last visited on January 2017). Available: http://watir.com/
[11] H. P. Q. Center. (last visited on January 2017). Available: http://www8.hp.com/us/en/software-solutions/quality-center-quality-management/
[12] B. Marín, C. Gallardo, D. Quiroga, G. Giachetti, and E. Serral, "Testing of model-driven development applications," Software Quality Journal, pp. 1-29, 10ht Feb. 2016. Available: http://dx.doi.org/10.1007/s11219-016-9308-8
[13] OMG, "Unified Modeling Language (UML) 2.4.1 Superstructure Specification " 2011.
[14] C. Pérez and B. Marín, "Generación automática de casos de prueba basados en modelos UML," XX Iberoamerican Conference on Software Engineering (CIBSE), Software Engineering Track (SET), Buenos Aires, Argentina, 2017.
[15] M. Utting and B. Legeard, Practical Model-Based Testing - A Tools Approach: Morgan Kaufmann, 2007.
[16] B. Kitchenham, "Guidelines for performing Systematic Literature Reviews in Software Engineering," UK, EBSE Technical ReportJuly 9 2007.
[17] P. Bunyakiati and A. Finkelstein, "The compliance testing of software tools with respect to the UML standards specification - The ArgoUML case study," ICSE Workshop on Automation of Software Test, 2009, pp. 138– 143. Available: http://dx.doi.org/10.1109/IWAST.2009.5069052
[18] K. Nylund, E. O?stman, D. Truscan, and R. Teittinen, "Towards Rapid Creation of Test Adaptation in On-line Model-Based Testing," IEEE 35th Annual Computer Software and Applications Conference Workshops (COMPSACW), 2011, pp. 174–179. Available: http://dx.doi.org/10.1109/COMPSACW.2011.38
[19] V. Santiago, A. Martins-do-Amaral, N. Vijaykumar, M. Mattiello-Francisco, E. Martins, and O. Cuesta-Lopes, "A practical approach for automated test case generation using statecharts," 2nd International Workshop on Testing and Quality Assurance for Component-Based Systems - IEEE COMPSAC Conference, 2006, pp. 183–189. Available: http://dx.doi.org/10.1109/COMPSAC.2006.100
[20] Y. Yang, H. Zhang, M. Pan, J. Yang, F. He, and Z. Li, "A model-based fuzz framework to the security testing of tcg software stack implementations," International Conference on Multimedia Information Networking and Security, 2009, pp. 149–152. Available: http://dx.doi.org/10.1109/MINES.2009.111
[21] D. Xu, W. Xu, M. Kent, L. Thomas, and L. Wang, "An automated test generation technique for software quality assurance," IEEE Transactions on Reliability, vol. 64, pp. 247–268, 2015. Available: http://dx.doi.org/10.1109/TR.2014.2354172
[22] C. Doungsa-ard, K. Dahal, A. Hossain, and T. Suwannasart, "Test data generation from uml state machine diagrams using gas," International Conference on Software Engineering Advances, 2007, pp. 47–52. Available: http://dx.doi.org/10.1109/ICSEA.2007.70
[23] Z. Wei and W. Xiaoxue, "Graph theory model based automatic test platform design," 2nd International Conference on Software Enginee- ring and Data Mining (SEDM), 2010, pp. 263–267.
[24] C. Bertolini and A. Mota, "A framework for gui testing based on use case design," Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), 2010, pp. 252–269. Available: http://dx.doi.org/10.1109/ICSTW.2010.37
[25] E. Rodrigues, R. Saad, F. Oliveira, L. Costa, M. Bernardino, and A. Zorzo, "Evaluating capture and replay and model-based performance testing tools: an empirical comparison," 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2014. Available: http://dx.doi.org/10.1145/2652524.2652587
[26] M. Shirole and R. Kumar, "Uml behavioral model based test case generation: a survey," ACM SIGSOFT Software Engineering Notes, vol. 38, pp. 1–13, 2013. Available: http://dx.doi.org/10.1145/2492248.2492274
[27] M. Shirole, A. Suthar, and R. Kumar, "Generation of improved test cases from uml state diagram using genetic algorithm," 4th India Software Engineering Conference, 2011, pp. 125–134.
[28] M. Sarma, P. Murthy, S. Jell, and A. Ulrich, "Model-based testing in industry: a case study with two mbt tools," 4th India Software Engineering Conference, 2011, pp. 87-90. Available: http://dx.doi.org/10.1145/1808266.1808279
[29] M. Mlynarski, B. Gu?ldali, M. Spa?th, and G. Engels, "From design models to test models by means of test ideas," 6th International Workshop on Model-Driven Engineering, Verification and Validation, 2009. Available: http://dx.doi.org/10.1145/1656485.1656492
[30] B. Pe?rez-Lamancha, P. Mateo, I. Guzma?n, M. Usaola, and M. Piattini, "Automated model-based testing using the uml testing profile and qvt," 6th International Workshop on Model- Driven Engineering, Verification and Validation, 2009. Available: http://dx.doi.org/10.1145/1656485.1656491
[31] A. Nayak and D. Samanta, "Model-based test cases synthesis using uml interaction diagrams," ACM SIGSOFT Software Engineering Notes, vol. 34, pp. 1–10, 2009. Available: http://dx.doi.org/10.1145/1507195.1507209
[32] F. Bouquet, C. Grandpierre, B. Legeard, and F. Peureux, "A test generation solution to automate software testing," 3rd international workshop on Automation of software test, 2008, pp. 45-48. Available: http://dx.doi.org/10.1145/1370042.1370052
[33] F. Bouquet, C. Grandpierre, B. Legeard, F. Peureux, N. Vacelet, and M. Utting, "A subset of precise uml for model-based testing," 3rd international workshop on Advances in model-based testing, 2007, pp. 95–104. Available: http://dx.doi.org/10.1145/1291535.1291545
[34] L. Wang, E. Wong, and D. Xu, "A threat model driven approach for security testing," Third International Workshop on Software Engineering for Secure Systems, 2007. Available: http://dx.doi.org/10.1109/SESS.2007.2
[35] P. Murthy, P. Anitha, M. Mahesh, and R. Subramanyan, "Test ready UML statechart models," ICSE Workshop on Automation of Software Test, 2006, pp. 138–143. Available: http://dx.doi.org/10.1145/1138953.1138968
[36] L. Chang, H. Miao, and G. Lu, "An Implementation Framework for Optimizing Test Case Generation Using Model Checking," in Structured Object-Oriented Formal Language and Method, ed, 2015, pp. 3-16. Available: https://doi.org/10.1007/978-3-319-17404-4_1
[37] R. Chen and H. Miao, "A selenium based approach to automatic test script generation for refactoring javascript code," IEEE/ACIS 12th International Conference on Computer and Information Science (ICIS), 2013, pp. 341 – 346. Available: http://dx.doi.org/10.1109/ICIS.2013.6607864
[38] A. Gibson. (Last visited October 2015). Data Factory. Available: https://github.com/andygibson/datafactory
[39] Eclipse-Papyrus, last visited on January 2017.
[40] IntelliJ IDEA. (last visited on January 2017). Available: https://www.jetbrains.com/idea/
[41] JavaFX. (last visited on January 2017). Available: http://www.oracle.com/technetwork/java/javase/downloads/sb2download-2177776.html
[42] N. Juristo and A. Moreno, Basics of Software Engineering Experimentation: Springer, 2001.
[43] P. Runeson and M. Host, "Guidelines for conducting and reporting case study research in software engineering," Empirical Software Engineering Journal, vol. 14, pp. 131–164, 2009. Available: http://dx.doi.org/10.1007/s10664-008-9102-8
Downloads
Published
Issue
Section
License
CLEIej is supported by its home institution, CLEI, and by the contribution of the Latin American and international researchers community, and it does not apply any author charges whatsoever for submitting and publishing. Since its creation in 1998, all contents are made publicly accesibly. The current license being applied is a (CC)-BY license (effective October 2015; between 2011 and 2015 a (CC)-BY-NC license was used).
