Automatic Classification of Web Images as UML Static Diagrams Using Machine Learning Techniques
Source abstract
Abstract
Our purpose in this research is to develop a method to automatically and efficiently classify web images as Unified Modeling Language (UML) static diagrams, and to produce a computer tool that implements this function. The tool receives a bitmap file (in different formats) as an input and communicates whether the image corresponds to a diagram. For pragmatic reasons, we restricted ourselves to the simplest kinds of diagrams that are more useful for automated software reuse: computer-edited 2D representations of static diagrams. The tool does not require that the images are explicitly or implicitly tagged as UML diagrams. The tool extracts graphical characteristics from each image (such as grayscale histogram, color histogram and elementary geometric forms) and uses a combination of rules to classify it. The rules are obtained with machine learning techniques (rule induction) from a sample of 19,000 web images manually classified by experts. In this work, we do not consider the textual contents of the images. Our tool reaches nearly 95% of agreement with manually classified instances, improving the effectiveness of related research works. Moreover, using a training dataset 15 times bigger, the time required to process each image and extract its graphical features (0.680 s) is seven times lower.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Semantic Web and Ontologies · Web Data Mining and Analysis · Software Engineering Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Knowledge Reuse Group, Departamento de Informática, Universidad Carlos III de Madrid. Av. Universidad 30, 28911 Leganés (Madrid), Spain
Funding
ARTEMIS Joint Undertaking · H2020-ECSEL
References (51)
Show source-provided references
1. Booch, G., Rumbaugh, J., and Jacobson, I. (2005). The Unified Modeling Language User Guide, Addison-Wesley. [2nd ed.].
2. Object Management Group (2019, December 05). Unified Modeling Language Specification, Version 2.5.1. Available online: https://www.omg.org/spec/UML/2.5.1/.
3. Growing the UML10.1007/s10270-002-0013-7 ↗
4. On the Unification Power of Models10.1007/s10270-005-0079-0 ↗
5. Ernesto, D., Lakhmi, C.J., and Mauro, M. (2004). RSHP: An Information Representation Model Based on Relationships. Soft Computing in Software Engineering (Studies in Fuzziness and Soft Computing Series, Vol. 159), Springer.
6. Towards an Ontology-based Retrieval of UML Class Diagrams10.1016/j.infsof.2011.07.003 ↗
7. On the Difference between Analysis and Design, and Why It Is Relevant for the Interpretation of Models in Model Driven Engineering10.5381/jot.2009.8.1.c7 ↗
8. Object Management Group (2019, December 05). XML Metadata Interchange Specification, Version 2.5.1. Available online: https://www.omg.org/spec/XMI/2.5.1/.
9. Moreno, V., Génova, G., Alejandres, M., and Fraga, A. (2016, January 14–16). Automatic classification of web images as UML diagrams. Proceedings of the 4th Spanish Conference in Information Retrieval, Granada, Spain.10.1145/2934732.2934739 ↗
10. Karasneh, B.H.A. (2016). An Online Corpus of UML Design Models: Construction and Empirical Studies. [Ph.D. Thesis, Leiden University].
11. France, R., Bieman, J., and Cheng, B.H. (2006, January 1–6). Repository for Model Driven Development (ReMoDD). Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MoDELS’06), Genoa, Italy.
12. France, R., Bieman, J.M., Mandalaparty, S.P., Cheng, B.H., and Jensen, A.C. (2012, January 2–9). Repository for Model Driven Development (ReMoDD). Proceedings of the 34th International Conference on Software Engineering (ICSE’12), Zurich, Switzerland.10.1109/ICSE.2012.6227059 ↗
13. A taxonomy of tool-related issues affecting the adoption of model-driven engineering10.1007/s10270-015-0487-8 ↗
14. Basciani, F., Di Rocco, J., Di Ruscio, D., Pierantonio, A., and Iovino, L. (2015, January 29). Model Repositories: Will they become reality? A Position Statement. Proceedings of the 3rd International Workshop on Model-Driven Engineering on and for the Cloud (CloudMDE’2015). Held in Conjunction with the 18th International Conference on Model Driven Engineering Languages and Systems (MoDELS’2015), Ottawa, Canada.
15. Bislimovska, B. (2014). Textual and Content Based Search in Software Model Repositories. [Ph.D. Thesis, Politecnico di Milano].
16. Textual and Content-Based Search in Repositories of Web Application Models
17. Robles, G., Ho-Quang, T., Hebig, R., Chaudron, M.R.V., and Fernández, M.A. (2017, January 20–21). An extensive dataset of UML models in GitHub. Proceedings of the 14th International Conference on Mining Software Repositories (MSR’2017), Buenos Aires, Argentina.10.1109/MSR.2017.48 ↗
18. Lank, E., Thorley, J., Chen, S., and Blostein, D. (2001, January 13). On-line recognition of UML diagrams. Proceedings of the Sixth International Conference on Document Analysis and Recognition, Seattle, WA, USA.
19. Maggiori, E., Gervasoni, L., Antúnez, M., Rago, A., and Díaz Pace, J.A. (2014, January 4–5). Towards recovering architectural information from images of architectural diagrams. Proceedings of the XV Simposio Argentino de Ingeniería de Software, Buenos Aires, Argentina.
20. Babalola, O.T. (2015). Automatic Recognition and Interpretation of Finite State Automata Diagrams. [Master’s Thesis, Faculty of Science at Stellenbosch University].
21. Karasneh, B., and Chaudron, M.R.V. (2013, January 27–28). Extracting UML models from images. Proceedings of the 5th International Conference on Computer Science and Information Technology CSIT–2013, Amman, Jordan.10.1109/CSIT.2013.6588776 ↗
22. Karasneh, B., and Chaudron, M.R.V. (2013, January 4–6). Img2UML: A system for extracting UML models from images. Proceedings of the 39th Euromicro Conf. on Software Engineering and Advanced Applications SEAA 2013, Santander, Spain.10.1109/SEAA.2013.45 ↗
23. Karasneh, B., and Chaudron, M.R.V. (2013, January 1). Online Img2UML repository: An online repository for UML models. Proceedings of the 3rd International Workshop on Experiences and Empirical Studies in Software Modeling EESSMOD 2013, Miami, FL, USA.
24. Hjaltason, J., and Samúelsson, I. (2014). Automatic Classification of UML Class Diagrams through Image Feature Extraction and Machine Learning. [Bachelor Thesis, Department of Computer Science and Engineering, University of Gothenburg, Chalmers University of Technology].
25. Ho-Quang, T., Chaudron, M.R.V., Samúelsson, I., Hjaltason, J., Karasneh, B., and Osman, H. (2014, January 1–4). Automatic classification of UML class diagrams from images. Proceedings of the 21st Asia-Pacific Software Engineering Conference APSEC 2014, Jeju, Korea.10.1109/APSEC.2014.65 ↗
26. Hebig, R., Ho-Quang, T., Chaudron, M.R.V., Robles, G., and Fernández, M.A. (2016, January 2–7). The quest for open source projects that use UML: Mining GitHub. Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MoDELS’16), Saint-Malo, France.10.1145/2976767.2976778 ↗
27. Deans, S.R. (1983). The Radon Transform and Some of Its Applications, John Wiley & Sons.
28. Use of the Hough Transformation to Detect Lines and Curves in Pictures10.1145/361237.361242 ↗
29. Random sample consensus—A paradigm for model fitting with applications to image analysis and automated cartography10.1145/358669.358692 ↗
30. Voss, K., Suesse, H., and Ortmann, W. (2004). Radon, Hough, Acumulación y el método SDR. Comunicación Técnica No I-04-05. CC/CIMAT, Centro de Investigación en Matemáticas.
31. Moreno, V., Ledezma, A., and Sanchis, A. (2006, January 13–16). A static images based-system for traffic signs detection. Proceedings of the 24th IASTED International Multi-Conference on Applied Informatics, Innsbruck, Austria.
32. The programmed grammars and automata as tools for a construction of analytic expert systems
33. Flasiński, M. (2002). Automata-based multi-agent model as a tool for constructing real-time intelligent control systems. From Theory to Practice in Multi-Agent Systems, Springer.10.1007/3-540-45941-3_11 ↗
34. Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer.
35. Search-based software engineering10.1016/S0950-5849(01)00189-6 ↗
36. Russell, S., and Norvig, P. (2003). Artificial Intelligence: A Modern Approach, Prentice Hall. [2nd ed.].
37. Weiss, S.M., and Indurkhya, N. (1998). Predictive Data Mining: A Practical Guide, Morgan Kaufmann.
38. The CN2 induction algorithm10.1007/BF00116835 ↗
39. Hong, J., Mozetic, I., and Michalski, R.S. (1986). AQ15: Incremental Learning of Attribute-Based Descriptions from Examples, the Method and User’s Guide, Report ISG 85-5, UIUCDCS-F-86-949, Department of Computer Science, University of Illinois at Urbana-Champaign.
40. Quinlan, J.R. (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann.
41. Induction of Decision Trees (ID3 algorithm)10.1007/BF00116251 ↗
42. PRISM: An algorithm for inducing modular rules10.1016/S0020-7373(87)80003-2 ↗
43. Frank, E., and Witten, I.H. (1998, January 24–27). Generating accurate rule sets without global optimization. Proceedings of the 15th International Conference on Machine Learning ICML-98, Madison, WI, USA.
44. Selecting among rules induced from a hurricane database10.1007/BF00962821 ↗
45. Machine learning research: Four current directions
46. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization10.1023/A:1007607513941 ↗
47. Bagging predictors10.1007/BF00058655 ↗
48. The strength of weak learnability10.1007/BF00116037 ↗
49. Stacked generalization10.1016/S0893-6080(05)80023-1 ↗
50. Witten, I.H., and Frank, E. (2000). Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann.
51. Kohavi, R. (1995, January 20–25). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the 14th International Joint Conference on Artificial Intelligence IJCAI-95, Montreal, QC, Canada.