AMANDA: A Middleware for Automatic Migration between Different Database Paradigms
Source abstract
Abstract
In a world rich in interconnected and complex data, the non-relational database paradigm can better handle large volumes of data at high speed with a scale-out architecture, which are two essential requirements for large industries and world-class applications. This article presents AMANDA, a flexible middleware for automatic migration between relational and non-relational databases based on a user-defined schema that offers support for multiple sources and target databases. We evaluate the performance of AMANDA by assessing the migration speed, query execution, query performance, and migration correctness, from two Relational Database Management Systems (RBMSs), i.e., Postgres and MySQL, to a non-relational database (NoSQL), i.e., DGpraph. The results show that AMANDA successfully migrates data 26 times faster than previous approaches, when considering Northwind. Regarding the IMDB database, it took 7 days to migrate 5.5 GB of data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Database Systems and Queries · Data Management and Algorithms · Service-Oriented Architecture and Web Services
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Sidia R&D Institute, Manaus 69055-035, Brazil
Funding
Samsung Eletrônica da Amazônia Ltda.
References (45)
Show source-provided references
1. Sixty years–and more–of data modelling
2. Kellou-Menouer, K., Kardoulakis, N., Troullinou, G., Kedad, Z., Plexousakis, D., and Kondylakis, H. (2021). A survey on semantic schema discovery. VLDB J., 1–36.10.1007/s00778-021-00717-x ↗
3. Hamouda, S., and Zainol, Z. (2017, January 21–23). Document-Oriented Data Schema for Relational Database Migration to NoSQL. Proceedings of the 2017 International Conference on Big Data Innovations and Applications (Innovate-Data), Prague, Czech Republic.10.1109/Innovate-Data.2017.13 ↗
4. Modeling and Assessing the Temporal Behavior of Emotional and Depressive User Interactions on Social Networks10.1109/ACCESS.2021.3091801 ↗
5. Lee, C.H., and Zheng, Y.L. (2015, January 9–12). SQL-to-NoSQL schema denormalization and migration: A study on content management systems. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.10.1109/SMC.2015.353 ↗
6. Fonseca, S.C., Lucena, M.C., Reis, T.M., Cabral, P.F., Silva, W.A., de Santos, S.F., Giuntini, F.T., and Sales, J. (2021, January 15–19). Automatically Deciding on the Integration of Commits Based on Their Descriptions. Proceedings of the 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), Melbourne, Australia.10.1109/ASE51524.2021.9678906 ↗
7. Machado, R.d.S., Pires, F.d.S., Caldeira, G.R., Giuntini, F.T., Santos, F.d.S., and Fonseca, P.R. (2021). Towards Energy Efficiency in Data Centers: An Industrial Experience Based on Reuse and Layout Changes. Appl. Sci., 11.10.3390/app11114719 ↗
8. Use of UAVs for an efficient capsule distribution and smart path planning for biological pest control10.1016/j.compag.2020.105387 ↗
9. Vehicular Edge Computing: Architecture, Resource Management, Security, and Challenges10.1145/3485129 ↗
10. Schulte, J.P., Giuntini, F.T., Nobre, R.A., Nascimento, K.C.d., Meneguette, R.I., Li, W., Gonçalves, V.P., and Rocha Filho, G.P. (2022). ELINAC: Autoencoder Approach for Electronic Invoices Data Clustering. Appl. Sci., 12.10.3390/app12063008 ↗
11. Modeling and Transformation from Temporal Object Relational Database into Mongodb: Rules10.25046/aj050473 ↗
12. Giuntini, F.T., and Ueyama, J. (2022, January 03). Explorando a Teoria de Grafos e Redes Complexas na Análise de Estruturas de Redes Sociais: Um Estudo de Caso Com a Comunidade Online Reddit. Available online: https://www.researchgate.net/publication/317137094_Explorando_a_teoria_de_grafos_e_redes_complexas_na_analise_de_estruturas_de_redes_sociais_Um_estudo_de_caso_com_a_comunidade_online_Reddit.
13. Cazzolato, M.T., Giuntini, F.T., Ruiz, L.P., de Kirchner, F.L., Passarelli, D.A., de Jesus Dutra dos Reis, M., Traina, C., Ueyama, J., and Traina, A.J.M. (2019, January 7–10). Beyond Tears and Smiles with ReactSet: Records of Users’ Emotions in Facebook Posts. Proceedings of the XXXIV Simpósio Brasileiro de Banco de Dados—Dataset Showcase Workshop (SBBD-DSW), Fortaleza, Brazil.
14. Schema design advisor model for RDBMS to NoSQL database migration
15. Tracing the Emotional Roadmap of Depressive Users on Social Media Through Sequential Pattern Mining10.1109/ACCESS.2021.3095759 ↗
16. (2021, October 04). Oracle. What is Big Data? Big Data Defined. 2022. Available online: www.oracle.com/big-data/what-is-big-data/.
17. Uncertainty in big data analytics: Survey, opportunities, and challenges10.1186/s40537-019-0206-3 ↗
18. Unal, Y., and Oguztuzun, H. (2018, January 11–12). Migration of data from relational database to graph database. Proceedings of the 8th International Conference on Information Systems and Technologies, Amman, Jordan.10.1145/3200842.3200852 ↗
19. On First-Order-Logic Databases10.1145/27629.27630 ↗
20. Freitas, A., Sales, J.E., Handschuh, S., and Curry, E. (2015, January 14–17). How hard is this query? Measuring the Semantic Complexity of Schema-agnostic Queries. Proceedings of the 11th International Conference on Computational Semantics, London, UK.
21. Namdeo, B., and Suman, U. (2021, January 19–20). A Model for Relational to NoSQL database Migration: Snapshot-Live Stream Db Migration Model. Proceedings of the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.10.1109/ICACCS51430.2021.9441829 ↗
22. (2022, May 09). PostgreSQL Global Development Group PostgreSQL—The World’s Most Advanced Open Source Relational Database. Available online: https://www.postgresql.org/.
23. (2022, May 09). MySQL, MySQL—The World’s Most Popular Open Source Database. Available online: https://dev.mysql.com/doc/.
24. (2022, May 09). Oracle Database. Available online: https://www.oracle.com/database/.
25. Chamberlin, D.D., and Boyce, R.F. (1974, January 1–3). SEQUEL: A Structured English Query Language. Proceedings of the 1974 ACM SIGFIDET (Now SIGMOD) Workshop on Data Description, Access and Control, Ann Arbor, MI, USA.
26. Dormando (2022, May 05). A Distributed Memory Object Caching System. Available online: http://memcached.org/.
27. (2022, May 05). The Application Data Platform. Available online: https://www.mongodb.com/.
28. Stax, D. (2022, March 21). Apache Cassandra: About Transactions and Concurrency Control. Available online: https://docs.datastax.com/en/cassandra-oss/2.1/cassandra/dml/dl_about_transactions_c.html.
29. Neo4j (2022, May 09). Concepts: Nosql to Graph—Developer Guides. Available online: https://neo4j.com/developer/graph-db-vs-nosql/.
30. Khasawneh, T.N., AL-Sahlee, M.H., and Safia, A.A. (2020, January 7–9). SQL, NewSQL, and NOSQL Databases: A Comparative Survey. Proceedings of the 2020 11th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan.10.1109/ICICS49469.2020.239513 ↗
31. Li, Y., and Manoharan, S. (2013, January 27–29). A performance comparison of SQL and NoSQL databases. Proceedings of the 2013 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), Victoria, BC, Canada.10.1109/PACRIM.2013.6625441 ↗
32. Martins, P., Abbasi, M., and Sá, F. (2019, January 16–19). A study over NoSQL performance. Proceedings of the World Conference on Information Systems and Technologies, Galicia, Spain.10.1007/978-3-030-16181-1_57 ↗
33. Comparative Analysis of Graph Databases for Git Data10.1088/1742-6596/1944/1/012004 ↗
34. Property oriented relational-to-graph database conversion10.7305/automatika.2017.02.1581 ↗
35. From Relational Database to Big Data: Converting Relational to Graph Database, MOOC Database as Example
36. An efficient graph database model10.35940/ijitee.I7805.0881019 ↗
37. The study on data migration from relational database to graph database10.1088/1742-6596/1345/2/022061 ↗
38. Techniques and guidelines for effective migration from RDBMS to NoSQL10.1007/s11227-018-2361-2 ↗
39. De Virgilio, R., Maccioni, A., and Torlone, R. (2013, January 23–24). Converting relational to graph databases. Proceedings of the First International Workshop on Graph Data Management Experiences and Systems, New York, NY, USA.10.1145/2484425.2484426 ↗
40. Palod, S. (2004). Transformation of Relational DATABASE domain into Graph-Based Domain for Graph-Based Data Mining, The University of Texas at Arlington.
41. R2G: A Tool for Migrating Relations to Graphs
42. Megid, Y.A., El-Tazi, N., and Fahmy, A. (2018, January 3–6). Using functional dependencies in conversion of relational databases to graph databases. Proceedings of the International Conference on Database and Expert Systems Applications, Regensburg, Germany.10.1007/978-3-319-98812-2_31 ↗
43. Migration from an SQL to a hybrid SQL/NoSQL data model
44. Yugabyte (2022, March 21). About the Northwind Sample Database. Available online: https://docs.yugabyte.com/latest/sample-data/northwind/#about-the-northwind-sample-database.
45. (2022, May 15). IMDB. IMDb Datasets. Information Courtesy of IMDb (https://www.imdb.com). Used with Permission. Available online: https://www.imdb.com/interfaces/.