CloudOps: Towards the Operationalization of the Cloud Continuum: Concepts, Challenges and a Reference Framework
Source abstract
Abstract
The current trend of developing highly distributed, context aware, heterogeneous computing intense and data-sensitive applications is changing the boundaries of cloud computing. Encouraged by the growing IoT paradigm and with flexible edge devices available, an ecosystem of a combination of resources, ranging from high density compute and storage to very lightweight embedded computers running on batteries or solar power, is available for DevOps teams from what is known as the Cloud Continuum. In this dynamic context, manageability is key, as well as controlled operations and resources monitoring for handling anomalies. Unfortunately, the operation and management of such heterogeneous computing environments (including edge, cloud and network services) is complex and operators face challenges such as the continuous optimization and autonomous (re-)deployment of context-aware stateless and stateful applications where, however, they must ensure service continuity while anticipating potential failures in the underlying infrastructure. In this paper, we propose a novel CloudOps workflow (extending the traditional DevOps pipeline), proposing techniques and methods for applications’ operators to fully embrace the possibilities of the Cloud Continuum. Our approach will support DevOps teams in the operationalization of the Cloud Continuum. Secondly, we provide an extensive explanation of the scope, possibilities and future of the CloudOps.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Cloud Computing and Resource Management · Software System Performance and Reliability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
TECNALIA, Basque Research and Technology Alliance (BRTA), Parque Científico y Tecnológico de Bizkaia, 48160 Derio, Spain
Department of Communications Engineering, Faculty of Engineering, University of the Basque Country UPV/EHU, Alda. Urquijo S/N, 48013 Bilbao, Spain
Funding
European Comision
References (82)
Show source-provided references
1. Internet of Things (IoT): A vision, architectural elements, and future directions10.1016/j.future.2013.01.010 ↗
2. The Internet of Things, Fog and Cloud continuum: Integration and challenges10.1016/j.iot.2018.09.005 ↗
3. Mobility-Aware Application Scheduling in Fog Computing10.1109/MCC.2017.27 ↗
4. Cloud Federation Management and Beyond: Requirements, Relevant Standards, and Gaps10.1109/MCC.2016.15 ↗
5. Ometov, A., Molua, O.L., Komarov, M., and Nurmi, J. (2022). A Survey of Security in Cloud, Edge, and Fog Computing. Sensors, 22.10.3390/s22030927 ↗
6. (2022, February 11). Download Infrastructure as Code: Fueling the Fire for Faster Application Delivery—Forrester TLP from Official Microsoft Download Center. Available online: https://www.microsoft.com/en-us/download/details.aspx?id=46403.
7. (2022, February 11). Infrastructure as Code (IaC): The Complete Beginner’s Guide—BMC Software Blogs. Available online: https://www.bmc.com/blogs/infrastructure-as-code/.
8. (2022, February 11). Survey: DevSecOps Progress Remains Elusive—DevOps.com. Available online: https://devops.com/survey-devsecops-progress-remains-elusive/.
9. (2022, February 11). 9 Top Software Development Trends for 2021. Available online: https://www.standardfirms.com/top-software-development-trends/.
10. Fault Analysis and Debugging of Microservice Systems: Industrial Survey, Benchmark System, and Empirical Study10.1109/TSE.2018.2887384 ↗
11. Shacklett, M. (2022, February 11). How to Implement Edge Computing—TechRepublic. Available online: https://www.techrepublic.com/article/how-to-implement-edge-computing/.
12. (2022, February 11). Cloud Edge Computing: Beyond the Data Center—OpenStack is Open Source Software for Creating Private And Public Clouds. Available online: https://www.openstack.org/use-cases/edge-computing/cloud-edge-computing-beyond-the-data-center/.
13. Cloud Benchmarking for Maximising Performance of Scientific Applications10.1109/TCC.2016.2603476 ↗
14. Kousiouris, G., Aisopos, F., Psychas, A., Varvarigou, T., Domaschka, J., Baur, D., Griesinger, F., Nikolov, V., Lyberopoulos, G., and Theodoropoulou, E. (2017, January 17–21). A Toolkit Based Architecture for Optimizing Cloud Management, Performance Evaluation and Provider Selection Processes. Proceedings of the 2017 International Conference on High Performance Computing & Simulation (HPCS), Genoa, Italy.10.1109/HPCS.2017.42 ↗
15. Palit, T., Shen, Y., and Ferdman, M. (2016, January 17–19). Demystifying cloud benchmarking. Proceedings of the 2016 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Uppsala, Sweden.10.1109/ISPASS.2016.7482080 ↗
16. (2022, February 16). CloudSuite—A Benchmark Suite for Cloud Services. Available online: https://www.cloudsuite.ch/.
17. (2022, February 16). Round 20 Results—TechEmpower Framework Benchmarks. Available online: https://www.techempower.com/benchmarks/.
18. (2022, February 16). Home—Filebench/Filebench Wiki—GitHub. Available online: https://github.com/filebench/filebench/wiki.
19. Resource Management in Fog/Edge Computing: A Survey on Architectures, Infrastructure, and Algorithms10.1145/3326066 ↗
20. Tocze, K., Schmitt, N., Brandic, I., Aral, A., and Nadjm-Tehrani, S. (2019, January 16–20). Towards Edge Benchmarking: A Methodology for Characterizing Edge Workloads. Proceedings of the 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W), Umea, Sweden.10.1109/FAS-W.2019.00030 ↗
21. A Taxonomy for Management and Optimization of Multiple Resources in Edge Computing10.1155/2018/7476201 ↗
22. Das, A., Patterson, S., and Wittie, M. (2018, January 17–20). EdgeBench: Benchmarking Edge Computing Platforms. Proceedings of the 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), Zurich, Switzerland.10.1109/UCC-Companion.2018.00053 ↗
23. Zheng, C. (2022, February 16). Bench 2021: 2021 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing. Available online: http://wikicfp.com/cfp/servlet/event.showcfp?eventid=132576©ownerid=168997.
24. Alonso, J., Orue-Echevarria Arrieta, L., Escalante, M., Benguria, G., and Echevarria, G. (2017, January 24–26). Federated Cloud Service Broker (FCSB): An Advanced Cloud Service Intermediator for Public Administrations. Proceedings of the 7th International Conference on Cloud Computing and Services Science, Porto, Portugal.
25. (2022, February 16). GitHub—ale93p/namb: Not Only a Micro-Benchmark. Available online: https://github.com/ale93p/namb.
26. Pagliari, A., Huet, F., and Urvoy-Keller, G. (2019, January 9–12). Towards a High-Level Description for Generating Stream Processing Benchmark Applications. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9006278 ↗
27. Haq, O., Raja, M., and Dogar, F.R. (2017, January 3–7). Measuring and Improving the Reliability of Wide-Area Cloud Paths. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052560 ↗
28. A First Comparative Characterization of Multi-cloud Connectivity in Today’s Internet
29. Jin, Y., Renganathan, S., Ananthanarayanan, G., Jiang, J., Padmanabhan, V.N., Schroder, M., Calder, M., and Krishnamurthy, A. (2019, January 19–23). Zooming in on wide-area latencies to a global cloud provider. Proceedings of the ACM Special Interest Group on Data Communication, Beijing, China.10.1145/3341302.3342073 ↗
30. Osmotic Computing: A New Paradigm for Edge/Cloud Integration10.1109/MCC.2016.124 ↗
31. Next generation cloud computing: New trends and research directions10.1016/j.future.2017.09.020 ↗
32. Hybridizing exact methods and metaheuristics: A taxonomy10.1016/j.ejor.2007.07.035 ↗
33. Heuristics and their design: A survey10.1016/0377-2217(81)90024-2 ↗
34. Gendreau, M., and Potvin, J.-Y. (2010). Handbook of Metaheuristics, Springer.10.1007/978-1-4419-1665-5 ↗
35. Good practice proposal for the implementation, presentation, and comparison of metaheuristics for solving routing problems10.1016/j.neucom.2016.11.098 ↗
36. A heuristic approach to the multicriteria design of IaaS cloud infrastructures for Big Data applications10.1111/exsy.12259 ↗
37. Alonso, J., Stefanidis, K., Orue-Echevarria, L., Blasi, L., Walker, M., Escalante, M., López, M.J., and Dutkowski, S. (2019, January 28–30). DECIDE: An Extended DevOps Framework for Multi-cloud Applications. Proceedings of the 2019 3rd International Conference on Cloud and Big Data Computing, Oxford, UK.
38. (2022, February 17). Terraform by HashiCorp. Available online: https://www.terraform.io/.
39. (2022, February 17). Heat—OpenStack. Available online: https://wiki.openstack.org/wiki/Heat.
40. (2022, February 17). Template Structure and Syntax—Azure Resource Manager—Microsoft Docs. Available online: https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/syntax.
41. (2022, February 17). Google Cloud Deployment Manager Documentation—Cloud Deployment Manager Documentation. Available online: https://cloud.google.com/deployment-manager/docs/.
42. (2022, February 17). Enterprise Kubernetes Management—Rancher. Available online: https://rancher.com/.
43. (2022, February 17). Spinnaker. Available online: https://spinnaker.io/.
44. (2022, February 17). ALIEN 4 Cloud. Available online: https://alien4cloud.github.io/.
45. (2022, February 17). Cloudify Orchestration Platform—Multi Cloud, Cloud Native & Edge. Available online: https://cloudify.co/.
46. (2022, February 17). Kubernetes. Available online: https://kubernetes.io/.
47. Haja, D., Szalay, M., Sonkoly, B., Pongracz, G., and Toka, L. (2019, January 19–23). Sharpening Kubernetes for the Edge. Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos on—SIGCOMM Posters and Demos’ 19, Beijing, China.10.1145/3342280.3342335 ↗
48. Masip-Bruin, X., Marín-Tordera, E., Sánchez-López, S., Garcia, J., Jukan, A., Juan Ferrer, A., Queralt, A., Salis, A., Bartoli, A., and Cankar, M. (2021). Managing the Cloud Continuum: Lessons Learnt from a Real Fog-to-Cloud Deployment. Sensors, 21.10.20944/preprints202104.0074.v1 ↗
49. Cloud resource management: Towards efficient execution of large-scale scientific applications and workflows on complex infrastructures10.1186/s13677-017-0081-4 ↗
50. Towards Self-adaptation Planning for Complex Service-Based Systems10.1007/978-3-319-06859-6_38 ↗
51. (2022, February 11). Push vs. Pull Monitoring Configs. How Do Monitoring Agents Know—by Steve Mushero—Medium. Available online: https://steve-mushero.medium.com/push-vs-pull-configs-for-monitoring-c541eaf9e927.
52. Alonso, J., Orue-Echevarria, L., and Escalante, M. (2019, January 18–20). Contribution to the uptake of Cloud Computing solutions: Design of a cloud services intermediator to foster an ecosystem of trusted, interoperable and legal compliant cloud services. Application to Multi-Cloud Aware Software. Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST), Vienna, Austria.
53. Fowler, M. (2021, March 29). Infrastructure as Code. Available online: https://martinfowler.com/bliki/InfrastructureAsCode.html.
54. Holistic Performance Monitoring of Hybrid Clouds: Complexities and Future Directions10.1109/MCC.2016.13 ↗
55. A survey on web services composition10.1504/IJWGS.2005.007545 ↗
56. Leitner, P., Michlmayr, A., Rosenberg, F., and Dustdar, S. (2010, January 5–10). Monitoring, Prediction and Prevention of SLA Violations in Composite Services. Proceedings of the 2010 IEEE International Conference on Web Services, Miami, FL, USA.10.1109/ICWS.2010.21 ↗
57. Constraint-Based Runtime Prediction of SLA Violations in Service Orchestrations
58. Runtime Prediction of Service Level Agreement Violations for Composite Services10.1007/978-3-642-16132-2_17 ↗
59. De Francisci Morales, G., Bifet, A., Khan, L., Gama, J., and Fan, W. (2016, January 13–17). IoT Big Data Stream Mining. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2945385 ↗
60. A large-scale comparison of concept drift detectors10.1016/j.ins.2018.04.014 ↗
61. Data-driven decision support under concept drift in streamed big data10.1007/s40747-019-00124-4 ↗
62. Rahman, A., Elder, S., Shezan, F.H., Frost, V., Stallings, J., and Williams, L. (2018). Categorizing Defects in Infrastructure as Code. arXiv.
63. Xu, X., Zhu, L., Weber, I., Bass, L., and Sun, D. (2014, January 23–26). POD-Diagnosis: Error Diagnosis of Sporadic Operations on Cloud Applications. Proceedings of the 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Atlanta, GA, USA.10.1109/DSN.2014.94 ↗
64. Angarita, R., Manouvrier, M., and Rukoz, M. (2016, January 23–25). An Agent Architecture to Enable Self-healing and Context-aware Web of Things Applications. Proceedings of the International Conference on Internet of Things and Big Data, Rome, Italy.10.5220/0005932200820087 ↗
65. Self-adapting cloud services orchestration for fulfilling intensive sensory data-driven IoT workflows10.1016/j.future.2020.02.066 ↗
66. RADAR: Self-configuring and self-healing in resource management for enhancing quality of cloud services10.1002/cpe.4834 ↗
67. Toward service selection for workflow reconfiguration:An interface-based computing solution10.1016/j.future.2018.04.064 ↗
68. Self-managing cloud-native applications: Design, implementation, and experience10.1016/j.future.2016.09.002 ↗
69. Trust enforcement through self-adapting cloud workflow orchestration10.1016/j.future.2019.03.004 ↗
70. (2022, February 16). Achieving Application Portability with Kubernetes & Cloud Native. Available online: https://kublr.com/blog/application-portability-with-kubernetes-and-cloud-native/.
71. (2022, February 16). Data Gravity—In the Clouds—Data Gravitas. Available online: https://datagravitas.com/2010/12/07/data-gravity-in-the-clouds/.
72. (2022, February 16). Stateful Services—The Black Sheep of the Container World|D2iQ. Available online: https://d2iq.com/blog/stateful-services-black-sheep-container-world.
73. Towards a computing continuum: Enabling edge-to-cloud integration for data-driven workflows10.1177/1094342019877383 ↗
74. A Unified Model for the Mobile-Edge-Cloud Continuum10.1145/3226644 ↗
75. European Commission (2022, February 12). Work Programme 2018–2020. Available online: https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-leit-ict_en.pdf.
76. Edge Computing: Vision and Challenges10.1109/JIOT.2016.2579198 ↗
77. (2022, March 11). JHipster—Full Stack Platform for the Modern Developer. Available online: https://www.jhipster.tech/.
78. A fast and elitist multiobjective genetic algorithm: NSGA-II10.1109/4235.996017 ↗
79. (2022, April 11). MOEA Framework—A Java Library for Multiobjective Evolutionary Algorithms. Available online: http://moeaframework.org/.
80. (2022, March 11). Telegraf Open Source Server Agent—InfluxDB. Available online: https://www.influxdata.com/time-series-platform/telegraf/.
81. (2022, April 11). InfluxDB: Open Source Time Series Database. Available online: https://www.influxdata.com/.
82. (2022, April 13). Grafana Cloud—Grafana Labs. Available online: https://go2.grafana.com/grafana-cloud.html?src=ggl-s&mdm=cpc&camp=b-grafana-exac-emea&cnt=118483912276&trm=grafana&device=c&gclid=CjwKCAjw6dmSBhBkEiwA_W-EoMb226-I3urBt952aGsX0ocblLsqRjX8W7hia68oxUNGmdBgIdVgBBoCK8cQAvD_BwE.