A platform-agnostic middleware for seamless data portability in multi-cloud application deployments
Modern organizations are increasingly using multi-cloud to distribute workloads across different cloud platforms, with the aim of optimizing costs and performance. However, the use of multiple cloud providers creates interoperability and portability issues within the cloud federation. These arise from the lack of a common interface that allows data to be transferred seamlessly between platforms.
Background and existing approaches
Previous research has shown that interoperability and portability between clouds are possible at the semantic level. Many projects introduce a common modeling language to make multi-cloud environments semantically interoperable. However, only a few—and often application-specific—solutions have been proposed for moving an application's data layer between different cloud platforms.
Proposed holistic approach
This thesis proposes a holistic approach to breaking vendor lock-in and enabling data portability between heterogeneous cloud platforms. The approach is validated by the implementation of Prestige, a middleware responsible for live migration of an application's data state between two container-orchestrated clusters, each hosted by a different cloud provider.
Experimental design
A scenario was simulated in which two Kubernetes clusters consume data from two different database solutions: the first hosted on AWS as an RDS instance and the second on GCP as a Cloud SQL instance. Database requests on the first system were then simulated by generating different workloads for various database sizes using sysbench. Finally, a live data migration was performed between AWS and GCP.
Prestige middleware process
Prestige's live migration process is based on three algorithms: resource mapping, dump and restore, and query duplication.
The resource mapping algorithm maps the resources of the source Kubernetes cluster (the cluster to be migrated) to the resources used by the target cluster. This mapping includes authentication data that enables the middleware to connect to both storage systems. The data is then dumped from the source system and restored to the target system. During and after the dump and restore process, all requests that change the state of the source system are duplicated and cached. Once the restore is complete, these changes are replayed on the target system. When the cache is empty, traffic can be redirected to the migrated system with only very limited downtime.
Validate prototype
To validate the Prestige prototype, more than 200 experiments were conducted in which various migration scenarios were simulated by increasing both the database size and the frequency of client requests.
Results
The experiments show promising results. In all cases, data remained 100% consistent after migration, while the downtime required to switch between cloud providers was reduced to a minimum of 0.012 seconds.
