Panoptes: Recovering rogue resources in a multicloud environment with Infrastructure as Code
Historically, managing system architectures has been a time-consuming and error-prone process. DevOps, a set of practices aimed at improving deployment speed and quality, addresses these challenges by promoting automation and consistency. A core practice within DevOps is Infrastructure as Code (IaC), in which infrastructure is described in code and systems are automatically configured based on these definitions. This contrasts with traditional methods, in which administrators configure systems manually and interactively.
Infrastructure as Code in Cloud and Multicloud Environments
IaC is ideally suited for cloud computing because cloud services make it possible to allocate system resources on demand and provision them automatically using IaC. When adopting cloud computing, organizations can choose a single cloud provider or multiple providers at the same time, which is referred to as a multi-cloud environment. Multi-cloud strategies help prevent vendor lock-in and reduce dependence on a single provider. At the same time, managing multiple vendors introduces additional complexity, partly due to differences in products and APIs, which hinders interoperability.
The Challenge of Configuration Drift and Rogue Resources
Despite the advantages of IaC, this approach also presents challenges, including configuration drift, often described as “undocumented configuration changes in a running system.” This research focuses specifically on detecting and remediating rogue resources: resources that fall outside the “state” managed by IaC but still affect resources within that state. Here, “state” refers to the set of resources managed by IaC. Rogue resources can disrupt the operation of IaC, for example, by causing deployments to fail when they depend on infrastructure created outside of IaC. Additionally, they pose a security risk because their existence outside of the IaC documentation makes them difficult to monitor and control. These rogue resources must be addressed to restore the system to a known, manageable state.
Research scope and implementation
This study investigated methods for detecting and remediating rogue resources in multicloud environments. Based on the research findings, a tool was designed and developed to address these challenges in a practical manner. To ensure feasibility within the available timeframe, the scope was deliberately limited. AWS and Hetzner Cloud were chosen as the focus platforms due to their accessibility and the combination of a large cloud provider with a smaller provider. Terraform was selected as the IaC tool due to its widespread adoption and popularity. In addition, the research focuses specifically on compute resources, which allows for a more in-depth analysis of rogue resource management.
Contributions and results
By examining these challenges and implementing concrete solutions, this research contributes to better management of multicloud environments and to increasing the reliability of systems that use IaC. The findings provide in-depth insight into how rogue resources can be effectively addressed, ensuring that systems remain secure, controllable, and operational.
