Announcing Driftctl because drift matters

Why? Because infrastructure is a living thing and changes are risky.

In the team, we all managed cloud infrastructures at some level at our previous jobs, as developers with serverless function needs or DevOps managing large production systems. And we all knew that sometimes an emergency required a manual change, a customer tweaked a setting on the console, the boss activated something that had an impact later…or put more simply, that the whole infrastructure was just not totally under control by Terraform (or similar tool), just because ops life in production is rarely perfect and ideal. 

And all those tiny changes sometimes got forgotten about, or partially implemented in infrastructure-as-code later on, unfortunately leading to security issues, unstable environments, failed deployments, unexpectedly important bills. And thus this makes us lose the advantages of IaC.

Keeping infrastructures secure & in sync with code.

Like many of you, we use and love infrastructure-as-code, and especially Terraform, and for years we relied on its features to ensure the coherence of our infrastructures (most of us ended up with terraform plan in crontabs or in CI, expecting that return code 0 would ensure a fully synchronized infrastructure!).

While Terraform is excellent at provisioning cloud infrastructures, telling you about changes in your AWS account is not where it shines, mainly because it’s simply not its job.

Until now, to be sure that no change happened in your infrastructure, it implied an excellent level of trust in your tools and dependencies, a very stable IaaS provider, every single resource in every region carefully described in Terraform, then the revocation of all admin credentials of your team (and we really think this is as bad for developers’ sense of ownership and empowerment as it is deeply against DevOps culture), most of the time strong CI/CD pipelines and processes, usage of compliance and policy tools, regular auditing, and probably more great ideas to slow down teams velocity. 

But that was clearly not the reality we experienced daily out there! That was often either the goal to reach someday, or a naive belief from management when the team knew very well how to work their way around. But most of the time, credentials were passed around in the team, many resources were still not in Terraform, processes were not perfect, people sometimes forgot to push their locally applied changes, changes happened outside of Terraform, emergencies had to be dealt with quickly (and usually not in Terraform!), consultants came and went, projects were still handed over to customers and no one had time or money to implement complex policies (not to mention that not everyone requires the hassle of PCI/DSS compliance).

And yet, absolutely no one had a good overview of how good or bad things were, easily, so as to take action at some point.

Hope is not a strategy and…that’s why we built driftctl. Driftctl is like your IaC security belt!

Driftctl -  a new open-source project to help you monitor IaC changes.

We built driftctl to help developers know when things change on their IaaS, whatever reason it does, and give them a good overview of their infrastructure-as-code coverage. 

This tool is open-source at its core (Apache-2.0 license) and we decided to release driftctl very early, so we can gather feedback sooner than later and prioritize accordingly. Also, by building this tool publicly from the beginning, we hope that many users will contribute ideas, suggestions, and maybe some code early on. 

The initial releases are focusing on detecting drift on AWS and on Terraform support. Future releases will add support for a lot more, but we didn’t want to wait to release this early. Those releases support what we thought were the most common services on AWS, and this was also backed with hours of interviews with DevOps practitioners around the world those last months. So we started with EC2, S3, IAM, RDS and Lambda, and we’re already working on supporting VPC, CloudFront, Aurora, DynamoDB, API Gateway, SNS/SQS, ECS/EKS/ECR, or KMS. The coming weeks will be exciting!

We also included in the first release a view of your cloud / IaC coverage, so you can quickly know with simple data if your IaaS account is more or less under control with Terraform. Also, if that percentage decreases, it means something changed outside of Terraform! The human-readable output can be switched to JSON if you want to process it. 

Early on, we realized that for our own usage, this wasn’t enough: our own AWS accounts were filled with temporary tests, IAM roles, etc., and even our pretty much ideally locked down and mostly automated production account had a few manual resources (like the initial IAM user!). The output was deceptive as it included a lot of garbage we knew of but couldn’t do anything about. So we also added to the first release two different ways to ignore drifted resources, so you can start fresh with a clean output and start from there. 

This is a community project, and this is the first step in our mission to build an open-source suite of tools to protect codified infrastructures, so definitely feel free to give us a shout to prioritize another service or resource: open a GitHub issue or discussion, and we’d love to chat with you on our Discord. Come say Hi!

Getting Started

Let me show you how to quickly get started with driftctl, once downloaded from https://github.com/cloudskiff/driftctl/releases . We’ll start with the most simple case where you have the Terraform state locally available (beware, usually not a good practice!). 

First, export your AWS environment variables as usual: 

Now, you probably follow good practices and don’t store your state locally or on git but on S3. Here’s how it would work: 

$ driftctl scan --from tfstate+s3://acmecorp/states/terraform.tfstate [...]

Maybe, also as a good practice, you used tags extensively on your AWS resources, so you’d like driftctl to scan only for the Environment: Production key and value? 

$ driftctl scan --filter "Attr.Tags.Environment == 'Production'"

Alternatively, you can exclude some resources directly in the .driftignore file (if you want to ignore the “terraform” IAM user, simply add a line containing aws_iam_user.terraform). Obviously, at any time, you can check the help by executing driftctl scan help.

This is only a quick introduction, a lot more details are included in the Github documentation: https://github.com/cloudskiff/driftctl/tree/main/doc.

We really hope driftctl will find its place in your DevOps toolbox, and that you’ll join us on GitHub and on Discord to make it even better soon!

Stephane Jourdan – CTO and founder