How to Scale End-to-End Observability in AWS Environments

Adding IaC security scans to your CI pipeline

The adoption of infrastructure as code (IaC) has skyrocketed in recent years as engineers seek ways to deploy cloud infrastructure faster and more efficiently. IaC refers to the technologies and processes that manage and provision infrastructure using machine-readable languages (code) as opposed to inefficient manual operations.

Languages and frameworks, like Terraform and Serverless, provide a unified language to codify infrastructure and streamline cloud orchestration across different environments and providers. Alternatively, cloud providers’ native IaC frameworks, such as AWS CloudFormation and Azure Resource Manager (ARM), transform manual, one-off processes into consistent, scalable, and repeatable provisioning.

Shifting cloud security left with IaC

IaC shifts infrastructure deployment left, providing new opportunities to automate, scale, and secure cloud environments.

Historically, teams have had to approach cloud security after the fact. They did this through resource monitoring for policy violations during run-time. Even with automation in place, this approach can end up being time-consuming and arduous for engineers that need to address identified issues reactively. By extrapolating cloud infrastructure into code and embedding it into the development lifecycle, teams can now address cloud security preventatively.

With IaC in place, you can enforce security best practices alongside your container scanning, dependency scanning, SAST, DAST, and more. However, for IaC security to be truly effective, it must be embedded into automated CI pipelines.

Getting started with security enforcement

How do you find policy violations and misconfigurations in the infrastructure used to build AWS, Google Cloud, Azure, and Kubernetes resources? By adding IoT security scans to your CI pipeline. Bridgecrew works by continuously scanning infrastructure to identify known security vulnerabilities and providing the code to fix them. For example, here are a few commonly overlooked policies:

Ensuring all data stored in S3 bucket is securely encrypted at rest
Ensuring no security groups allow ingress from 0.0.0.0:0 to port 22
Ensuring all data stored in the Launch configuration EBS volume is securely encrypted
To start identifying those issues, and scan your IaC locally, install Bridgecrew for the appropriate build environment:

## Standard installation
pip install bridgecrew

## Installation on Linux / Mac distros where `python` references python2
## (this is usually the case - run `python --version` to verify)
pip3 install bridgecrew

## Installation on Alpine
pip3 install --upgrade pip && pip3 install --upgrade setuptools\npip3 install bridgecrew

## Installation using homebrew (MacOS only)
brew tap bridgecrewio/bridgecrew https://github.com/bridgecrewio/bridgecrew\nbrew update\nbrew install bridgecrew

## Installation on Windows (ensure you add the cmd file to your path)
pip install bridgecrew\n
echo %PATH%\n
cd C:\\Users\\<my_username>\\AppData\\Local\\Microsoft\\WindowsApps\n
curl -o bridgecrew.cmd https://raw.githubusercontent.com/bridgecrewio/bridgecrew/master/bin/bridgecrew.cmd

Next, sign up for a free Bridgecrew account, retrieve your unique API token, and scan a local directory or file:

##directory
- bridgecrew -d <directory> --bc-api-key <key> --repo-id <repo_id> --branch <branch>

##file
- bridgecrew -f <file_1> <file_2> ... <file_n> --bc-api-key <key> --repo-id <repo_id> --branch <branch>

Bridgecrew will then print the following results to your command line, showing failing and passing issues:

_          _     _ 
| |__  _ __(_) __| | __ _  ___  ___ _ __ _____      __ 
| '_ \| '__| |/ _` |/ _` |/ _ \/ __| '__/ _ \ \ /\ / / 
| |_) | |  | | (_| | (_| |  __/ (__| |  | __/\ V  V / 
|_.__/|_|  |_|\__,_|\__, |\___|\___|_|  \___| \_/\_/ 
                    |.___/ 

by bridgecrew.io | version: 1.0.455

terraform scan results:

Passed checks: 2, Failed checks: 1, Skipped checks: 0

Check: "Ensure all data stored in the S3 bucket is securely encrypted at rest"
PASSED for resource: aws_s3_bucket.foo-bucket
File: /example.tf:1-25

Check: "Ensure the S3 bucket has access logging enabled"
PASSED for resource: aws_s3_bucket.foo-bucket
File: /example.tf:1-25

Check: "S3 Bucket has an ACL defined which allows public access."
FAILED for resource: aws_s3_bucket.foo-bucket
File: /example.tf:1-25

1 | resource "aws_s3_bucket" "foo-bucket" {
2 | region = var.region
3 | bucket = local.bucket_name
4 | force_destroy = true
5 |
6 | tags = {
7 | Name = "foo-${data.aws_caller_identity.current.account_id}"
8 | }
9 | versioning {
10 | enabled = true
11 | }
12 | logging {
13 | target_bucket = "${aws_s3_bucket.log_bucket.id}"
14 | target_prefix = "log/"
15 | }
16 | server_side_encryption_configuration {
17 | rule {
18 | apply_server_side_encryption_by_default {
19 | kms_master_key_id = "${aws_kms_key.mykey.arn}"
20 | sse_algorithm = "aws:kms"
21 | }
22 | }
23 | }
24 | acl = "public-read"
25 | }

In the output above, two checks passed, and one check for “S3 Bucket has an ACL defined which allows public access” failed. This was done manually and locally. Automating IaC security with scanning is the only way to get complete coverage across security industry best practices and compliance benchmarks.

Using Bridgecrew for automated IaC security

For IaC security to be efficient and actionable, it needs to provide consistent feedback. The best way to get that consistency is by embedding IaC security into your automated CI pipelines.

Integrating Bridgecrew with CircleCI is done with the Bridgecrew orb. Configure a new CircleCI environment variable, BC_API_KEY, using the API token we found earlier. Next, configure your CircleCI workflow to trigger a Bridgecrew scan on every one of your builds. Your .circleci/config.yml will look similar to what you see below:

version: 2.1
orbs:
    bridgecrew: bridgecrew/bridgecrew@1.0.3
jobs:
    build:
      executor: bridgecrew/default
      steps:
        - checkout
        - bridgecrew/scan:
            directory: ‘.’
            api-key-variable: BC_API_KEY

Once you’ve added the Bridgecrew orb to your configuration file, the designated IaC directory will be scanned against hundreds of security compliance policies. Optionally, you can configure your CircleCI builds to fail should any policy violations be detected.

To conduct further investigation on incidents by severity, category, or compliance benchmarks, you can head to the Bridgecrew platform. Once there, you are able to drill into issues to understand their impact and affected resources, and fix the code appropriately:

With a pull request integration in place, you can instantly push remediations to your VCS, merge, and build again. This continuous workflow is the best way to ensure misconfigurations don’t get deployed into provisioned cloud resources. 🙌

Conclusion

IaC is an incredibly powerful tool to help cloud-native teams take their infrastructure to the next level with benefits related to speed, scalability, cost-savings, and security. By automating your IaC security, you can leverage the inherent immutability of IaC to harden your cloud security posture and save time doing it.

This post originally appeared in theCircleCI Blog.

Get similar stories in your inbox weekly, for free

Share this story:

Bridgecrew

Bridgecrew is the cloud security platform for developers. By leveraging automation and delivering security-as-code, Bridgecrew empowers teams to find, fix, and prevent misconfigurations in deployed cloud resources and in infrastructure as code.