Manav.id
Developer · 4 min read

The CI runner that can rewrite your entire cloud estate

The CI runner that can rewrite your entire cloud estate

A pull request modifies a module. The runner produces a plan, applies it, and a storage bucket holding customer data becomes publicly readable. Every step executed exactly as configured.

Why is the Terraform runner the most privileged principal in your cloud account?

Because it must be able to create, modify and destroy every resource type it manages. In a mature estate that is broad administrative authority across networking, identity, storage and compute — wider than any individual engineer holds — and it is driven by repository events.

Key takeaways
  • The infrastructure pipeline typically holds wider cloud privilege than any human, because it must create and destroy everything.
  • A plan is reviewable and usually unreviewed — it is long, and nothing binds an approval to the specific plan that applies.
  • Signing a rendered summary of the plan, then verifying the digest at apply time, closes both the review and the drift problem.

The privilege concentration

terraform plan -out=tfplanplan file pinnedRender the security-relevant changesnot the whole diffSign the summary digest + plan hashone gestureApply verifies bothmismatch → refuse
A re-plan between approval and apply produces a different hash, and the apply refuses.

An infrastructure pipeline must be able to create, modify and destroy every resource type it manages. In a mature estate that means broad administrative authority across networking, identity, storage and compute.

Individual engineers usually hold narrower permissions than the pipeline does. The pipeline is the most privileged principal in the account and it is driven by repository events.

TriggerWho controls itTypical review
Merge to mainRepository write accessPull request review
Tag pushAnyone who can push tagsOften none
Scheduled applyThe scheduleNone
Manual dispatchAnyone with workflow dispatchNone
Upstream module updateThe module's publisherDepends on version pinning

The last row is the supply chain dimension. A module sourced from a registry and referenced loosely can change what your plan does without any change in your repository.

Why plan review does not happen

Plans are long. A non-trivial change produces hundreds of lines of diff across resources that mostly did not change in interesting ways.

The security-relevant lines are a handful buried in that output: a policy document, an ingress rule, a public access block, an encryption setting. A reviewer scrolling a CI log reliably misses them.

The second problem is that even a careful review is not binding. The plan reviewed and the plan applied are separate executions, and state can change between them.

Rendering the plan for a decision

The approval prompt should not be the plan. It should be the facts that change the answer.

  Production apply — authorisation required

  Account:  prod (4471-8823-9910)   Region: eu-west-1
  Plan:     +4  ~11  -2

  Security-relevant changes:
    ~ aws_s3_bucket_public_access_block.customer_data
        block_public_acls:  true → false
        block_public_policy: true → false
    ~ aws_security_group.api
        + ingress 0.0.0.0/0 : 5432
    - aws_cloudtrail.audit          (destroy)

  Destroys:  2 resources, 1 stateful
  Plan hash: 3f91c8e2b7d40568

  Touch your security key to authorise this plan.

The classification is mechanical — a fixed list of resource types and attributes that affect exposure, encryption, logging and identity. It is perhaps two hundred lines of matching rules and it turns an unread log into a readable decision.

Binding approval to the plan that applies

# Plan phase
terraform plan -out=tfplan
plan_hash=$(sha256sum tfplan | cut -d' ' -f1)
render_summary tfplan > summary.json
request_signature --digest $(canonical_digest summary.json) \
                  --plan-hash "$plan_hash"

# Apply phase — same plan file, verified
verify_receipt receipt.json --issuer-jwks "$JWKS" || exit 1
[ "$(sha256sum tfplan | cut -d' ' -f1)" = "$(jq -r .plan_hash receipt.json)" ] || {
  echo "plan changed since approval"; exit 1; }
terraform apply tfplan

A re-plan between approval and apply produces a different hash and the apply refuses. This closes the window where infrastructure state or a module version changed after someone said yes.

Scoping so this stays usable

Gating every apply would be unbearable; most applies are tag changes and instance resizes. Two dimensions narrow it sensibly.

In a typical estate that combination fires on a small fraction of applies — which is what makes the prompt something an engineer reads rather than dismisses.

What remains uncovered

Changes made outside the pipeline. Console access, direct API calls and emergency manual fixes bypass this entirely, and every organisation has them.

Drift detection catches the resulting divergence after the fact. Closing the path properly means gating the destructive cloud operations directly, which is the same control applied one layer down.

Why plan review does not happen

Plans are long. A non-trivial change produces hundreds of lines across resources that mostly did not change in interesting ways, and the security-relevant lines are a handful buried in that output: a policy document, an ingress rule, a public access block, an encryption setting. A reviewer scrolling a CI log reliably misses them.

What triggers an apply, and what reviews it
TriggerTypical review
Merge to mainPull request review
Tag pushOften none
Scheduled applyNone
Manual dispatchNone
Upstream module updateDepends entirely on version pinning

The last row is the supply chain dimension: a module referenced loosely can change what your plan does without any change in your repository.

Objections and honest limits

“We review the plan in the pull request.” Even a careful review is not binding, because the plan reviewed and the plan applied are separate executions unless the plan file is pinned and its hash verified.

“Every apply would need a signature.” It should not. Gate production and data-holding environments, and only when the plan touches identity, exposure, encryption, logging or destroys stateful resources. That fires on a small fraction of applies.

Making plans reviewable

  1. Classify the plan mechanically. A fixed list of resource types and attributes affecting exposure, encryption, logging and identity. About two hundred lines of matching rules.
  2. Render the classified subset, not the diff. Plus counts of creates, changes and destroys, and whether any destroyed resource is stateful.
  3. Pin the plan file and hash it. Sign the summary digest together with the plan hash.
  4. Verify both at apply. Refuse if either the receipt fails or the plan file changed.
  5. Gate destructive cloud operations directly too. Console access and manual fixes bypass the pipeline entirely.

Terms used here

Plan file
The saved output of a plan, which an apply can consume directly. Pinning it is what makes an approval binding.
Drift
Divergence between declared infrastructure and actual state, usually caused by changes made outside the pipeline.
Module pinning
Referencing a module by an immutable version so its contents cannot change between plans.

Frequently asked questions

Why not just review the plan in the pull request? Plans are long and the security-relevant lines are few. More importantly, the plan reviewed and the plan applied are separate executions unless the plan file is pinned and verified.

Should every apply require a signature? No. Gate production and data-holding environments, and only when the plan touches identity, exposure, encryption, logging or destroys stateful resources.

What is the plan hash for? Binding approval to the exact plan file. A re-plan between approval and apply produces a different hash, and the apply refuses.

What about changes made in the console? Uncovered. Drift detection finds them afterwards; closing the path means gating the destructive cloud operations directly.

Where this fits in Manav

Manav binds the signature to both the rendered security summary and the plan file hash. A re-plan between approval and apply changes the hash, and the apply stops rather than proceeding under an approval that covered something else.

See plan gating →

Sources and further reading