Configure an S3 Backend with DynamoDB Locking in CDKTF

A shared team workflow needs remote state with locking, or two concurrent applies will corrupt it. This guide — part of state backend configuration for CDKTF under CDKTF workflows and Terraform synthesis — wires an S3 bucket for state and a DynamoDB table for locks directly from typed Python, so the backend is defined in the same program as the resources it tracks.

Context

By default CDKTF writes state to a local terraform.tfstate file. That is invisible to teammates, unencrypted, and unlocked — the moment a second engineer runs cdktf deploy, both applies race and the last writer wins, silently dropping resources from state. Moving state to S3 with a DynamoDB lock table, as described for choosing a state backend, makes state durable, encrypted, versioned, and safe for concurrent use.

The DynamoDB table is not a second copy of state, and treating it as one leads people to over-provision it. Terraform keeps exactly two small items there per state object. The lock item has a partition key of <bucket>/<key> and an Info attribute holding JSON — the lock ID, the operation (OperationTypeApply), the user, the host and the creation timestamp. The digest item has a partition key of <bucket>/<key>-md5 and a Digest attribute containing the MD5 of the state document Terraform last wrote, so a run that reads a stale object can detect the mismatch instead of planning against outdated facts.

Acquisition is a conditional write, not a poll. Terraform issues a PutItem carrying an attribute_not_exists(LockID) condition expression; if another apply already owns the row, DynamoDB rejects the write with ConditionalCheckFailedException and the CLI surfaces Error acquiring the state lock. Because the condition is evaluated inside a single-item DynamoDB write, the guarantee is real mutual exclusion rather than an advisory flag that a determined engineer can ignore.

CDKTF changes none of that machinery. cdktf deploy synthesizes cdk.tf.json into cdktf.out/stacks/<stack>/, then shells out to the Terraform binary in that directory, so the backend, the lock and the state document behave exactly as they would for a hand-written HCL configuration. What Python buys you is that the backend arguments become computed values — a stack name becomes a key prefix, an environment name selects a bucket, a naming convention is enforced by a function rather than by review comments — instead of being copy-pasted into every workspace and drifting apart over a year.

Prerequisites

• Python 3.9+ with cdktf>=0.20 and the AWS provider bindings installed
• An S3 bucket with versioning enabled and a DynamoDB table with a LockID string partition key
• AWS credentials with s3:GetObject/PutObject on the bucket and dynamodb:GetItem/PutItem/DeleteItem on the table

# CLI: confirm the backend resources exist before pointing state at them
aws s3api get-bucket-versioning --bucket tf-state-prod
aws dynamodb describe-table --table-name tf-locks-prod --query 'Table.KeySchema'

The partition key must be a string named exactly LockID. Terraform does not read the schema and adapt; it writes an item keyed on that attribute name, so a table created with id or lock_id fails at acquisition time rather than at init.

The permission set is narrower than most teams assume, and getting it wrong produces the two most common errors later in this guide. State access needs s3:GetObject, s3:PutObject and s3:DeleteObject on arn:aws:s3:::tf-state-prod/*, plus s3:ListBucket on the bucket ARN itself — without ListBucket, Terraform cannot distinguish "no state exists yet" from "you may not look", and a first run fails instead of bootstrapping. Locking needs dynamodb:GetItem, dynamodb:PutItem and dynamodb:DeleteItem on the table ARN; DescribeTable is not required for normal operation.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "StateObjectAccess",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::tf-state-prod/*"
    },
    {
      "Sid": "StateBucketList",
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::tf-state-prod"
    },
    {
      "Sid": "StateLockAccess",
      "Effect": "Allow",
      "Action": ["dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:DeleteItem"],
      "Resource": "arn:aws:dynamodb:us-east-1:111122223333:table/tf-locks-prod"
    }
  ]
}

Attach that policy to the CI role and to a break-glass role, not to individual engineers. Anyone holding s3:PutObject on the state prefix can rewrite the recorded facts of your estate, which makes the state bucket a higher-value target than most of the infrastructure it describes — the reasoning in enforcing IAM least privilege in Python IaC applies to the backend before it applies to anything the backend tracks.

Implementation

CDKTF exposes S3Backend as a construct you attach to the stack. Because it is Python, you can compute the state key from the stack name so every stack lands in its own path. The three steps below provision the backend resources, point a stack at them, and move any existing local state across without losing history.

1. Provision the bucket and lock table in a bootstrap stack

The backend cannot store the state of the resources that implement it, so keep them in a separate, rarely-changed stack. That stack starts with local state, is applied once, and is then either left on local state (committed nowhere, recreatable from code) or migrated into itself after the first apply.

# stacks/bootstrap.py — creates the backend resources themselves
# CLI: cdktf deploy backend-bootstrap
from constructs import Construct
from cdktf import App, TerraformStack
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.s3_bucket import S3Bucket
from cdktf_cdktf_provider_aws.s3_bucket_versioning import (
    S3BucketVersioningA,
    S3BucketVersioningVersioningConfiguration,
)
from cdktf_cdktf_provider_aws.s3_bucket_public_access_block import S3BucketPublicAccessBlock
from cdktf_cdktf_provider_aws.dynamodb_table import DynamodbTable, DynamodbTableAttribute


class BackendBootstrapStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str, *, bucket: str, table: str) -> None:
        super().__init__(scope, ns)
        AwsProvider(self, "aws", region="us-east-1")

        state = S3Bucket(self, "state", bucket=bucket)
        # State implication: versioning is what makes a bad apply recoverable —
        # every prior state document stays retrievable by version id.
        S3BucketVersioningA(
            self,
            "state-versioning",
            bucket=state.id,
            versioning_configuration=S3BucketVersioningVersioningConfiguration(status="Enabled"),
        )
        S3BucketPublicAccessBlock(
            self,
            "state-no-public",
            bucket=state.id,
            block_public_acls=True,
            block_public_policy=True,
            ignore_public_acls=True,
            restrict_public_buckets=True,
        )

        # Provider note: PAY_PER_REQUEST avoids capacity planning for a table that
        # sees a handful of writes per deploy.
        DynamodbTable(
            self,
            "locks",
            name=table,
            billing_mode="PAY_PER_REQUEST",
            hash_key="LockID",
            attribute=[DynamodbTableAttribute(name="LockID", type="S")],
        )


app = App()
BackendBootstrapStack(app, "backend-bootstrap", bucket="tf-state-prod", table="tf-locks-prod")
app.synth()

2. Attach the backend to the application stack

# main.py — attach an S3 backend with DynamoDB locking
# CLI: cdktf synth && cdktf deploy
from constructs import Construct
from cdktf import App, TerraformStack, S3Backend
from cdktf_cdktf_provider_aws.provider import AwsProvider

class NetworkStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str) -> None:
        super().__init__(scope, ns)
        AwsProvider(self, "aws", region="us-east-1")
        # State implication: key is per-stack; dynamodb_table serialises applies.
        S3Backend(self,
                  bucket="tf-state-prod",
                  key=f"{ns}/terraform.tfstate",
                  region="us-east-1",
                  dynamodb_table="tf-locks-prod",
                  encrypt=True)

app = App()
NetworkStack(app, "network")
app.synth()

The backend block is emitted into the synthesized cdk.tf.json, so terraform init picks it up automatically on the next deploy. Open that file after cdktf synth and the block sits under the top-level terraform key, next to required_providers:

{
  "terraform": {
    "backend": {
      "s3": {
        "bucket": "tf-state-prod",
        "dynamodb_table": "tf-locks-prod",
        "encrypt": true,
        "key": "network/terraform.tfstate",
        "region": "us-east-1"
      }
    }
  }
}

Two properties of that emission matter in practice. The backend is a stack-level singleton: constructing a second S3Backend in the same TerraformStack replaces the first rather than adding a sibling, so a base class that always attaches a backend and a subclass that overrides it will not collide. And every value inside the block is a literal — Terraform resolves backend configuration before variables or locals exist, so a TerraformVariable reference placed there fails with Variables not allowed. Anything you want parameterised has to be computed in Python at synth time.

That constraint is the argument for funnelling backend arguments through one typed factory instead of retyping them per stack:

# backend.py — one typed source of truth for every stack's backend arguments
# CLI: cdktf synth
from dataclasses import dataclass
from cdktf import S3Backend, TerraformStack


@dataclass(frozen=True)
class BackendSettings:
    environment: str
    region: str = "us-east-1"

    @property
    def bucket(self) -> str:
        return f"tf-state-{self.environment}"

    @property
    def lock_table(self) -> str:
        return f"tf-locks-{self.environment}"


def attach_backend(stack: TerraformStack, ns: str, settings: BackendSettings) -> S3Backend:
    # State implication: `key` is the address of the state object, not a label.
    # Pick the convention before the first apply — this one nests under the
    # environment, so it is NOT interchangeable with a bare "<ns>/terraform.tfstate".
    backend = S3Backend(
        stack,
        bucket=settings.bucket,
        key=f"{settings.environment}/{ns}/terraform.tfstate",
        region=settings.region,
        dynamodb_table=settings.lock_table,
        encrypt=True,
    )
    # Provider note: Terraform 1.10+ can also hold a native lock object at
    # <key>.tflock in the same bucket. Older cdktf bindings expose no typed
    # argument for it, so set it through the stack's JSON escape hatch.
    stack.add_override("terraform.backend.s3.use_lockfile", True)
    return backend

add_override writes straight into the synthesized JSON tree by dotted path, which is the standard way to reach a Terraform argument that the generated Python bindings predate. It is unvalidated — a typo in the path produces a silently ignored key, not an error — so keep overrides few and assert on them in a synth-time test.

3. Migrate existing local state into the bucket

A stack that has already run locally has a populated terraform.tfstate sitting beside cdk.tf.json. Adding the backend does not move it. The next cdktf deploy runs a plain terraform init, which detects the change and halts:

Error: Backend configuration changed

A change in the backend configuration has been detected, which may require
migrating existing state.

If you wish to attempt automatic migration of the state, use "terraform init -migrate-state".

CDKTF has no flag to pass that through, so run the migration by hand in the synthesized directory, exactly once:

# CLI: synth first so cdk.tf.json already carries the new backend block
cdktf synth
cd cdktf.out/stacks/network
terraform init -migrate-state
# Prompt: "Do you want to copy existing state to the new backend?" — answer yes.
# It uploads the local document and leaves terraform.tfstate.backup on disk.
cd -

Confirm the object and its first version landed before anyone applies again, then delete the local file and add *.tfstate* plus cdktf.out/ to .gitignore so a stale copy cannot be resurrected from a branch:

# CLI: the state object should exist and already have a version history
aws s3api list-object-versions --bucket tf-state-prod \
  --prefix network/terraform.tfstate \
  --query 'Versions[].[VersionId,LastModified,Size]' --output table

If the migration prompt reports 0 resources where you expected dozens, the local file you migrated was an empty scaffold from a cdktf synth in a clean checkout rather than the state from the machine that actually applied. Recover the real one from that machine's working directory before touching the backend again — a migration that copies an empty document over a populated one is indistinguishable, on the next plan, from someone having deleted the entire estate.

Verification

After the first deploy, confirm state moved off local disk and that a lock is taken during apply.

# CLI: state object exists in S3, and no stale local state remains
aws s3 ls s3://tf-state-prod/network/
test ! -f terraform.tfstate && echo "local state is gone (good)"

To see locking in action, start a deploy and, in another shell, run a second deploy — it should block with a Error acquiring the state lock message rather than proceeding.

Troubleshooting

Error acquiring the state lock that never clears — a previous apply crashed and left a lock row. The message carries the evidence you need before acting: Lock Info prints the lock ID, the Path (tf-state-prod/network/terraform.tfstate), Operation: OperationTypeApply, Who as user@host, the Terraform Version and Created timestamp. Compare Who and Created against your CI run history; only when you are certain no apply is still running, terraform force-unlock <ID> from inside cdktf.out/stacks/<stack>. Forcing a lock while an apply is genuinely mid-flight lets a second apply plan against a state document the first is about to overwrite, which is precisely the corruption the table exists to prevent.

AccessDenied on the DynamoDB table — the credentials can read state but not write locks; add the dynamodb:PutItem and DeleteItem permissions. The tell is that terraform init succeeds and terraform plan fails immediately, before any provider work, because init only touches S3 while plan is the first command to take a lock.

state data in S3 does not have the expected content — the digest item and the object disagree. Terraform's own advice in that message is to wait a minute and retry, because the usual cause is S3 read-after-write lag on an overwritten object. When it persists, someone has written the object out of band — a manual aws s3 cp, a restored version, or a second workspace sharing the key. The message ends with the MD5 Terraform expects; either restore the matching object version or update the Digest attribute on the <bucket>/<key>-md5 item to that value, then re-plan and read the diff carefully.

Error inspecting states in the "s3" backend: AccessDenied on a first run — this is the missing s3:ListBucket case. The object-level grant is present, so a targeted read would work, but Terraform enumerates the prefix to discover workspaces and gets nothing it can distinguish from a permissions wall.

State not encrypted — set encrypt=True (above) and enable default SSE-KMS on the bucket; otherwise a compliance scan with Checkov will flag the state object. If you supply kms_key_id, the CI role also needs kms:Encrypt, kms:Decrypt and kms:GenerateDataKey on that key, and the failure without them is a KMS.NotFoundException or AccessDeniedException raised during the state write — after the infrastructure change has already been made in the cloud but before it has been recorded.

Operational Notes

Once the backend is live, most of the operational surface is the lock table and the object versions behind it. Knowing exactly what is written where turns an incident from guesswork into two aws commands.

Neither item has a TTL. DynamoDB will hold the lock row until Terraform deletes it, which means a runner killed by a pipeline timeout, a spot reclaim or an engineer's Ctrl-C at the wrong instant leaves a lock that outlives the process by definition. Inspect it directly rather than guessing:

# CLI: read the live lock row before deciding whether to force-unlock
aws dynamodb get-item --table-name tf-locks-prod \
  --key '{"LockID": {"S": "tf-state-prod/network/terraform.tfstate"}}' \
  --query 'Item.Info.S' --output text

Give applies a bounded wait instead of an instant failure by passing -lock-timeout, which makes Terraform retry acquisition rather than exit on the first ConditionalCheckFailedException. CDKTF forwards extra Terraform arguments through the TF_CLI_ARGS_apply and TF_CLI_ARGS_plan environment variables, since cdktf deploy builds the Terraform command line itself:

# CLI: queue behind a running apply for up to five minutes instead of failing fast
TF_CLI_ARGS_apply="-lock-timeout=5m" TF_CLI_ARGS_plan="-lock-timeout=5m" cdktf deploy network

A pipeline-level concurrency group is the cheaper first line of defence: serialise the job so two runs never reach acquisition at the same time, and treat a lock error as the failure of that guard rather than as routine. The pattern is covered end to end in running CDKTF pipelines in GitHub Actions.

Versioning makes the bucket grow without bound, because every apply writes a full new state document and the old one becomes a noncurrent version. A stack with a few hundred resources produces a state object measured in hundreds of kilobytes, and a busy repository writes it dozens of times a day. Keep a lifecycle rule that expires noncurrent versions after a retention window long enough to cover a rollback — thirty to ninety days is typical — rather than deleting them eagerly:

# lifecycle.py — bound state history without losing the rollback window
# CLI: cdktf deploy backend-bootstrap
from cdktf_cdktf_provider_aws.s3_bucket_lifecycle_configuration import (
    S3BucketLifecycleConfigurationA,
    S3BucketLifecycleConfigurationRule,
    S3BucketLifecycleConfigurationRuleNoncurrentVersionExpiration,
)

# State implication: only NONCURRENT versions expire — the live state object is
# never touched by this rule, so a misconfigured window cannot delete state.
S3BucketLifecycleConfigurationA(
    self,
    "state-history",
    bucket=state.id,
    rule=[
        S3BucketLifecycleConfigurationRule(
            id="expire-old-state-versions",
            status="Enabled",
            noncurrent_version_expiration=(
                S3BucketLifecycleConfigurationRuleNoncurrentVersionExpiration(
                    noncurrent_days=90,
                )
            ),
        )
    ],
)

Rolling back is then a bucket operation, not a Terraform one. List the versions of the state key, copy the version from before the bad apply over the current object, and let the digest mismatch surface if anything else changed in the meantime. Because the copy bypasses Terraform entirely, take the lock first — or do it in a window where you know nothing else can run — and follow it with a plan you read line by line rather than an apply.

FAQ

Can several stacks share one bucket?

Yes — give each stack a distinct key (e.g. derived from the stack name) so their state objects never overlap while sharing the same bucket and lock table.

Do I need DynamoDB if I use Terraform Cloud?

No. Terraform Cloud provides its own locking; the DynamoDB table is only for the S3 backend. See using Terraform Cloud with CDKTF.

Is the backend config in state?

No — backend settings live in the synthesized configuration, not in state, so changing them requires a terraform init -migrate-state. Terraform does cache the last-used backend in .terraform/terraform.tfstate inside the synthesized directory, which is why a stale cdktf.out/ can make a corrected backend look like it was ignored.

How do I recover state after a bad apply?

Use S3 versioning. aws s3api list-object-versions on the state key gives you every document Terraform has written; copy the version from immediately before the bad apply back over the current object with aws s3api copy-object, then run a plan and read it in full before applying anything. Recovery restores what Terraform believed, not what exists in the account, so expect the plan to show the drift between the two.

What does the lock table cost to run?

Effectively nothing on PAY_PER_REQUEST. Each Terraform operation performs one PutItem, one GetItem for the digest, one digest PutItem and one DeleteItem — four small writes and reads per apply, so even a repository deploying hundreds of times a day stays well inside cents per month. Provisioned capacity on a lock table is wasted money and adds throttling as a new failure mode.

Should I still use DynamoDB on Terraform 1.11 or later?

The dynamodb_table argument is deprecated in favour of the native S3 lock object enabled by use_lockfile, but it still works and still interoperates. Run both for one migration window so that older and newer Terraform binaries in the same organisation cannot bypass each other's locks, then drop the table once every runner is on a version that writes the lock file.

Related

• State Backend Configuration for CDKTF — the parent topic on remote state and locking.
• Using Terraform Cloud with CDKTF Python Projects — an alternative backend with managed locking and runs.
• Choosing a State Backend for Python IaC — how to weigh S3, Terraform Cloud, and Pulumi Cloud.