The DynamoDB Connector is now available in Tinybird Forward.

DynamoDB doesn't do aggregations. It's built for point lookups and individual record operations, and it's very good at them. Cross-partition queries are a different story: expensive full-table scans, or secondary indexes you have to maintain by hand. Once your DynamoDB table is mirrored in Tinybird, you can run arbitrary SQL over the full dataset. GROUP BY, window functions, joins with other data sources. Publish the results as low-latency API endpoints. The connector keeps the mirror current with under 5 seconds of lag.

DynamoDB handles your writes and transactional reads. Tinybird handles your analytics. You write Pipes.

Two groups can use this today.

Forward users: pull your DynamoDB tables into Tinybird as part of your standard project structure. Same Git repo, same CI/CD pipeline.

Classic users: the connector was the last missing piece blocking DynamoDB-backed workspaces from migrating to Forward. That gap is now closed.

How to set it up

Run the interactive wizard. It generates the IAM access policy and trust policy, stores the role ARN as a secret, and writes the .connection and .datasource files:

tb connection create dynamodb

Then build to validate, and deploy:

tb build
tb deploy

Full details in the DynamoDB connector documentation.

What the files look like

The connection file holds the IAM role ARN and region:

# connections/my_ddb.connection

TYPE dynamodb
DYNAMODB_ARN {{ tb_secret("dynamodb_role_arn_my_ddb") }}
DYNAMODB_REGION us-east-1

The ARN lives in a secret so it stays out of git. The Data Source file references the connection, sets the table ARN and export bucket, and uses ReplacingMergeTree to track the current state of each row:

# datasources/orders.datasource

SCHEMA >
    `pk`           String                 `json:$.Item.pk`,
    `sk`           String                 `json:$.Item.sk`,
    `_record`      String                 `json:$.NewImage`,
    `_timestamp`   DateTime64(3)          `json:$.ApproximateCreationDateTime`,
    `_event_name`  LowCardinality(String) `json:$.eventName`,
    `_is_deleted`  UInt8                  `json:$._is_deleted`

ENGINE "ReplacingMergeTree"
ENGINE_SORTING_KEY pk, sk
ENGINE_VER _timestamp
ENGINE_IS_DELETED _is_deleted

IMPORT_CONNECTION_NAME my_ddb
IMPORT_TABLE_ARN arn:aws:dynamodb:us-east-1:123456789012:table/orders
IMPORT_EXPORT_BUCKET my-orders-export-bucket

Each row represents a change to your DynamoDB table, not the current state. To query current state, use FINAL. It collapses duplicates and filters deleted rows automatically:

SELECT
    JSONExtractString(_record, 'status') AS status,
    count()                              AS total
FROM orders FINAL
GROUP BY status

Cloud branches and local development

Deploying to your main workspace triggers a full export of your DynamoDB table. Branches don't. The branch Data Source starts empty. That's on purpose: a full PITR export on every branch is slow, and it runs up AWS egress costs for data you mostly don't need in a test environment.

Pull in a sample instead:

tb --branch=my_branch datasource sample orders --wait

By default that scans and imports 1,500 rows from your table. Need more? Bound it by row count or by size up to 10GB (support can update those defaults):

tb --branch=my_branch datasource sample orders --rows 100000 --wait
tb --branch=my_branch datasource sample orders --max-bytes 1GB --wait

If you actually need the full table on a branch, --full-export triggers the same PITR export the main workspace uses:

tb --branch=my_branch datasource sample orders --full-export --wait

The sample is a one-off bounded scan. If you also want live changes while you test, build the branch with --with-connections:

tb --branch=my_branch build --with-connections

That keeps CDC running for the branch: new writes to your table stream into the branch Data Source just like they do in production. Sample for the backfill, stream for the changes. Your main workspace keeps its full, live mirror the whole time.

Local development works the same way. Restart Tinybird Local with your AWS credentials so it can read the table, build with connections, then sample:

tb local restart --use-aws-creds
tb build --with-connections
tb datasource sample orders --wait

Using the TypeScript or Python SDK? Run the wizard to generate the .connection and .datasource files, then run tb migrate to convert them to .ts or .py. See the SDK resources reference for the connection and Data Source definitions.

For Classic users: migrate in two commands

If you've been running DynamoDB workloads in Classic, this was the blocker to moving Forward. It's gone. The migration is:

tb --cloud pull        # downloads your project as datafiles, including .connection and .datasource files
tb migrate-to-forward  # validates, switches the workspace, deploys

Before running migrate-to-forward, check that IMPORT_TABLE_ARN and IMPORT_EXPORT_BUCKET in the pulled files match your existing Classic setup. The migration preserves the DynamoDB binding. It doesn't re-backfill.

After migration, your connector config is a file in your repo. The same .connection file works across environments. Secrets manage the credentials per environment. New branches get the connection automatically as part of the project.

Full walkthrough: Migrate from Classic with connectors.

Get started

tb connection create dynamodb

Full setup: DynamoDB connector documentation.

Migrating from Classic: Migrate from Classic with connectors.

Questions? Slack community.