Migrate a workspace to a new region¶

Use this guide to move a Workspace and its data to a different region. The process sets up ingestion in the new region, backfills historical data, then switches your application over once the new region is validated.

Before you start¶

Before you migrate, ensure:

• You have permission to create a new Organization and Workspace in the target region
• If you're using Option A in Step 2, a timestamp column on your landing Data Sources is preferred, but not required, for the backfill step
• You have a local or Git-hosted version of your Tinybird project so that you can deploy to a new Workspace (tb pull)
• You've decided whether you can tolerate a gap in data during migration. This determines which ingestion approach you use in Step 2

Step 1: Set up the new Workspace¶

1. Create a new Organization and subscription in the target region.
2. Create a new Workspace in that Organization and deploy your project to it:

tb login
tb --cloud deploy --check
tb --cloud deploy

Step 2: Set up ingestion in the new region¶

Choose one of the following approaches, depending on whether you can tolerate a gap in data during migration.

Option A: Duplicate ingestion (no data gap)¶

Point your ingestion process at both the original and new landing Data Sources so both regions receive live ingests. Keep the original region as the source of truth until you complete validation.

Confirm that ingestion in the new region performs identically to the original region before moving on.

Option B: Stop ingestion (accept a data gap)¶

If a gap in data is acceptable, stop ingestion to the original Workspace instead of duplicating it. Complete the Workspace setup and backfill, then start ingestion to the new Workspace only once you're ready to switch over.

Step 3: Backfill historical data¶

Historical data in the original region needs to be exported and replayed into the new Workspace.

1. Create a Sink for each Data Source you need to migrate, to export its data to S3.

   Sinks have a 30 second execution limit. For large Data Sources, filter by time range and run the export in batches.

2. Append the exported data from S3 using an S3 URL.

   • Option A: Append to the new Workspace's temporary landing Data Sources. You reconcile this data with the live stream in the next step.
   • Option B: Append directly to the new Workspace's live Data Sources. There's no live stream to reconcile against, so you can skip the rest of this step.

3. Option A only. Determine your cutoff timestamp. This is the point in time that separates historical data (which you backfill) from live data (which is already ingesting).

   • Preferred, if your landing Data Source has a timestamp column: note the timestamp of the first live event ingested into the new Workspace's live Data Sources:

     Find the cutoff timestamp

     SELECT min(<timestamp_column>) - interval 5 minute
     FROM <live_data_source>
     

     Subtract a few minutes to account for ingestion latency.

   • If you don't have a timestamp column: use the time you started live ingestion to the new Workspace in Step 2 as your cutoff timestamp instead. This is less precise than querying the minimum event timestamp, but works as a fallback.

4. Option A only. Create a Copy Pipe from each temporary landing Data Source to its corresponding live Data Source. Filter for rows that predate the cutoff timestamp, then run the copy.

   Copy Pipes have a 30 second execution limit. Filter by time range and run the copy in batches, or use on-demand compute for larger jobs.

   Filtering by the cutoff timestamp minimizes duplicate data, but late-arriving or out-of-order events can still create gaps. Using ReplacingMergeTree for the landing Data Source absorbs any overlap safely.

Step 4: Validate and switch over¶

1. Test the new region's Endpoints to confirm they return the expected data.
2. Option B only. Start ingestion to the new Workspace.
3. Point your application's Endpoint calls to the new Workspace and region.
4. Option A only. Once traffic is confirmed working in the new region, stop ingestion to the original Workspace.
5. Delete the Workspace and subscription from the original region.

See also¶

• S3 Sink
• Copy Pipes
• Limits