Ship data as you ship code: Deploy changes with a single command

Last week I shared that Tinybird is changing to a local-first experience.

This was the first step in adapting Tinybird to better fit a developer’s workflow, but deploying changes to production was still a manual, error-prone process. Not anymore.

We redesigned the Tinybird CLI to handle the hard parts of managing state changes in data infrastructure so you can focus on shipping.

Painful: With a raw database

Imagine this common scenario: you have an insights page in your live app.

The latency is good, but you realize you can optimize your queries by adding a materialized view. If you’re making changes in a raw database, you need to think about:

1. If you have enough database resources
2. How to break the backfill into chunks and use the right settings
3. How to manage timing to repoint your production workload once the backfill job finishes
4. If it fails, you need to start from scratch or find a way to handle the problem
5. Probably a lot more edge cases...

Less painful: The old Tinybird way

Tinybird has handled some of those things for you, but you still had to figure out how to:

1. Create a new materialized view and push it to Tinybird
2. Backfill the materialized view with data, ensuring no duplicates exist between historical data and live events
3. Edit your endpoints to use the new materialized view and push the changes to Tinybird
4. Clean up the old resources

A seemingly simple optimization required manual steps that were easy to get wrong. This added a ton of friction that prevented you from shipping fast.

The new, happy path: one CLI command

With these changes, it's just this:

1. Make your changes locally
2. Run tb deploy

We’re changing the Tinybird CLI to better manage data projects and state changes. We believe you should be able to do everything locally, even test a deployment.

Think about the new CLI like:

1. A compiler that validates your project
2. A VM that builds and runs your project end-to-end (like a small SaaS in your laptop)
3. A tool to deploy from local → production, managing state changes for you

You don’t usually see so many features at the same time.

Most CLIs follow the UNIX philosophy of “do one thing and do it well.” But in Tinybird’s case, that “one thing” is handling large, complex data projects, so "one thing" becomes many things.

Read on to learn about some of our design decisions…

The onboarding process

We were inspired heavily by Next.js. You run npx create-next-app@latest, answer some basic questions, and – boom – your app is running. In Tinybird, you can now do the same: 

tb create

Like create-next-app, this generates all of the scaffolding for your project. We also added some nice features:

1. Generate .cursorrules to teach your AI code editor to play nicely with the data project resources.
2. Use --prompt to allow an LLM to generate the basic resources for your app.

This way, you can use your AI-assisted IDE to start working fast, no Tinybird experience required (btw, you should try Cursor Agent Mode. I made this video to show how it works).

Two resources

We decided to have a massively reduced set of primitives:

1. Data sources: where you store your data
2. Pipes: how you transform the data and expose it as an API

With those primitives, you can build any project. You only need to learn 2 things, and one of them is pretty much just SQL.

Seed data

Any project needs data, but it becomes critical when your only tool is SQL. You can start up a project with:

That generates the data sources and fixtures to use during the project build.

Sometimes, your data is generated by your app and you don’t have a sample. Don’t worry, we got you covered:

Again, we are using LLMs to help you move faster. Tinybird even saves the prompt in case you want to change it later.

Iteration

Nobody wants to wait 20 minutes for a build. Programming is about iteration, the faster the better. We are investing heavily in a fast feedback loop. To give you some data: When we started testing our first iteration of the new local experience, the build was 30+ seconds. Now, it’s 150ms. And we don't even use Rust. It's Python and C++.

We also added a command to get feedback about your project: tb dev

Again, we were inspired by Next.js and other frameworks. Like those frameworks, Tinybird has a dev command that listens to changes in data project resources and rebuilds the project with each change. The feedback is instant. 

We decided to take it further. Instead of just checking the syntax, we rebuild the project and print the results so you can validate the logic and see the endpoint results in your terminal.

The dev command does not just listen for .datasource and .pipe resources. It also listens for changes in your fixtures and reloads it. So you can edit the fixture and see how different data affects your build.

We are still improving on error reporting. SQL is tricky, and sometimes giving the exact location for an issue is not trivial. More on that later.

Tests

Like before, you can run tb sql “select …. from table” and get results to check things ad hoc.

But now, if you already have the logic, the CLI can generate tests for you:

Again, we are using LLMs to analyze the SQL, generate a set of tests that cover all the possible paths based on the parameters, and generate the expected results. This is usually boring (and often overlooked). Now Tinybird will handle for you.

To run the tests, use the command:

But what happens when you update the logic? You can update the test as well:

This reruns the test and updates the expected result.

Different environments

We are clearly differentiating the execution environments. Each environment has different properties.

There is a disposable build environment that is regenerated on each build. You can still access it and check what’s inside. Think about it like an in-memory database (sqlite).

There is a local environment where you can test the deployment using the Tinybird local container before moving to production.

And finally, there is a production environment where you can deploy to the Tinybird cloud service.

You can access environments with:

You need to pick the right environment based on your objective, but it gives you a lot of flexibility.

Tinybird is moving FORWARD

Internally, we've given this new CLI a codename: FORWARD. It's a good name, not only because it represents a step forward in terms of improving the Tinybird developer experience and continuing to shift toward a local-first workflow, but also because it represents the broader shift Tinybird is making to better serve all developers who need an analytics backend for their app (and want to skip the complex infra work this typically requires).

This is now in beta. If you want to start working with the new CLI, check out the beta docs. Get help and provide feedback in the #forward channel of our Slack Community.