Get started with Tinybird¶
Follow these steps to install Tinybird Local and Tinybird CLI on your machine, build your first data project, and deploy it to Tinybird Cloud.
See Core concepts for a complete overview of Tinybird.
Before you begin¶
To get started, you need the following:
- A container runtime, like Docker or Orbstack
- Linux or macOS
Deploy a new project in five minutes¶
Create a Tinybird account¶
If you don't already have a Tinybird account, you can create one at cloud.tinybird.co -- it's free!
Install and authenticate¶
Run the following command to install the Tinybird CLI:
curl https://tinybird.co | sh
Then, authenticate with your Tinybird account using tb login:
tb login
In the browser, create a new workspace or select an existing one.
Run Tinybird Local¶
After you've authenticated, run tb local start to start a Tinybird Local instance in a Docker container, allowing you to develop and test your project locally.
tb local start
Create a project¶
Pass the data to the CLI using the --data flag and a --prompt explaining what you want to build:
If you don't have any data handy, just use the Washington State Department of Transportation's Electric Vehicle Population Data in CSV.
tb create --data https://data.wa.gov/api/views/f6w7-q2d2/rows.csv\?accessType\=DOWNLOAD --prompt "I am passing you a url with electric vehicle population. I want an endpoit that serves the ranking of the most common models. By default I want to see all types, but I want to be able to pass vehicle type as parameter. I also need a limit parameter to control the number of rows returned." » Creating new project structure... Learn more about data files https://www.tinybird.co/docs/forward/datafiles /datasources → Where your data lives. Define the schema and settings for your tables. /endpoints → Expose real-time HTTP APIs of your transformed data. /materializations → Stream continuous updates of the result of a pipe into a new data source. /copies → Capture the result of a pipe at a moment in time and write it into a target data source. /pipes → Transform your data and reuse the logic in endpoints, materializations and copies. /fixtures → Files with sample data for your project. /tests → Test your pipe files with data validation tests. /connections → Connect to and ingest data from popular sources: Kafka, S3 or GCS. ✓ Scaffolding completed! » Creating resources... /datasources/rows.datasource /endpoints/rows_endpoint.pipe /endpoints/model_ranking.pipe » Creating project description... README.md ✓ Resources created! » Generating fixtures... /fixtures/rows.csv ✓ Done!
The previous prompt creates a project in the current directory. Let's take a look at some of the files it created:
/datasources/rows.datasource¶
Data sources are the definition of the database tables where you will store the data. More information about data sources here. Inspecting the file you see a description and the schema, with the column names and their types.
datasources/rows.datasource
DESCRIPTION >
Generated from https://data.wa.gov/api/views/f6w7-q2d2/rows.csv?accessType=DOWNLOAD
SCHEMA >
`vin__1_10_` String,
`county` String,
`city` String,
`state` String,
`postal_code` Int64,
`model_year` Int32,
`make` String,
`model` String,
`electric_vehicle_type` String,
`clean_alternative_fuel_vehicle__cafv__eligibility` String,
`electric_range` Int32,
`base_msrp` Int64,
`legislative_district` Int16,
`dol_vehicle_id` Int64,
`vehicle_location` String,
`electric_utility` String,
`c_2020_census_tract` Int64
/fixtures/rows.csv¶
A file with a sample of the data so you can test locally.
fixtures/rows.csv
VIN (1-10),County,City,State,Postal Code,Model Year,Make,Model,Electric Vehicle Type,Clean Alternative Fuel Vehicle (CAFV) Eligibility,Electric Range,Base MSRP,Legislative District,DOL Vehicle ID,Vehicle Location,Electric Utility,2020 Census Tract 1C4JJXP68P,Yakima,Yakima,WA,98901,2023,JEEP,WRANGLER,Plug-in Hybrid Electric Vehicle (PHEV),Not eligible due to low battery range,21,0,15,249905295,POINT (-120.50729 46.60464),PACIFICORP,53077001601 1G1FY6S03L,Kitsap,Kingston,WA,98346,2020,CHEVROLET,BOLT EV,Battery Electric Vehicle (BEV),Clean Alternative Fuel Vehicle Eligible,259,0,23,141133765,POINT (-122.4977 47.79802),PUGET SOUND ENERGY INC,53035940000 KNDCE3LG9K,King,Seattle,WA,98125,2019,KIA,NIRO,Battery Electric Vehicle (BEV),Clean Alternative Fuel Vehicle Eligible,239,0,46,3410074,POINT (-122.30253 47.72656),CITY OF SEATTLE - (WA)|CITY OF TACOMA - (WA),53033000101 1N4AZ0CP8E,Thurston,Olympia,WA,98506,2014,NISSAN,LEAF,Battery Electric Vehicle (BEV),Clean Alternative Fuel Vehicle Eligible,84,0,22,182436474,POINT (-122.87741 47.05997),PUGET SOUND ENERGY INC,53067012100 5YJXCAE29L,Kitsap,Silverdale,WA,98383,2020,TESLA,MODEL X,Battery Electric Vehicle (BEV),Clean Alternative Fuel Vehicle Eligible,293,0,23,1843054,POINT (-122.69275 47.65171),PUGET SOUND ENERGY INC,53035091206
/endpoints/model_ranking.pipe¶
Endpoints are a kind of pipe that you can call from other applications. You have data in a data source, use a pipe to build SQL logic, and then publish the result of your query as a REST API endpoint. Pipes contain just SQL and a templating language that lets you add query parameters to the API. More details about Endpoints here.
endpoints/model_ranking.pipe
DESCRIPTION >
Returns the ranking of most common electric vehicle models.
NODE model_ranking_node
SQL >
%
SELECT
make,
model,
electric_vehicle_type,
count() as vehicle_count
FROM rows
WHERE
{% if defined(vehicle_type) %}
electric_vehicle_type = {{String(vehicle_type, 'Battery Electric Vehicle (BEV)')}}
{% else %}
1=1
{% end %}
GROUP BY make, model, electric_vehicle_type
ORDER BY vehicle_count DESC
LIMIT {{Int32(limit, 10)}}
TYPE ENDPOINT
Run the development server¶
To start developing, run the tb dev command and start editing the data files within the created project directory. This command starts the development server and also provides a console to interact with the database. The project will automatically rebuild and reload upon saving changes to any file.
tb dev » Building project... ✓ datasources/rows.datasource created ✓ endpoints/rows_endpoint.pipe created ✓ endpoints/model_ranking.pipe created Error appending fixtures for 'rows': There was an error with file contents: 564 rows in quarantine. ✓ Build completed in 9.1s Watching for changes... tb »
The build returns a Quarantine error.
tb » select distinct c__error from rows_quarantine » Running QUERY ──────────────────────────────────────────────────────────────────────────────────────────── c__error: ["value '' on column 'postal_code' is not Int64", "value '' on column 'legislative_district' is not Int16", "value '' on column 'c_2020_census_tract' is not Int64"] ──────────────────────────────────────────────────────────────────────────────────────────── c__error: ["value '' on column 'electric_range' is not Int32", "value '' on column 'base_msrp' is not Int64"] ──────────────────────────────────────────────────────────────────────────────────────────── c__error: ["value '' on column 'legislative_district' is not Int16"] ────────────────────────────────────────────────────────────────────────────────────────────
The problem is that some columns should be Nullable or have a DEFAULT value. Let's proceed with adding a DEFAULT value of 0 for them.
Edit the datasources/rows.datasource file
datasources/rows.datasource
DESCRIPTION >
Generated from https://data.wa.gov/api/views/f6w7-q2d2/rows.csv?accessType=DOWNLOAD
SCHEMA >
`vin__1_10_` String,
`county` String,
`city` String,
`state` String,
`postal_code` Int64 DEFAULT 0,
`model_year` Int32,
`make` String,
`model` String,
`electric_vehicle_type` String,
`clean_alternative_fuel_vehicle__cafv__eligibility` String,
`electric_range` Int32 DEFAULT 0,
`base_msrp` Int64 DEFAULT 0,
`legislative_district` Int16 DEFAULT 0,
`dol_vehicle_id` Int64,
`vehicle_location` String,
`electric_utility` String,
`c_2020_census_tract` Int64 DEFAULT 0
The dev server will rebuild the edited resources.
tb » ⟲ Changes detected in rows.datasource » Rebuilding project... ✓ datasources/rows.datasource changed ✓ Rebuild completed in 1.1s
No errors now, let's continue.
Test the API Endpoint¶
The goal is to have a working endpoint that you can call from other applications.
Default local url is http://localhost:7181 and token with admin permissions is admin local_testing@tinybird.co.
Outside the dev server, copy the token value with tb token copy and use it to call the endpoint. Send the limit parameter with a value of 3 for a smaller response.
tb token copy "admin local_testing@tinybird.co" && TB_LOCAL_TOKEN=$(pbpaste)
curl -X GET "http://localhost:7181/v0/pipes/model_ranking.json?token=$TB_LOCAL_TOKEN&limit=3"
Running against Tinybird Local
** Token 'admin local_testing@tinybird.co' copied to clipboard
{
"meta":
[
{
"name": "make",
"type": "String"
},
{
"name": "model",
"type": "String"
},
{
"name": "electric_vehicle_type",
"type": "String"
},
{
"name": "vehicle_count",
"type": "UInt64"
}
],
"data":
[
{
"make": "TESLA",
"model": "MODEL Y",
"electric_vehicle_type": "Battery Electric Vehicle (BEV)",
"vehicle_count": 49588
},
{
"make": "TESLA",
"model": "MODEL 3",
"electric_vehicle_type": "Battery Electric Vehicle (BEV)",
"vehicle_count": 36085
},
{
"make": "NISSAN",
"model": "LEAF",
"electric_vehicle_type": "Battery Electric Vehicle (BEV)",
"vehicle_count": 13853
}
],
"rows": 3,
"rows_before_limit_at_least": 175,
"statistics":
{
"elapsed": 0.041374271,
"rows_read": 239183,
"bytes_read": 16906256
}
}
Deploy to Tinybird Cloud¶
To deploy to Tinybird Cloud, create a deployment using the --cloud flag. This prepares all the resources in the cloud environment.
tb --cloud deploy
Append data to Tinybird Cloud¶
Use tb datasource append with the --cloud flag to ingest the data from the URL into Tinybird Cloud.
tb --cloud datasource append rows https://data.wa.gov/api/views/f6w7-q2d2/rows.csv\?accessType\=DOWNLOAD
Open the project in Tinybird Cloud¶
To open the project in Tinybird Cloud, run the following command:
tb --cloud open
Go to Endpoints and select an endpoint to see stats and snippets.
Next steps¶
- Familiarize yourself with Tinybird concepts. See Core concepts.
- Learn about datafiles, like .datasource and .pipe files. See Datafiles.
- Get data into Tinybird from a variety of sources. See Get data in.
- Browse the Tinybird CLI commands reference. See Commands reference.
- Learn with a more detailed example. See Learn.
- Detect and fix Quarantine errors. See Quarantine.