Events API

The Events API is designed to enable high throughput streaming ingestion into Tinybird from an easy to use HTTP API.

On this page you will find details of how to use the Events API to perform various tasks. You can find the Events API Reference here.

Sending individual JSON events

You can send individual JSON events to the Events API by including the JSON event in the Request body.

For example, to send an individual JSON event using cURL:

Sending individual JSON events
curl \
-H "Authorization: Bearer <DATASOURCE:APPEND token>" \
-d '{"date": "2020-04-05 00:05:38", "city": "Chicago"}' \
'https://api.tinybird.co/v0/events?name=events_test'

The name parameter defines the name of the Data Source in which to insert events. If the Data Source does not exist, Tinybird will create the Data Source by inferring the schema of the JSON.

The Auth Token used to send data to the Events API must have the appropriate scopes. To append data to an existing Data Source, the DATASOURCE:APPEND scope is required. If the Data Source does not already exist, the DATASOURCE:CREATE scope is required to create the new Data Source.

Sending batches of JSON events

You can send batches of JSON events to the Events API by formatting the events as NDJSON (newline delimited JSON). Each individual JSON event should be separated by a newline (\n) character.

Sending batches of events enables you to achieve much higher total throughput than sending individual events.

Sending batches of JSON events
curl \
-H "Authorization: Bearer <import_token>" \
-d $'{"date": "2020-04-05 00:05:38", "city": "Chicago"}\n{"date": "2020-04-05 00:07:22", "city": "Madrid"}\n' \
'https://api.tinybird.co/v0/events?name=events_test'

The name parameter defines the name of the Data Source in which to insert events. If the Data Source does not exist, Tinybird will create the Data Source by inferring the schema of the JSON.

The Auth Token used to send data to the Events API must have the appropriate scopes. To append data to an existing Data Source, the DATASOURCE:APPEND scope is required. If the Data Source does not already exist, the DATASOURCE:CREATE scope is required to create the new Data Source.

Limits

The Events API will deliver an out of the box capacity of:

  • Up to 1000 requests/second

  • Up to 20MB/s

Throughput above these limits is offered as best-effort.

The Events API is able to scale well beyond these limits. If you are reaching these limits, please contact support@tinybird.co.

Compression

NJDSON events sent to the Events API can be compressed with Gzip. However, it is only recommended to do this when necessary, such as when you have big events that are grouped into large batches. Compressing events adds overhead to the ingestion process, which can introduce latency, although it is typically minimal.

Here is an example of sending a JSON event compressed with Gzip from the command line:

echo '{"timestamp":"2022-10-27T11:43:02.099Z","transaction_id":"8d1e1533-6071-4b10-9cda-b8429c1c7a67","name":"Bobby Drake","email":"bobby.drake@pressure.io","age":42,"passport_number":3847665,"flight_from":"Barcelona","flight_to":"London","extra_bags":1,"flight_class":"economy","priority_boarding":false,"meal_choice":"vegetarian","seat_number":"15D","airline":"Red Balloon"}' | gzip > body.gz

curl \
    -X POST 'https://api.tinybird.co/v0/events?name=gzip_events_example' \
    -H "Authorization: Bearer <AUTH_TOKEN>." \
    -H "Content-Encoding: gzip" \
    --data-binary @body.gz

Write acknowledgements

When you send data to the Events API, you will usually recieve a HTTP202 response, which indicates that the request was successful, however it does not confirm that the data has been committed into the underlying database. This is useful when guarantees on writes are not strictly necessary. Typically, it should take under 2 seconds to recieve a response from the Events API in this case.

curl \
    -X POST 'https://api.tinybird.co/v0/events?name=events_example' \
    -H "Authorization: Bearer <AUTH_TOKEN>" \
    -d $'{"timestamp":"2022-10-27T11:43:02.099Z"}'

< HTTP/2 202
< content-type: application/json
< content-length: 42
<
{"successful_rows":2,"quarantined_rows":0}

However, if your use cases requires absolute gauarantees that data is committed, you can use the wait parameter.

The wait parameter is a boolean that accepts a value of true or false. A value of false is the default behaviour, equivalent to ommitting the parameter entirely.

Using wait=true with your request will ask the Events API to wait for acknowledgement that the data you sent has been committed into the underlying database. You will recieve a HTTP200 response that confirms data has been committed.

Note that adding wait=true to your request can result in a slower response time, and we recommend having a time-out of at least 10 seconds when waiting for the response.

For example:

curl \
    -X POST 'https://api.tinybird.co/v0/events?name=events_example&wait=true' \
    -H "Authorization: Bearer <AUTH_TOKEN>" \
    -d $'{"timestamp":"2022-10-27T11:43:02.099Z"}'

< HTTP/2 200
< content-type: application/json
< content-length: 42
<
{"successful_rows":2,"quarantined_rows":0}

It is good practice to log your requests to, and responses from, the Events API. This will help to give you visibility into any failures for reporting or recovery.