Ingest from OpenTelemetry¶
OpenTelemetry is an open-source observability framework for collecting, processing, and exporting telemetry data (metrics, traces, and logs) from your applications and infrastructure.
By integrating OpenTelemetry with Tinybird, you can analyze observability data in real time, build dashboards, and enrich it with other data sources.
Some common use cases for sending OpenTelemetry data to Tinybird include:
- Centralizing metrics, traces, and logs for unified analytics.
- Building custom dashboards and alerts on top of observability data.
- Enriching telemetry with business or application data.
Read on to learn how to send data from OpenTelemetry to Tinybird.
Before you start¶
Before you connect OpenTelemetry to Tinybird, ensure:
- You have a Tinybird workspace.
- You have a Tinybird Token with append permissions to the target Data Sources.
- You are running the Tinybird distribution of the OpenTelemetry Collector.
You can find the latest release of the Tinybird OpenTelemetry Collector here:
Use the Tinybird OpenTelemetry project template¶
To get started quickly, you can use the Tinybird OpenTelemetry project template. This template provides ready-to-use Data Sources and Pipes for storing and analyzing your telemetry data in Tinybird.
Recommended OpenTelemetry Collector configuration¶
Below is an example configuration for the Tinybird OpenTelemetry Collector to export metrics, traces, and logs to Tinybird:
receivers: otlp: protocols: grpc: http: processors: batch: timeout: 10s send_batch_size: 8192 exporters: tinybird: endpoint: ${OTEL_TINYBIRD_API_HOST} # Your Events API endpoint, e.g. https://api.us-east.aws.tinybird.co token: ${OTEL_TINYBIRD_TOKEN} # Token with append permissions sending_queue: enabled: true queue_size: 104857600 # Total memory buffer in bytes (100 MB) sizer: bytes batch: flush_timeout: 5s # Max wait time before flushing min_size: 1024000 # Min batch size: 1 MB max_size: 8388608 # Max batch size: 8 MB (Events API limit is 10 MB) retry_on_failure: enabled: true metrics_sum: datasource: opentelemetry_metrics_sum metrics_histogram: datasource: opentelemetry_metrics_histogram metrics_exponential_histogram: datasource: opentelemetry_metrics_exponential_histogram metrics_gauge: datasource: opentelemetry_metrics_gauge traces_datasource: otel_traces logs_datasource: otel_logs service: pipelines: metrics: receivers: [otlp] processors: [batch] exporters: [tinybird] traces: receivers: [otlp] processors: [batch] exporters: [tinybird] logs: receivers: [otlp] processors: [batch] exporters: [tinybird]
Environment variables¶
OTEL_TINYBIRD_API_HOST
: The API host for your Tinybird workspace (e.g.,https://api.tinybird.co
).OTEL_TINYBIRD_TOKEN
: A Tinybird token with append permissions to the target Data Sources (otel_metrics
,otel_traces
,otel_logs
).
You can create a token in the Tinybird UI under Tokens. Make sure it has the required append permissions for the Data Sources you want to ingest into.
Run the Collector¶
Using the binary¶
Precompiled binaries for Linux and macOS are available for both amd64
and arm64
architectures.
Download them from the GitHub Releases page.
./otelcol_contrib_linux_amd64 --config config.yaml
Using Docker¶
docker run --rm -v config.yaml:/etc/otelcol/config.yaml tinybirdco/opentelemetry-collector-contrib:v0.128.0
Next steps¶
- Explore and customize the Tinybird OpenTelemetry project template to fit your needs.
- Use the ingested data to build real-time analytics, dashboards, and alerts in Tinybird.
For more details on the available configuration options, see the Tinybird OpenTelemetry Collector documentation.