Druid vs Athena: Segments or Files

The same query is valid in Apache Druid and Amazon Athena:

SELECT
  floor(__time TO MINUTE) AS minute,
  country,
  COUNT(*) AS events,
  SUM(revenue) AS revenue
FROM page_views
WHERE __time >= CURRENT_TIMESTAMP - INTERVAL '1' HOUR
GROUP BY 1, 2

In Druid that is a scatter/gather over already-indexed segments, often tens to hundreds of milliseconds if you rolled up at ingest. In Athena that is a planner + S3 list + scan over whatever Parquet you wrote last. AWS still prices that scan at $5 per TB on the on-demand SQL path. Their own example: 3 TB uncompressed text, one query, $15.

Calling them alternatives is how architecture decks go wrong. One is a real-time OLAP database. The other is a serverless lake engine. This post follows 1 query through both systems: ingest, layout, execution, money, people. Then 2 variants that break the happy path.

If the client of that query is an app, not a Broker or a workgroup, neither path publishes it as an HTTP API. Tinybird is the third path for that case: same events, ClickHouse® SQL, a pipe with a token. The peer comparisons are ClickHouse vs Druid and ClickHouse vs Amazon Athena. This page is the Druid-versus-lake argument those 2 posts do not make.

2 data paths for the same events

Events
  |- Kafka --> Druid supervisor --> realtime task --> segment handoff --> historicals
  |- Firehose --> S3 (buffer 0-900s) --> Glue/Iceberg --> Athena workgroup
  |- Kafka or HTTP --> Tinybird datasource --> SQL pipe --> HTTP endpoint

Druid pulls from Kafka or Kinesis with a supervisor spec. MiddleManagers run index tasks. Realtime tasks answer queries on in-flight data, then handoff immutable segments to Historicals. Deep storage (S3) holds the segments. A metadata DB (MySQL/Postgres) records the cluster. ZooKeeper or equivalent coordinates.

A minimal Kafka supervisor looks like this. The numbers are the ones that decide freshness and segment size:

{
  "type": "kafka",
  "dataSchema": {
    "dataSource": "page_views",
    "timestampSpec": { "column": "ts", "format": "iso" },
    "dimensionsSpec": {
      "dimensions": ["country", "device", "path"]
    },
    "metricsSpec": [
      { "type": "count", "name": "events" },
      { "type": "doubleSum", "name": "revenue", "fieldName": "revenue" }
    },
    "granularitySpec": {
      "segmentGranularity": "HOUR",
      "queryGranularity": "MINUTE",
      "rollup": true
    }
  },
  "ioConfig": {
    "topic": "page_views",
    "consumerProperties": {
      "bootstrap.servers": "kafka:9092"
    },
    "taskCount": 4,
    "replicas": 1,
    "taskDuration": "PT4H"
  },
  "tuningConfig": {
    "maxRowsPerSegment": 5000000,
    "maxTotalRows": 20000000,
    "intermediateHandoffPeriod": "PT1H"
  }
}

A task writes events into a segment for the current hour until it hits maxRowsPerSegment, maxTotalRows, or intermediateHandoffPeriod. Then it publishes to deep storage and waits for a Historical to load. Default handoffConditionTimeout on Kafka is 900 seconds. If Historicals are full, ingest looks "fine" in Kafka lag and queries look "empty" in the UI. Those are different alerts.

taskCount: 4 means at least 4 segment chains per interval. Events that share a dimension set but arrived on different Kafka partitions do not collapse into one rolled-up row until compaction. Netflix-style telemetry writeups cite 10-100× storage reduction when the dimension set is closed and compaction has run. Before compaction you have more segments, worse caching, and rollup that looks broken in COUNT(*).

Athena never sees Kafka. Amazon Data Firehose buffers by size (1-128 MiB) and interval (0-900 seconds, default 300). Then it writes objects, optionally straight into Iceberg. You register the table in Glue. Athena's engine (Trino-lineage under the current SQL engine) reads those objects when someone runs a query. Nothing is hot unless you reserved capacity. AWS's published reservation example is $0.30 per DPU-hour.

If the event is still in the topic and not in an object, Athena cannot answer. If the Druid supervisor is lagging, Druid answers stale, which you can alert on. Different failure modes.

What a segment is, what a file is

Druid segment. Columnar, time-chunked, dictionary-encoded, with bitmap indexes for filters on dimensions. Optional ingest rollup collapses rows that share the same timestamp granularity plus dimensions. Target size after compaction is the folklore range of 500-700 MB per segment. Smaller than that and Brokers scatter too many pieces. Larger and Historicals take longer to load and fail louder.

The cost of ingest rollup: you cannot later group by a column you dropped at ingest. Raw-or-rollup is a spec decision, not a query hint. Changing device from a dimension to a metric, or adding campaign 6 months later, is a reindex. Lookups exist for dimension enrichment. They are another moving part, not a warehouse JOIN.

Late events write new segments that Historicals must load. Compaction supervisors rewrite. This is why Druid clusters have Overlords, not just "a database."

Athena file. Whatever you put in S3: text, JSON, ORC, Iceberg, Parquet. Athena's own pricing example is the lesson:

• Uncompressed 3 TB text, one-column query: scan the whole file, $15
• Gzip ~3:1: $5
• Parquet + 1 of 4 columns: $1.25

Layout is the index. Hive-style dt=2026-09-23/hr=08/ partitions are how you avoid full-prefix lists. 128 MB-class Parquet row groups are the folklore that Firehose talks and AWS talks keep repeating because small files kill both time and money. A 5-second Firehose buffer at moderate ingest produces a pile of tiny objects. You then pay Glue, S3 LIST, and Athena to discover that you should have buffered 60-300 seconds or compacted overnight.

Partition projection tells Athena to calculate partition values from table properties instead of calling Glue GetPartitions for every hour prefix. That cuts planning time on highly partitioned tables. It does not cut bytes scanned. Iceberg helps snapshots, MERGE, deletes, and hidden partitioning. OPTIMIZE is how you compact. Iceberg still does not give you Druid's in-memory dimension bitmaps.

CREATE EXTERNAL TABLE page_views (
  ts timestamp,
  country string,
  device string,
  path string,
  revenue double
)
PARTITIONED BY (dt string, hr string)
STORED AS PARQUET
LOCATION 's3://lake/page_views/'
TBLPROPERTIES (
  'projection.enabled' = 'true',
  'projection.dt.type' = 'date',
  'projection.dt.range' = '2026-01-01,NOW',
  'projection.dt.format' = 'yyyy-MM-dd',
  'projection.hr.type' = 'integer',
  'projection.hr.range' = '0,23',
  'projection.hr.digits' = '2',
  'storage.location.template' = 's3://lake/page_views/dt=${dt}/hr=${hr}/'
);

Without the dt/hr predicates, Athena lists and scans more than the last hour. The SQL looks the same. The bill does not.

Execution: scatter/gather vs ephemeral workers

Druid Brokers parse SQL (Calcite). They scatter to Historicals (and realtime tasks) that already have the segment columns mapped. Filters use bitmaps. Aggregations run where the data sits. Joins exist and are still the part of Druid SQL people work around. High concurrency is the design center: many dashboards, same rollup, predictable p95.

Athena starts from a shared or reserved pool. Planning, splits, S3 GETs, and shuffle happen per query. Result reuse helps when the string matches inside the cache window. A dashboard that parameterizes tenant_id does not hit that cache. During regional busy hours the shared pool adds queueing you will not see in a Sunday POC. On-demand queries also have a 10 MB minimum billed per query, which does not matter at 8 GB and does matter at "I hit it every 5 seconds for a 2 MB result."

2 variants that break the happy path

High-cardinality breakdown. Add user_id to the GROUP BY. In Druid, if user_id was not a dimension at ingest, the query is impossible without a raw datasource. If it was a dimension, ingest rollup barely rolls up, segment count explodes, and Historical heap follows. In Athena the query runs. It scans whatever columns you left in Parquet. It will be slow and expensive, but it will not refuse. That is why product teams who cannot name every GROUP BY column at spec time bounce off classic Druid.

Join to a 1M-row dimension. Druid lookups or a broadcast-ish join, depending on version and spec. People who arrived from warehouses are surprised. Athena can join the lake to a Glue table or a federated JDBC source. Federated queries add Lambda charges on top of TB scanned. Fine for "enrich this once." Miserable as the inner loop of an app.

Neither system gives you per-request API tokens and branch deploys. Both assume a human or a BI tool is the client.

People: 6 Druid processes vs a catalog

A production Druid cluster is not one StatefulSet.

Plus metadata SQL, deep storage, and ZK. Imply exists because this list is a team. Apache Druid alternatives is mostly people looking at that table.

Athena's people cost is file hygiene: partition projection, compaction of tiny Firehose objects, Glue permissions, workgroup scan limits, Iceberg OPTIMIZE cadence. Fewer daemons. More conversations with whoever owns the bucket layout. The older AWS path (Kinesis to S3 to Glue to Athena) is spelled out in comparing Tinybird to Kinesis, S3, Glue, and Athena. Firehose's 60-900 s buffer is the freshness tax that post and the ClickHouse-vs-Athena post both keep repeating because teams forget it in POCs.

3 cost scenarios for the same tile

Facts that stay fixed:

• After compaction, the last hour is 8 GB of Parquet (Athena) or 1.2 GB of rolled-up Druid segments
• 12 hours/day, 22 days/month
• On-demand Athena: $5/TB scanned
• Assume the Athena query is well partitioned and reads 8 GB each time (the hour prefix only)

Internal workbook, once an hour. Athena: 8 GB × 12 × 22 = 2,112 GB ≈ 2.1 TB/month ≈ $10.50. Druid: you still paid for the cluster all month. Athena wins by a lot. Stop building a cluster.

Analyst dashboard, every 5 minutes. Athena: 8 GB × 12 refreshes/hour × 12 × 22 = 25,344 GB ≈ 25 TB/month ≈ $124. Still cheap if the query stays partitioned. Druid is optional. A materialized view or a nightly rollup in the lake may be enough.

Customer tile, every 15 seconds. Athena: 8 GB × (3600/15) × 12 × 22 = 8 GB × 240 × 12 × 22 = 506,880 GB ≈ 495 TB/month. × $5 = about $2,475/month for one tile. 10 tiles with poor overlap is a 5-figure scan line before S3 GET charges. This is why Athena dashboards that "just work" in a demo get killed by FinOps.

Provisioned capacity changes the last formula to DPU-hours. AWS's own example: 160 DPU × $0.30 × 0.25 h = $12 to cover a 15-minute peak. That can beat on-demand if you staff the reservation. You are now managing capacity, which was the thing Athena was supposed to avoid.

Druid on that 15-second tile: you pay instances (or Imply). A compact serving cluster (brokers + historicals + ingest) that holds hot segments in cache might be a few thousand a month in cloud VMs, plus the humans. The marginal cost of the 15-second refresh is ~0 once segments are local. That is the whole economic argument for Druid, and for any serving OLAP.

If you refresh once an hour for an internal workbook, Athena wins. If you refresh every 15 seconds for customers, Druid's fixed cost is the cheaper sentence to say to finance. If you also need arbitrary SQL and HTTP tokens, neither sentence is complete.

SQL you will fight

Druid. Time column is __time. Ingest specs declare dimensions vs metrics. Changing a dimension is a reindex. The native JSON query API is what older clusters actually run. Calcite SQL has improved and still surprises people who arrived from warehouses. Subqueries and joins are the sharp edge.

Athena. Standard-ish Trino SQL. Geospatial, Iceberg MERGE, CTAS into another bucket, time travel (FOR SYSTEM_TIME AS OF). Federated connectors add Lambda. Great for "join the lake to a JDBC source once." There is no first-class "this SELECT is an endpoint with a tenant token."

Decision tree

Need the answer on events that are not in S3 yet?
  yes -> Druid (or another streaming OLAP). Athena is ineligible.
  no  -> Are queries a few hundred/day, seconds OK?
           yes -> Athena. Stop building a cluster.
           no  -> Is the dimension set closed and the query always time+filter+group?
                    yes -> Druid earns its processes.
                    no  -> Tinybird (or another general columnar SQL engine). Not Druid rollup-at-ingest.

Closed dimension set means you can name every GROUP BY column at spec time. User-facing "break down by any JSON key" is a ClickHouse or warehouse job, not classic Druid rollup.

Tinybird for the same GROUP BY as an API

The decision tree above already exits to Tinybird when the dimension set is open and the client is an app. That is the same GROUP BY as the Druid and Athena SQL at the top of this post. Druid still assumes a Broker and a dashboard. Athena still assumes a workgroup and a human. Tinybird takes the Kafka or HTTP events from the third branch of the diagram and publishes the query as a pipe.

Ingest maps to the Druid supervisor without the process graph. The Kafka connector reads the topic; offsets and retries are the connector's job, not an Overlord plus MiddleManagers. If the producer is an app, the Events API accepts JSON over HTTP at 1K+ events/sec. There is no Firehose 0-900 s buffer. A row is queryable in seconds. Late events are just more inserts into MergeTree. toStartOfMinute(ts) uses event time, so an 8-minute retry still lands in the minute it happened, the same correctness rule Druid's __time gives you when the timestamp spec is right.

Layout is a sort key, not a segment spec. ORDER BY (country, ts) (or tenant_id, country, ts if the tile is multi-tenant) is the sparse index. Adding campaign 6 months later is a column plus a deploy, not a reindex of every historical segment. Ingest rollup is optional: a materialized pipe with countState / sumState if the cube is closed, or raw MergeTree if product will group by a JSON key that did not exist at spec time. That last case is where classic Druid refuses and Athena merely becomes expensive.

The query as an endpoint, with the same grain as the Druid/Athena SQL:

DESCRIPTION >
    Last-hour page views by minute and country. Product tile.

NODE hourly_tile
SQL >
    SELECT
        toStartOfMinute(ts) AS minute,
        country,
        count() AS events,
        sum(revenue) AS revenue
    FROM page_views
    WHERE ts >= now() - INTERVAL 1 HOUR
    GROUP BY minute, country
    ORDER BY minute, country

TYPE ENDPOINT

Auth is a resource token on that pipe. Preview branches give the PR its own data (npx tinybird preview). Production is npx tinybird deploy. The app never talks to a Broker or an Athena workgroup. Resend publishes product analytics at 62 ms p90 on this serving shape. The 15-second customer tile that costs ~$2,475/month on Athena on-demand is a mid-tier Developer plan plus vCPU overage ($0.0002/vCPU-second above baseline) once the rollup exists, because the meter is compute on a small aggregated table, not $5/TB rescanned every refresh.

What you do not get: Druid's closed-cube bitmap indexes tuned by an OLAP team, or Athena's ability to join the entire lake to a JDBC source once. Tinybird is the wrong buy for a 1-hour internal workbook on files you refuse to index, and the wrong buy if a dedicated Druid CoE already runs supervisors and the dimension set is frozen. It is the right buy when the cube keeps changing, freshness must be seconds, and the consumer is an app.

The older AWS path this replaces (Kinesis to S3 to Glue to Athena) is in comparing Tinybird to Kinesis, S3, Glue, and Athena.

Pick by freshness, refresh rate, and the client

Use Athena for the lake already in the account: audits, one-off joins, data that will not be indexed. Use Druid when a dedicated OLAP team already knows supervisors and the query set is a closed cube. Use Tinybird when the cube keeps changing and the consumer is an app.

If a single cell in a comparison matrix is required: Druid = indexed stream. Athena = priced scan. Tinybird = SQL API. Those are not 3 brands of the same thing.

Frequently Asked Questions (FAQs)

Are Apache Druid and Amazon Athena alternatives?

No. Druid indexes a stream into segments and answers from local data. Athena scans objects in S3 at $5/TB. If the event is still in Kafka and not in an object, Athena cannot answer.

When is Athena cheaper than Druid?

An internal workbook that refreshes once an hour on an 8 GB partitioned prefix is about $10.50/month on-demand. A customer tile that refreshes every 15 seconds on that same prefix is about $2,475/month per tile. Druid's fixed cluster cost wins at high refresh rates. Athena wins at low query volume.

Where does Tinybird fit?

When the requirement is Kafka or HTTP ingest, arbitrary ClickHouse SQL, and an HTTP endpoint with a token, without Coordinators, Overlords, or Historicals. Monthly plan plus vCPU overage. Druid remains correct for a closed cube run by an OLAP team. Athena remains correct for the lake.