Deprecation of Kafka non-binary headers¶

To improve how we process Kafka headers, we are deprecating the option of storing them as a JSON string field. This change will take effect from September 1st 2024.

We recently changed how we process Kafka headers: since April 23rd 2024 we treat them as a binary map, instead of assuming it can be stored as a JSON (which was occasionally error-prone behavior). This new change provides you with a more stable Tinybird platform.

How to prepare for the deprecation¶

You don't have to do anything. We will remind you in advance, and upgrade your Data Source from the Tinybird side.

If you are importing Kafka headers in any of your Data Sources from before April 23rd 2024, you are already getting them as a JSON string in the __headers field. Don't worry - we will move any existing headers to the __old_headers field. Then Tinybird will start importing the new headers in __headers as a Map(String,String). As a reminder, String is the ClickHouse type for storing binary data.