Privacy-preserving identity resolution

Privacy-preserving identity resolution is how two teams answer "how many users overlap?" without emailing each other CSVs of emails and device IDs. Regulators treat deterministic joins on raw PII as data sharing. Security teams treat them as breach multipliers. Product teams still need matched audiences for attribution, churn analysis, and partner reporting.

The workable pattern hashes identifiers with agreed rules, resolves in controlled environments, and exports only aggregates that pass suppression thresholds. Raw keys never cross the trust boundary twice.

What "privacy-preserving" actually requires

Privacy-preserving resolution is not "we deleted the column in the export." It is a chain of constraints:

Deterministic matching on SHA-256(email + partner_salt) is still personal data in many jurisdictions if the partner can reverse or relink. Treat hashed tokens as sensitive. Store salts in HSM-backed secret managers, not repo env files.

Deterministic vs probabilistic matching

Deterministic links use exact tokens: hashed email, phone, loyalty ID, billing account key. Confidence is binary. False merges happen when households share inboxes or typos normalize wrong.

Probistic links use signals: IP + UA + timestamp windows, device graphs, behavioral similarity. Confidence is a score. False positives rise when you optimize for reach.

Store both link types with metadata:

CREATE TABLE identity_links (
    internal_user_id UUID,
    partner_id LowCardinality(String),
    partner_token FixedString(64),
    link_type Enum('deterministic', 'probabilistic'),
    confidence Float32,
    linked_at DateTime,
    source_rule LowCardinality(String)
)
ENGINE = ReplacingMergeTree(linked_at)
ORDER BY (partner_id, partner_token);

Downstream SQL filters confidence >= 0.9 for activation and link_type = 'deterministic' for financial reporting. Mixing both without labels is how compliance reviews fail.

Hashing rules partners must agree on before any join

1. Normalize before hash. Lowercase email, strip plus-tags per policy, E.164 phone formatting.
2. Per-partner or per-clean-room salt. Never reuse consumer-app salts in partner rooms.
3. Document algorithm and version. v1_sha256_email_salt_a vs v2_sha256_email_salt_b must not silently coexist.
4. Rotate on breach or employee departure affecting salt access, not on every nightly job.

ClickHouse® integration Segment notes identity graphs want merge events, not wide trait columns on every fact row. Privacy-preserving resolution follows the same separation: links live in a governed table, events stay keyed by internal ID.

k-anonymity and aggregate-only exports

Many partnerships allow aggregate measurement, not row-level overlap lists. Enforce k-anonymity at query time:

SELECT
    campaign_id,
    count() AS matched_users
FROM activation_overlap
GROUP BY campaign_id
HAVING matched_users >= 100

Blocks with fewer than k users (often 50–250 depending on contract) return nothing, not rounded noise alone. Differential privacy adds calibrated noise for public statistics. Most ad-tech partnerships still use k-thresholds plus legal review, not full DP libraries.

Analyst temptation: keep filtering until a segment "works." Product and SQL endpoints should enforce minimum counts server-side so BI tools cannot bypass suppression.

Clean rooms vs in-house resolution

Data clean rooms (Snowflake, AWS, Google, Habu-class vendors) run joins inside vendor-controlled compute with policy templates. In-house resolution runs on your ClickHouse cluster with IAM and row policies you own.

Privacy-preserving identity resolution can happen in either environment. The mistake is running the same raw email join in Snowflake and calling it a clean room because the worksheet title says "restricted."

ClickHouse patterns after tokens exist

Once internal IDs and partner tokens map in a governed table, analytics behaves like any other funnel:

SELECT
    toDate(event_time) AS day,
    uniqExact(internal_user_id) AS converted_users
FROM product_events e
INNER JOIN identity_links l
    ON e.internal_user_id = l.internal_user_id
WHERE l.partner_id = 'retail_partner'
  AND event_type = 'purchase'
  AND event_time >= today() - 14
GROUP BY day
ORDER BY day;

Columnar storage keeps join costs manageable when fact tables are billions of rows and link tables are millions. Rollups pre-aggregate partner overlap counts so activation dashboards never scan full histories.

ClickHouse fast queries applies: PREWHERE on date, narrow SELECT lists, avoid SELECT * on wide event schemas when building match pipelines.

Tinybird with governed identity endpoints

Tinybird sits after your resolution service materializes links. It does not replace legal review or clean room contracts.

Ingest link tables and events on separate datasources

Keep identity_links ingestion restricted. Product events flow through Kafka or the Events API with only internal_user_id, never partner tokens in client-visible logs.

Publish partner metrics with fixed_params

NODE partner_overlap_counts
SQL >
    SELECT
        campaign_id,
        count() AS matched_users
    FROM partner_overlap_daily
    WHERE partner_id = {{ String(partner_id, required=True) }}
    GROUP BY campaign_id
    HAVING matched_users >= {{ Int32(min_k, 100) }}

TYPE endpoint

JWT fixed_params bind partner_id and min_k so URL tampering cannot widen or de-suppress results. Multi-tenant SaaS options describes the same pattern for tenant isolation in product analytics.

Branches for rule changes

When hashing moves from v1 to v2, replay links on a Tinybird Branch before prod cutover. Service Data Sources catch ingest gaps that would undercount overlap and trigger false campaign optimism.

Resend reports 62 ms p90 at 100 TB/month scale. Tinybird is SOC 2 Type II certified when partner metrics leave batch systems for operational dashboards.

Red flags in privacy-preserving programs

Salts in application config repos. Anyone with repo access can rebuild identifiers.

Shared match tables without partner column. One table joins all partners; row policies slip.

Probabilistic links for billing or medical adjacency. High-confidence deterministic rules only.

Exports as CSV email attachments. Clean room exists; workflow bypasses it.

Analyst notebooks with raw tokens. Governance stops at the warehouse login.

No unlink/delete propagation. User erasure requests leave partner tokens joinable forever.

What to implement first

Agree normalization and hashing v1 with legal. Materialize links with type and confidence. Enforce k-thresholds in SQL endpoints, not spreadsheet honor codes. Only then scale partner count.

Privacy-preserving identity resolution is measurable joins on tokens plus enforced aggregate floors. Speed comes from columnar rollups after those rules exist, not from skipping them.

Frequently Asked Questions (FAQs)

Is hashed email always GDPR-safe?

No. Hashed identifiers can still be personal data if relinking is practical. Treat salts and tokens as sensitive.

What k threshold do partners use?

Common ranges are 50–250 for overlap reports. Contracts specify exact floors.

Can ML improve probabilistic matching?

Yes, with labeled training data and human review. Do not promote probabilistic scores to financial use cases without governance.

Do clean rooms eliminate legal review?

No. They provide controlled compute and audit. Contracts still define allowed outputs.

Should identity links live in the warehouse or OLAP?

Governed link tables often sit beside high-volume events in ClickHouse for fast joins. Warehouses hold audit copies and batch exports.

How does Tinybird help?

It publishes suppressed aggregate endpoints with scoped tokens after your resolution pipeline materializes links.