Skip to main content

Privacy & Data Model

:::caution Pre-release This is a pre-1.0 library (v0.1.1) — API may change without notice. :::

Privacy is built into the data model, not added afterward. Four invariants are enforced by both client-side normalization and server-side re-verification.

Privacy Invariants​

1. Route Patterns Only (No Raw URLs)​

Invariant: URLs are stored as route_pattern only — never a full URL, never query strings, never hash fragments.

Implementation:

  • Client-side: packages/journey-recorder/src/shared/route-pattern.ts exports normalizeRoutePattern(). This function strips query and hash before any other processing.

  • Server-side: packages/journey-recorder/src/server/pii.ts exports enforceRoutePattern(), which re-runs the shared normalizer defensively. If the client sent a URL with a query string or hash that somehow survived the client's normalizer, the server rejects it. See the docstring: "Belt and braces. normalizeRoutePattern strips query/hash before anything else, so this should be unreachable — which is exactly why it is checked."

Why: Query strings often contain sensitive data (email addresses, access tokens, session IDs). Hash fragments may contain navigation state intended to be private. By normalizing away both before storage, the collector never persists that data.

Example:

Input:  /checkout?email=user@example.com&utm_source=email#billing
Output: /checkout

2. No Raw IP Storage​

Invariant: Raw IP is never stored anywhere. Bot detection and per-IP rate limiting use a hashed IP only.

Implementation:

export async function hashIp(ip: string, salt: string, now: number = Date.now()): Promise<string> {
  const day = utcDay(now);
  return (await sha256Hex(`${salt}:${day}:${ip}`)).slice(0, 32);
}

  • Rotation: The salt is provided by the operator via NS_IP_SALT environment variable. The day component (UTC YYYY-MM-DD) makes rotation implicit even if the operator forgets to rotate the secret itself. After 24 hours, the hash for the same IP under the same salt is no longer correlatable across days.

  • Use: The hashed IP is used only for per-IP rate limiting (key for distributed rate limit counters) and bot heuristics. It is never written to raw_event or any user-facing table.

Why: Storing raw IPs enables tracking and re-identification. A daily-rotating salt bounds how long an IP can be re-identified or linked to a single user across sessions.

3. Scrubbed Click Target Labels​

Invariant: Click target labels (button text, input placeholders, etc.) are scrubbed of sensitive data before storage.

Implementation:

Why: Button and input labels might inadvertently contain PII. Redacting password fields, email fields, and credit card fields prevents accidental leakage.

Example:

// Client sees and redacts:
scrubLabel("password");       // → null (excluded)
scrubLabel("Email Address");  // → "Email Address" (safe)
scrubLabel("Email: user@x.com");  // → "Email: [REDACTED]"

4. Partition-Drop Retention​

Invariant: Raw event retention is enforced by partition drop, never row-level DELETE.

Implementation:

  • Partitioning: The raw_event table is partitioned by day. After N days (default 30, operator-configurable), the partition for that day is dropped in full.

  • GDPR Compliance: To honor an erasure request, only the raw_event and session_summary tables need to be touched. Aggregated tables (summary, rollup, etc.) carry no user identifiers, so GDPR scope is bounded.

  • Enforcement: This is asserted by a CI schema-audit test, not by application code.

Why: Partition drop is a fast, atomic operation that avoids expensive row-level scans and deletes. It ensures deterministic retention without requiring background job infrastructure.

Defense in Depth​

Every privacy invariant is enforced twice:

  1. Client-side: The SDK normalizes data before sending it. This is the first line of defense.
  2. Server-side: The collector re-runs the same pure functions defensively. The server does not trust the client.

As noted in packages/journey-recorder/src/server/pii.ts:

The client already normalizes route patterns and scrubs labels — and this module runs the SAME pure functions from ../shared again anyway. That is the point: the client is not a trust boundary. An out-of-date SDK, a tampered bundle, or a hand-rolled POST to the collect endpoint can all send a raw /checkout?email=a@b.com#access_token=xyz, and nothing downstream of here re-checks.

See Events & Data Model — Consent for how to gate data collection behind user consent.

Audit and Verification​

The privacy invariants are asserted by tests in packages/journey-recorder/:

Check the test files for concrete examples of what passes and what is rejected.

Secrets Management​

Secrets (NS_TOKEN_SECRET, NS_IP_SALT) are passed via environment variables and never logged or exposed:

  • Keep secrets in .env.local (development) or your Vercel / platform secrets manager (production).
  • Rotate NS_TOKEN_SECRET by setting NS_TOKEN_SECRET_PREVIOUS during the rotation window.
  • Rotate NS_IP_SALT immediately if compromised; future hashes will not correlate to past hashes.

See Installation & Environment Variables for the complete setup guide.