Activation KPI — PostHog

Activation KPI — PostHog

Deployment status

The existing dashboard is in PostHog EU. The configuration below is the
required operator change: create and save its new headline insight, replace the
existing headline tile, and add the annotation. Until that work is completed in
PostHog, the external dashboard is not yet the thread-level satisfaction
measure described here.

Dashboard configuration

Required PostHog operator actions:

  1. Create and save the headline HogQL insight specified below.
  2. Replace the dashboard’s current headline tile with that saved insight.
  3. Add the exact dashboard annotation specified below at the release boundary.
  4. Reconfigure or replace both existing secondary insights to the exact
    sequences specified below before presenting them as diagnostics.
  • Dashboard name: Activation KPI
  • Headline insight: Prompt → generation → audio delivery started per thread
  • Supporting insights: Anonymous prompt → OTP → generation → download and
    OTP → signed-in prompt → generation → download (secondary acquisition and
    identity diagnostics only)
  • Headline insight type: saved HogQL insight; use a native strict-order funnel
    only if its property aggregation can faithfully deduplicate by thread_id
  • Headline aggregation: completed threads, deduplicated by event property
    thread_id — never unique persons
  • Date range: last 30 days
  • Conversion window: 7 days
  • Conversion reference: total completion from the first step, retaining
    per-step conversion/drop-off where supported

The headline thread-completion flow is:

  1. prompt.submitted
  2. generation.started
  3. audio.delivery_started

Every event must have the same non-empty thread_id. The strict sequence must
complete within seven days of prompt.submitted. slug identifies the take;
thread_id identifies the root conversation that owns that take. A person may
have several threads, and a thread may have several take slugs, so neither
person nor slug is the headline deduplication key.

Use this as the fallback saved HogQL insight query until native funnel property
aggregation has been verified to preserve the same strict per-thread_id
behavior. It anchors candidate prompt.submitted events to the last 30 days
and returns one row containing the number of completed threads:

WITH eligible_events AS (
    SELECT
        properties.thread_id AS thread_id,
        timestamp,
        event
    FROM events
    WHERE timestamp >= now() - INTERVAL 30 DAY
      AND event IN ('prompt.submitted', 'generation.started', 'audio.delivery_started')
      AND properties.thread_id IS NOT NULL
      AND properties.thread_id != ''
      AND (
          person.properties.email IS NULL
          OR (
              person.properties.email NOT IN ('usebrioso@gmail.com', 'carloshpiedra@gmail.com')
              AND person.properties.email NOT LIKE '%@brioso.test'
              AND person.properties.email NOT LIKE '%@usebrioso.com'
          )
      )
),
prompt_candidates AS (
    SELECT thread_id, timestamp AS prompt_at
    FROM eligible_events
    WHERE event = 'prompt.submitted'
),
candidate_generations AS (
    SELECT p.thread_id, p.prompt_at, min(e.timestamp) AS generation_at
    FROM prompt_candidates AS p
    INNER JOIN eligible_events AS e ON e.thread_id = p.thread_id
    WHERE e.event = 'generation.started'
      AND e.timestamp > p.prompt_at
    GROUP BY p.thread_id, p.prompt_at
),
completed_candidates AS (
    SELECT g.thread_id, g.prompt_at, min(e.timestamp) AS delivery_at
    FROM candidate_generations AS g
    INNER JOIN eligible_events AS e ON e.thread_id = g.thread_id
    WHERE e.event = 'audio.delivery_started'
      AND e.timestamp > g.generation_at
      AND e.timestamp <= g.prompt_at + INTERVAL 7 DAY
    GROUP BY g.thread_id, g.prompt_at
)
SELECT count(DISTINCT thread_id) AS completed_threads
FROM completed_candidates

prompt_candidates evaluates every eligible prompt rather than only the first
prompt in a thread. candidate_generations and completed_candidates preserve
strict > ordering for each candidate, while the final count(DISTINCT thread_id) deduplicates a thread that has any valid sequence. This supports
later takes without counting a completed thread more than once. Required
annotation text: “Thread completion data begins with the thread-level delivery
release (2026-08-14); count strict prompt → generation → audio delivery
completion once per thread_id within seven days. Earlier download-based
activation data is not comparable.”

The exact secondary person-level sequences are:

  1. Anonymous acquisition/identity diagnostic:
    prompt.otp_sent → otp.verified → generation.started → download.
    The existing anonymous saved insight requires operator reconfiguration or
    replacement before it is treated as this exact sequence.
  2. Signed-in acquisition/identity diagnostic:
    otp.verified → prompt.submitted → generation.started → download.
    The existing signed-in saved insight requires operator reconfiguration or
    replacement before it is treated as this exact sequence.

These are separate because their entry events are alternative journeys, not
sequential steps.

Exclude internal/test people whose email is usebrioso@gmail.com or matches
either @brioso.test or @usebrioso.com; retain the dashboard exclusion for
carloshpiedra@gmail.com.

Event contract

prompt.submitted is emitted after durable job insertion and before dispatch,
so a fast worker claim cannot place generation.started before the prompt in a
strict-order funnel. It has slug, runner, the resolved user ID, and must
carry the root conversation thread_id for headline measurement.

generation.started is emitted once from a successful JobsQueue.claimForRunner
claim for both home and runpod. It has slug, runner, traceId, the
resolved userId when the job email matches a user, and must carry thread_id.

audio.delivery_started is the delivery-start contract used by the headline.
It must carry thread_id and the take slug, so the full sequence can be
strictly matched to one conversation thread. download is a retired,
browser-originated event; do not combine it with audio.delivery_started in a
headline funnel or completion count. It remains useful only for the explicitly
labelled browser/acquisition diagnostics.

prompt.otp_sent is emitted server-side only after a new anonymous job has
been created and its OTP issued. It uses the same user ID as otp.verified, so
browser tracking blockers cannot break the person funnel. It is forward-looking
from this release. prompt.otp_required remains a best-effort browser-only
UI-friction diagnostic and is intentionally not a required funnel step.

Signed-in path

OTP → signed-in prompt → generation → download measures users who authenticate
before submitting a prompt. Together with the anonymous path, it remains a
person-level, browser/acquisition and identity diagnostic: strict order, unique
persons, seven-day window, 30-day range, and internal-account exclusions. It is
not the headline satisfaction metric and must not be compared directly with
thread completion.

Previous 2026-07-17 — VPS state management: migration vs reconciliation research Next Agentic engineering workflow assessment