# Analytics, Retention Metrics, and Decision Framework

Fan Passport should be measured as a live engagement product with collection, prediction, social, and reward loops. The analytics program must help operators understand whether fans are forming a daily habit, where they drop out, and which loops create durable retention after the World Cup.

The machine-readable event taxonomy is in `data/analytics/events.json`.

## North-star metric: Meaningful Passport Days

A **Meaningful Passport Day** is a local calendar day on which a user completes at least one meaningful Fan Passport action.

Qualifying actions:

- Claims a daily stamp.
- Completes a challenge.
- Answers a trivia question.
- Submits a prediction.
- Returns to view a resolved prediction.
- Completes a match check-in.
- Collects a sticker or completes a collection set.
- Creates a match memory.
- Joins or contributes to a squad.
- Redeems or claims a reward.
- Generates the tournament recap.

The north-star metric is:

> **Weekly Meaningful Passport Days per Active User**

During the World Cup, the daily operating view should also track:

> **Daily Meaningful Passport Users** and **Meaningful Passport Day rate among DAU**.

## Lifecycle funnel

| Funnel stage | User action | Primary events |
| --- | --- | --- |
| Acquire | Arrives from organic, referral, sponsor, search, or social share. | `app_opened`, `referral_attributed`, `share_completed`. |
| Onboard | Creates account, selects country or team, grants desired permissions. | `account_created`, `profile_completed`, `team_followed`, `notification_permission_updated`. |
| Activate | Completes first meaningful passport action. | `daily_stamp_claimed`, `trivia_answer_submitted`, `challenge_completed`. |
| Engage | Returns for match, challenge, prediction, collection, or memory. | `prediction_submitted`, `match_check_in_completed`, `sticker_collected`, `match_memory_created`. |
| Socialize | Joins squad, shares card, refers friends, trades stickers. | `squad_joined`, `share_completed`, `referral_attributed`, `sticker_trade_completed`. |
| Compete | Views and improves leaderboard rank. | `leaderboard_viewed`, `challenge_completed`, `prediction_resolved`. |
| Reward | Views, earns, redeems, or clicks sponsor offer. | `reward_viewed`, `reward_redeemed`, `sponsor_card_viewed`, `offer_clicked`. |
| Retain | Completes meaningful actions across multiple days and phases. | Repeated qualifying events by local date. |
| Transition | Opens recap and selects next passport. | `passport_viewed`, `achievement_unlocked`, `share_completed`. |

## Core KPIs

### Activation

| Metric | Definition | Operating target |
| --- | --- | ---: |
| Account creation conversion | New accounts divided by acquisition sessions. | 35% or higher for referral traffic; 20% or higher for cold traffic. |
| Profile completion | New accounts that follow at least one team or country. | 70% or higher. |
| First meaningful action | New accounts completing a qualifying action within first session. | 65% or higher. |
| Push permission opt-in | Mobile users granting push after value prompt. | 45-60% depending on platform and region. |
| First share generation | New users generating a share card within first 48 hours. | 8-12%. |

### Daily engagement

| Metric | Definition | Operating target |
| --- | --- | ---: |
| DAU | Unique users opening app or web on a local day. | Contextual to acquisition spend and match calendar. |
| Meaningful Passport Day rate | DAU with at least one qualifying action. | 55% or higher during group stage, 65% or higher during knockouts. |
| Daily stamp claim rate | DAU claiming daily stamp. | 45-60%. |
| Daily trivia answer rate | DAU submitting daily trivia. | 25-40%. |
| Primary challenge completion rate | Users starting primary challenge who complete it. | 45-70%, depending on difficulty. |
| Prediction submission rate | Eligible match viewers submitting prediction before lock. | 30-50%. |
| Result return rate | Users with prediction/check-in returning after result resolution. | 35-55%. |
| Sticker pack open rate | Eligible users opening daily pack. | 30-50%. |
| Second session rate | DAU with two or more sessions in a day. | 25-40% on matchdays. |

### Retention

| Metric | Definition | Operating target |
| --- | --- | ---: |
| D1 retention | New users returning the day after signup. | 35-45%. |
| D3 retention | New users active three days after signup. | 25-35%. |
| D7 retention | New users active seven days after signup. | 18-28%. |
| Tournament phase retention | Active in one phase and active again in next phase. | 45% from group to knockout among activated users. |
| Final-day return | Users active in first week who return on final day. | 15-25%. |
| Post-final recap open | Eligible users opening recap. | 40-60%. |
| Next-passport interest capture | Recap openers selecting Premier League, Champions League, or rewards interest. | 25-40%. |

### Social and viral

| Metric | Definition | Operating target |
| --- | --- | ---: |
| Share generation rate | Users generating share card divided by DAU. | 8-15% on peak days. |
| Share completion rate | Completed external shares divided by generated cards. | 50-70%. |
| Referral signup conversion | Referred visitors who create account. | 20-35%. |
| Referral activation conversion | Referred signups completing meaningful action. | 50-70%. |
| Viral coefficient | Activated referred users per active inviter. | 0.08-0.20 during normal days; higher around final recap. |
| Squad join rate | Activated users joining or creating squad. | 15-30%. |
| Squad retention lift | D7 retention of squad users compared with non-squad users. | Positive lift of 20% or higher. |

### Rewards and monetization

| Metric | Definition | Operating target |
| --- | --- | ---: |
| Sponsor card viewability | Sponsor card views divided by eligible impressions. | 70% or higher for in-feed placements. |
| Sponsored challenge completion | Completions divided by starts. | 40-65%. |
| Offer click-through | Offer clicks divided by reward or sponsor views. | 3-10% depending on offer. |
| Reward redemption rate | Redemptions divided by issued rewards. | 25-60% depending on reward type. |
| Reward support contact rate | Reward support tickets divided by redemptions. | Under 3%. |
| Premium cosmetic conversion | Purchases divided by eligible active users. | 1-4% if launched. |
| Revenue per meaningful passport user | Net revenue divided by users with meaningful action. | Track by channel and sponsor package. |

### Safety and quality

| Metric | Definition | Operating target |
| --- | --- | ---: |
| Report rate | User reports per 1,000 active users. | Contextual; spikes require review. |
| Confirmed violation rate | Actioned reports divided by reviewed reports. | Used for queue quality, not a standalone target. |
| Median report review time | Time from report to moderator decision. | Under target in policy. |
| Leaderboard fraud hold rate | Users or points held divided by competitive participants. | Low but non-zero. |
| Reward fraud rejection rate | Rejected reward claims divided by claims reviewed. | Monitor by campaign. |
| Content correction count | Live content corrections per day. | Downward trend; zero for sponsor legal issues. |
| Notification opt-out spike | Change in opt-out rate after campaigns. | No material spike after send. |

## Dashboard suite

### Executive live dashboard

Refresh: hourly during tournament.

- DAU.
- Meaningful Passport Users.
- Meaningful Passport Day rate.
- New accounts.
- D1 and D7 retention cohorts.
- Challenge completion.
- Prediction submissions.
- Share completions.
- Referral activations.
- Reward redemptions.
- Revenue or sponsor campaign delivery.
- Top incident or safety alert.

### Live-ops dashboard

Refresh: every 15 minutes on matchdays.

- Today's content status by slot.
- Daily stamp claims by hour.
- Challenge starts and completions by challenge ID.
- Prediction submissions by fixture and lock status.
- Match check-ins by fixture.
- Result resolution success and failures.
- Collection drop opens.
- Notification sends, opens, dismisses.
- Leaderboard top movers.
- Support tickets by category.
- Moderation queue size and severity.

### Growth dashboard

Refresh: daily.

- Acquisition source quality.
- Share card generation and completion by card type.
- Referral link creation and activation.
- Notification opt-in, open, dismiss, opt-out.
- Segment response by user lifecycle.
- Experiment assignments and primary outcomes.
- Dormant reactivation campaigns.
- Post-final next-passport preference capture.

### Collection and economy dashboard

Refresh: daily.

- Sticker pack opens.
- Stickers collected by rarity and set.
- Duplicate rate.
- Trade requests and completions.
- Set completion rate by group, stadium, team, and tournament phase.
- XP distribution.
- Badge unlock distribution.
- Reward inventory remaining.
- Reward redemption and fulfilment status.

### Trust and safety dashboard

Refresh: hourly during high-risk fixtures, daily otherwise.

- Reports by reason and surface.
- Enforcement actions by level.
- Median review time.
- Leaderboard anomaly flags.
- Reward fraud holds.
- Suspicious referral clusters.
- Content correction incidents.
- Appeal volume and reversal rate.

## Event quality rules

- Every event must include a unique `event_id`.
- Every event must include pseudonymous `user_id` when tied to a user.
- Every event must include `occurred_at` in UTC.
- Client events should also include `client_timestamp` and server receipt time where available.
- Do not log raw email, phone number, precise GPS, government IDs, payment card data, or private reward codes.
- Use stable IDs for teams, fixtures, stadiums, challenges, rewards, badges, and campaigns.
- Treat locale, timezone, app version, platform, and acquisition source as standard properties.
- Use idempotency keys for score-changing and reward-changing events.
- Server-side events are authoritative for scoring, prediction locks, rewards, purchases, and moderation.

## Sample cohort logic

The following example uses a table named `analytics.events` with columns matching the taxonomy. Teams can map this logic to their actual warehouse.

### Meaningful passport users by day

```sql
WITH meaningful_events AS (
  SELECT
    user_id,
    DATE(occurred_at, user_timezone) AS activity_date
  FROM analytics.events
  WHERE event_name IN (
    'daily_stamp_claimed',
    'challenge_completed',
    'trivia_answer_submitted',
    'prediction_submitted',
    'prediction_resolved',
    'match_check_in_completed',
    'sticker_collected',
    'collection_completed',
    'match_memory_created',
    'squad_joined',
    'reward_redeemed'
  )
)
SELECT
  activity_date,
  COUNT(DISTINCT user_id) AS meaningful_passport_users
FROM meaningful_events
GROUP BY activity_date
ORDER BY activity_date;
```

### D1 retention from account creation

```sql
WITH signups AS (
  SELECT
    user_id,
    DATE(MIN(occurred_at), user_timezone) AS signup_date
  FROM analytics.events
  WHERE event_name = 'account_created'
  GROUP BY user_id, user_timezone
),
activity AS (
  SELECT DISTINCT
    user_id,
    DATE(occurred_at, user_timezone) AS activity_date
  FROM analytics.events
  WHERE event_name IN (
    'app_opened',
    'daily_stamp_claimed',
    'challenge_completed',
    'trivia_answer_submitted',
    'prediction_submitted',
    'sticker_collected'
  )
)
SELECT
  signup_date,
  COUNT(*) AS new_users,
  COUNTIF(activity.user_id IS NOT NULL) AS retained_d1,
  SAFE_DIVIDE(COUNTIF(activity.user_id IS NOT NULL), COUNT(*)) AS d1_retention
FROM signups
LEFT JOIN activity
  ON activity.user_id = signups.user_id
  AND activity.activity_date = DATE_ADD(signups.signup_date, INTERVAL 1 DAY)
GROUP BY signup_date
ORDER BY signup_date;
```

### Referral activation rate

```sql
WITH referred_signups AS (
  SELECT
    user_id,
    MIN(occurred_at) AS referred_at
  FROM analytics.events
  WHERE event_name = 'referral_attributed'
  GROUP BY user_id
),
meaningful_after_referral AS (
  SELECT DISTINCT
    e.user_id
  FROM analytics.events e
  JOIN referred_signups r
    ON r.user_id = e.user_id
  WHERE e.occurred_at >= r.referred_at
    AND e.event_name IN (
      'daily_stamp_claimed',
      'challenge_completed',
      'trivia_answer_submitted',
      'prediction_submitted',
      'sticker_collected'
    )
)
SELECT
  COUNT(*) AS referred_users,
  COUNT(meaningful_after_referral.user_id) AS activated_referred_users,
  SAFE_DIVIDE(COUNT(meaningful_after_referral.user_id), COUNT(*)) AS referral_activation_rate
FROM referred_signups
LEFT JOIN meaningful_after_referral
  ON meaningful_after_referral.user_id = referred_signups.user_id;
```

## Alert thresholds

| Alert | Trigger | Response |
| --- | --- | --- |
| Daily content not visible | Daily feed view count is near zero 15 minutes after scheduled publish. | Check scheduler, CMS state, app config, and rollback if needed. |
| Prediction lock failure | Submissions accepted after lock or rejected before lock. | Pause fixture prediction, audit timestamps, correct affected users. |
| Result resolution failure | More than 1% of eligible predictions fail to resolve within 60 minutes. | Re-run result job or switch to manual resolution. |
| Notification opt-out spike | Opt-out rate doubles normal daily baseline after a campaign. | Pause similar campaign and review copy/frequency. |
| Reward inventory anomaly | Redemptions exceed expected pace by 25% or inventory mismatch appears. | Pause reward, run fraud and fulfilment audit. |
| Referral abuse spike | Referral activation cluster exceeds velocity or device thresholds. | Hold referral rewards and review cluster. |
| Leaderboard anomaly | Top user or squad score delta exceeds possible daily max. | Freeze affected ranks and audit score events. |
| Moderation surge | Severe reports exceed staffing capacity for 30 minutes. | Disable high-risk surfaces or increase staffing. |
| Share rendering failure | Share generation error rate exceeds 5%. | Disable affected card layout and fall back to generic card. |
| Recap generation backlog | Recap generation delay exceeds 30 minutes after final result verification. | Queue priority fix and communicate expected availability. |

## Experiment decision rules

- Define one primary metric and one guardrail before launch.
- Use random assignment when possible; use phased rollout only when randomization is impractical.
- Avoid experiments that change reward fairness during competitive phases.
- Do not run copy experiments on legal terms, age gates, or sponsor disclosures.
- Run experiments long enough to cover different matchday types when possible.
- Stop experiments early only for clear harm, technical failure, or overwhelming result.
- Record decision, metric movement, sample size, and next action in the daily close note.

## Retention interpretation by phase

| Phase | Healthy behavior | Warning sign | Operator response |
| --- | --- | --- | --- |
| Opening week | High onboarding and first meaningful action. | Many users create account but do not follow a team. | Move team selection earlier and simplify first challenge. |
| Early group stage | Daily stamp and trivia establish habit. | DAU exists but meaningful action rate is low. | Make daily challenge easier and improve feed hierarchy. |
| Mid group stage | Prediction and collection loops grow. | Notification opens decline sharply. | Personalize by followed team and reduce generic pushes. |
| Group climax | Qualification drama drives returns. | Users are confused by group completion rules. | Add explainer cards and checklist UI. |
| Knockouts | Prediction stakes and social sharing rise. | Leaderboard trust complaints increase. | Run integrity audits and communicate rules. |
| Rest days | Catch-up and collections prevent drop-off. | Dormant users ignore generic reactivation. | Use specific missing set or recap prompt. |
| Final week | Recap anticipation and final stamp drive activity. | Users complete final stamp but ignore next passport. | Make transition CTA part of recap, not a separate ad. |
| Post-final | Recap shares and next-passport preference capture. | Users view recap once and disappear. | Offer founder badge, club pledge, and weekly football ritual. |

## Privacy and governance

- Use pseudonymous analytics IDs.
- Store raw personal identifiers outside behavioral analytics where possible.
- Aggregate sponsor reporting.
- Honor deletion and access requests by removing or anonymizing analytics records according to policy.
- Limit data retention for high-risk safety and fraud signals.
- Do not enrich football fandom data with sensitive third-party profiles without explicit consent and legal basis.
- Review analytics taxonomy before launching Premier League or Champions League Passport so competition-specific fields do not leak inappropriate assumptions into other seasons.