Chapter 1 — The problem

Everyone wants the pulse of our users. Getting it takes weeks.

Every team is curious what users think — about Fetch, about a product, a feature, an experience. Traditionally we run a survey to get that pulse. But surveys cost money, take weeks, and are rarely ready the moment we need them. And it's cumbersome to ask every user why they rated the way they did.

Can we do better with LLMs? Get the pulse in minutes — and generate the reasoning behind each rating, so we have the qualitative why alongside the quantitative what. Both, at once, on demand.

By the time results land, the roadmap moved.

how it works today
1 · Draft surveydays
2 · Recruit paneldays
3 · Field2–3 wks
4 · Analyzedays
5 · Decide
Chapter 2 — The idea

What if Fetch's user base could answer in minutes — as software?

For this proof we sampled real archetype-survey respondents and turned each into a persona. An LLM (GPT- or Claude-class) answers as that persona.

Built from the archetype survey: real behavior, real attitudes, their own words.

Chapter 3 — The proof · Track 1

We didn't trust it either. So we made it re-take our own survey.

scorecard ·
9 items — real vs synthetic μ, Δ, KS (click to collapse)
Itemreal μsynth μΔKS

μ is the mean rating (1–5). Δ is synthetic minus real. KS = distribution overlap: 1 − the biggest gap between the two cumulative curves, so 1.00 = identical shapes, higher is better.

item explorer
Chapter 4 — The fun part · Track 2

Now point it at something that doesn't exist yet.

Same panel, new stimulus.

So — what do our users think about this feature?

Chapter 5 — What this unlocks

A user-research pre-screen for every team at Fetch.

Offer teams

Rank candidate offers before committing partner spend.

Product

Screen Discover / Nexus concepts before EPPO experiments — re-order the queue, don't replace it.

UXR

Pilot questionnaires on synthetic panels first; field the sharp version.

Brand pitches

Same-day "how would your buyers react" appetizers.

traditional → synthetic · per run
Chapter 6 — What we're honest about

What we're honest about.

Model phrasing repeats (deterministic model, n=1)
we display diversity-selected verbatims; production would sample multiple generations
Absolute numbers unproven on new stimuli
use relatively, rank concepts
Panel over-represents engaged users, unweighted
production reweights
Novel paradigms (LiDAR-style) are lower-confidence
lean on verbatims
Near-misses cluster at scale extremes
anchor tuning is the next lever
We can't verify Track 2 yet.
next steps
  • Per-archetype validation
  • Anchor calibration
  • Reweighting the panel to the real user base
  • EPPO-outcome backtesting