Roadmap
First, the boundary that governs everything on this page: every validation result we hold comes from synthetic personas driven by a language model. No human being has been measured, and no phase below changes that until the real-user study actually runs.
Second, how this page relates to what it used to say. Two earlier generations of the programme are recorded in the repository's history: a first-generation scoring chain that never ran in production and was decommissioned on 2026-08-04, and a second generation — an earlier corpus engine over an earlier persona library — whose corpora were deprecated on 2026-09-02 once the engine, the persona generator and the persona instruction had all changed. This page describes the third: the current engine, its one registered run, and what follows from it.
Delivered — the current engine and its first registered run
Status: built, run, banked.
- The persona generator, rebuilt (v5). A 3,280-persona library sampled
from the same copula architecture as before, with a rebuilt persona
instruction: every construct named (the earlier sheet named only the top
two moral foundations and one attachment word), the trait ladder read in a
randomised order per persona and stamped on every row, and every band's
wording placed by blind readers at the level it claims — fourteen of the
fifteen upper emotional-intelligence bands place exactly, where the
earlier sheet read a top-quartile persona as an average one
(
calibration/phaseA/wp22/–wp24/). - The wave engine, made resilient. Persona-driver calls run in bulk; a
vendor quota is a wait, not a death; the run resumes from checkpoints at
no re-spend; a stalled batch job is retried rather than fatal; and, after
the confirm run, each chain link's hand-off state is banked so a resume
can no longer re-derive it wrong (
functions/corpus_wave/). - The confirm run (1–3 September 2026): 40 v5 personas, 50 planned sessions each in chains of 20, one driver, $123.16. Registered before it ran; analysed once. The twelve historically unreadable families express above chance (pooled +0.126 against a bar of 0.082, one in 240 by chance), sitting 0.23 below the five classic families. In the served configuration the engine's own per-trait estimate reaches a median recovery of 0.23 against planted truth, ten of 27 families are individually confirmed, one — liberty/oppression — is confirmed backwards and three more read negative (all four withheld from the product), and per-person consistency at fifty sessions is a median of 0.15 — one family meets the 0.70 decision-grade bar. The validation page carries every number with its artifact.
- The served number, repaired (3 October 2026). Replaying the confirm
corpus through the production code showed that the product had been
pooling sessions in a way that counted the prior once per session: every
served score sat within a tenth of a point of average, and no trait ever
became confident enough to show a number. The served path now adds up each
session's evidence, applies the prior once, and calibrates each family's
interval to its own measured consistency. On the same corpus, served
scores now spread as an honest score should, and nine families become
consistent enough to show a number for most personas by about 49 sessions;
ranking was never affected (
calibration/phaseA/wp28/). Status, 5 October: live, together with a stricter rule: a trait shows a number only once it is shown to be both consistent and accurate, not consistent alone. On the current evidence two families pass (honesty–humility and psychopathy); the rest show their range. Every stored session was re-scored the same day.
Delivered — more information per session (October 2026)
Status: live. Each item is described on the methodology page and measured, with its limits, on the validation page.
- Seven broad areas (5 October). The profile opens with seven areas that pool their member traits' evidence; four pass the accuracy rule on the current evidence, two of them tagged "early reading".
- Options that keep their meaning (5 October). The model that words an option is told the act it is scored as. In a blind reading, the share of options still expressing it rose from 46 of 96 to 86 of 96.
- Options that stay inside the story (10 October). The model that words an option is told to leave open anything the scene has not shown, and a check re-writes a menu it catches inventing. In a blind reading of 96 options per version, options stating a story fact the scene never showed fell from 13 to 6; options still expressing their act stayed at 95–97%.
- The redesigned session (9 October). A fourth scene, an ending that finishes each act, the never-take moved into the second scene (scored since 10 October at a provisional weight), and Afterwards. With ideal players it roughly doubles the information per session (×2.12). That is a ceiling; for people it is unmeasured.
- Typed answers as options of their own (9 October). They are read into the catalogue the options are written from. They are scored at reduced weight by an explicit decision for the closed test, because the reading missed its pre-registered bar.
In progress — fixing what the run exposed
Status: underway, September–October 2026.
- Re-verifying the twelve weakest families against their scoring keys. Four families read backwards and eight read flat. Flipping a sign because a correlation is negative is a refuted lever in this programme — it looked like a large gain in-sample and made the engine worse held out — so the sanctioned method is semantic: independent readers compare each construct's language with the actual shelf actions that score positive and negative on its axis and rule aligned or inverted, with a fix only where two of three agree and only as the smallest text change. This is the method that found and fixed two resilience inversions in August. Some families will come back "aligned — the model simply will not play it", and those get no fix. Until re-verified, the backwards families are withheld from the product.
- A small paired run — ran on 3 September (24 of the confirm personas,
8 sessions each, $9.18). It settled the scene-writing question: the clause
that carries an option's scoring now survives in half of the
emotional-awareness options, from a tenth. It withdrew one staging cue
that had starved its own item. It was too small to answer whether the
touched families' signs moved, so the four withheld families stay
withheld (
calibration/phaseA/wp27/paired_result_v1.json). - Targeted rewrites for options read as the wrong trait. A
key-independent check found behaviour credited to a neighbouring trait's
options, replicating 14 of 14 on the current engine: openness read on
cognitive-flexibility options, extraversion on narcissism options,
conscientiousness and agreeableness on care and loyalty options,
machiavellianism on impulse-control options (
cross_key_v1.json). Fix: make each option discriminate between the two traits it is being confused with, then test the registered prediction on the next corpus. - Neuroticism's early-versus-late reversal. Its consistency is negative under every split, −0.82 between a persona's first and second half of sessions. Until the mechanism is found, no neuroticism number is shown.
- The fifty-session milestone. Completed sessions per persona reached a median of 46 on the confirm run, not 50, because 100 sessions died of replay drift across the run's eight restarts. That defect is fixed and tested; the next chained run should reach the milestone without further change.
- A better reading of typed answers. The decoder agrees with blind readers 0.64–0.68 on what a typed act expresses. The readers agree with each other only 0.72, so the registered bar of 0.80 may be unreachable as written. The follow-up work is twofold: re-measure on the model that decodes in production, and decide — before looking at any recovery figure — whether the bar should be re-stated against the readers' own agreement. Real typed answers, read blind, are what would settle it.
- The never-take's and the mark's weights. Both are scored since 10 October at weights chosen in advance (half a pick; a small self-report row). Fitting them on real answers, and checking them on players they were not fitted to, is the open work.
- A persona test that can answer. The first language-model test of the new session was too small, and its drivers followed their planted traits too weakly, to show any difference. An answer needs many more sessions per persona, and ideally drivers that follow their traits more closely.
Next — breadth, then people
A breadth run. The confirm run was sized to answer one pooled question, and it did. What it cannot do is certify each of 27 traits on its own: the per-family intervals are about ±0.27 wide at 40 personas and did not narrow between 27 and 50 sessions, because their width is set by the number of people, not sessions. Roughly 170 personas would pin each family to ±0.15; about 380 to ±0.10. That run is the next spend, once the re-verification above has landed.
Per-person consistency. The honest gap the confirm run measured is
between reading a population (median recovery 0.23; ten confirmed families)
and reading an individual steadily (median split-half 0.15). Every served
reading carries its own interval and a plain confidence word for exactly
this reason. Since August the stable score no longer discounts older
sessions, so playing longer does keep adding evidence. But at today's
information per session the median family would need about 570 sessions to
reach 0.70 — reaching it within fifty needs about 11 times more information
per session (served_calibration_v1.json). Closing the gap is information
work. The October session is the first large step: about twice the
information per session for ideal players. The next cheapest step is the
evidence currently credited to the wrong trait (item 3 above). Neither
closes an eleven-fold gap on its own, and only real players can show how
much of either survives.
Real-user validation. Status: planning; consent-blocked. Item-level rows from real players need a consent scope the current catalog does not carry; sessions are retained long enough that a later scope can still cover them. When it runs: recalibration against real-user behaviour with an explicit consent flow, measurement-invariance testing across age × sex × education × geography, and a real cohort distribution to replace the uniform 18–70 age prior. Earliest expected start: late 2026; realistic: 2027. It is the only step that can turn any synthetic result into a claim about people.
A first, smaller study is written down and pre-registered
(calibration/phaseA/wp32/real_player_proof/PLAN.md). It asks whether a
session carries more real information, using a measure that does not trust
our own model: how well a profile built from a player's other sessions
predicts their choices in a session left out. It was written before the
October redesign, so its arms will be re-fixed for the new session before
any data exists. It has not started: it needs a research consent scope and
a budget decision.
What's NOT on the roadmap
- No clinical assessment claims, ever. The 29 traits are normal-range psychological tendencies, not psychiatric diagnoses. The system is not a screening instrument; we do not plan to build one.
- No cultural conditioning yet. Cross-cultural personality variation is real (Schmitt 2008 documents it across 55 cultures). Adding cultural axes adds 5–10 dimensions to the conditioning structure; deferred until the real-user cohort shows which subgroups matter for the SoulMap user base.
- No per-user model. Calibration is global. The stack produces per-user estimates with uncertainty without per-user parameter fitting.
- No personalisation pipelines. SoulMap is a measurement instrument, not a recommender system.
- No real-time online learning. Any calibration that changes served
parameters must pass pre-registered gates first. The choice model itself
is still the deliberate uncalibrated baseline. One served calibration
exists, stated plainly because it was not pre-registered: since 3 October
each family's interval and score scale are set from its consistency
measured on the latest banked depth run. It was introduced to repair a
defect, is regenerated by a committed script, and is checked against three
level gates that go red on the old behaviour (
calibration/phaseA/wp28/). It never learns from a live user. - No dimensional expansion beyond 29 in the foreseeable future.
- No demographic features as inputs to the scoring engine. The demographic-blind scoring policy (personas page). Hard line.
- No sign-flipping by statistics. Refuted, with a reopening condition that only an independent channel — human responses, or a second driver family — can meet.
What would falsify the plan
- Run-level: a run whose attrition exceeds the registered 15% bar is void against its own registration. The confirm run passed at 8.45%.
- Test-level: a registered threshold is a threshold. The confirm run's pooled bar (0.0822) was fixed before the data existed and the analysis was run once; a miss would have been published as a miss, as two earlier registered tests were.
- Instrument-level: a family that reads negative on two independent corpora is a blocking finding until traced — the programme's pole-inversion contract — and four families are in that state now.
- Transfer-level: no synthetic result, however strong, counts as evidence about human beings. The falsifier for the whole programme is the first real-user study.
The next page, citations, is the alphabetical bibliography that backs every footnote on every page in this tier.