Methodology
Everything on this page was measured on synthetic personas driven by a language model. No human being has been measured. That is the programme's design boundary, stated first because every other claim depends on it. We built an instrument that can be scored against known ground truth — synthetic personas whose generating trait values we hold exactly — so that its failures would be measurable before any human study. Where a limit exists, this page reports it with the same precision as the successes.
One rule governs the writing: every number names the artifact it comes
from. A number without a named artifact does not appear here. Unless a
section says otherwise, the numbers on this page come from one run — the
pre-registered confirm run of September 2026
(calibration/phaseA/wp26/confirm_result_v1.json). The October 2026
sections name their own artifacts. Earlier
corpora, generated by an earlier engine over an earlier persona library,
were deprecated on 2026-09-02 and are not cited.
One estimator, served and measured
SoulMap has one measurement path, and an earlier version of this page had to explain why the served path and the measured path differed. Since 2026-08-27 the estimator that produces a user's trait scores is the estimator the calibration programme measures.
Until 2026-10-03 the two still differed in one place: how sessions were
added together. Each session was scored on its own and the results were
then pooled in a way that applied the standard-normal prior once per
session instead of once per person. The ranking of people was untouched —
which is why no correlation on this site moved — but every served score
sat within a tenth of a point of the population mean, and no trait ever
reached the point where a number is shown. The served path now adds up each
session's evidence and applies the prior once; it reproduces the measured
estimator's ranking at a median agreement of 0.99
(calibration/phaseA/wp28/).
Status: live since 5 October 2026, together with a stricter rule for showing a number (consistent and accurate, not consistent alone). Every stored session was re-scored with the repaired code the same day.
It runs in a deliberate uncalibrated configuration: every family's
discrimination is pinned to 1 and no nuisance terms are fitted, because no
calibration manifest is mounted in production. That is a measured choice,
not a placeholder. On the confirm corpus the served configuration recovers
planted traits at a median correlation of 0.233 across 27 families; the
same estimator with fitted discriminations and nuisance terms from an
earlier research fit reaches 0.234 — the same number to within noise
(confirm_result_v1.json, recovery_SERVED_CONFIG and
recovery_DESCRIPTIVE). Fitted parameters buy nothing yet, so none are
served. Everything below describes the instrument in the configuration a
user actually gets.
The shelf: the server writes the psychometrics, the model writes the prose
The psychometric core of a session is not the language model. It is a
versioned evidence shelf of 366 archetype entries — content hash
7bda5d9bf355 since the wording revision of 3 September 2026, stamped on
every session record and every banked corpus row (the confirm corpus below
was banked under its predecessor, cfbd7786eb4e; the revision re-worded
four option texts and changed no trait loading. A staging cue added to two
further entries in the same revision was withdrawn the same day, after a
registered re-measurement found it cut how often that pair was offered by
about three quarters). Each entry carries authored measurement metadata: signed trait
loadings per family, an intent tag, a difficulty rating and a polarity;
registered opposite-pole pairs carry a partner.
At each turn the server selects the four answer options by a deterministic, seeded search over the shelf under hard constraints — distinct primary constructs per menu, a cap per registry family on action turns, an opposite-pole gate for the registered pairs — with soft rules that relax down a fixed ladder rather than fail the turn. The language model then writes the scene and the wording of the four buttons; it never assigns psychometrics. After rendering, the server re-attaches each button's trait loadings, intent tag and difficulty from the table. Trait identity on every button is correct by construction, never model-authored.
The model writes the prose. The server writes the psychometrics.Two honest consequences of this design:
- The design matrix is the shelf's own declaration of what each button means. A recovery figure is therefore a joint test of the driver and the label registry — not of the driver alone. Where a family reads backwards, the first suspect is the language of its options against its scoring key, and the programme's contract treats a repeated negative as blocking until that has been traced by reading, not by statistics.
- Delivery counts, not shelf counts. On the confirm corpus, 312 of the 366 entries were offered at least once; the 54 never offered are almost all reflection-turn entries. Every one of the 27 families was genuinely contrasted — offered with differing loadings on the same menu — on at least 294 of 3,822 rows, and on as many as 1,908 (attachment avoidance).
The session as played
Redesigned 9 October 2026; the design is
docs/design/session-flow-v2-2026-10-09.md. A session is a short story in
four scenes, about five minutes long. Each answer a player gives is one of a
few kinds, and each kind has a fixed rule for whether and how it is scored:
| beat | what the player does | how it is read |
|---|---|---|
| Scenes 1–3 (action) | picks one of four options, or types their own | one choice row over the four dealt options; a typed answer is its own alternative (below) |
| The ending, after each tapped action | finishes their own sentence with one of three short endings — a gesture, where their attention goes, or a passing thought — or with "Full stop." | one choice row over the three tell signals the server dealt; "Full stop." adds nothing |
| Scene 2, after the choice | crosses out the remaining option they would never take | scored since 10 October, at half the weight of a pick (provisional) |
| Scene 4 (reflection) | picks one of four reflective answers about what happened | a self-report row, kept in its own channel |
| Afterwards | picks one of three next steps for their character, or "Let it end here" | one choice row over three dealt hooks |
| Looking back | may mark the moment that was most like them | a small self-report row since 10 October (provisional) |
The endings follow the rule the options follow: the server deals the signal, the model writes the words. For each action scene the server deals three tell signals from a 58-entry catalogue, always from three different trait families, each with authored loadings. The story model writes, for every option, one ending per signal in the scene's own voice, and the server re-attaches the signal after rendering. The chosen ending is part of the act: the next scene is written from the act and how it ended, under a reach rule — a passing thought stays private, a gesture can be seen by others in the scene, and a glance never moves the player toward what they looked at. Afterwards works the same way, from a catalogue of eight hooks: one is aimed at each of the two values families the action scenes reach least, and the third family is drawn per session.
A tell row is scored in the channel its three endings mostly belong to:
mostly passing thoughts, the self-report channel; otherwise, the behavioural
channel. The rule was fixed before the simulation that compared it with two
alternatives, and it read best of the three
(calibration/phaseA/wp33/session_flow_v2_sim/NOTES.md).
Two rules here are enforced by the build, not only by policy
(graph/contracts.yaml):
- The never-take is scored at a provisional weight
(
refusal_recorded_never_scored, amended 10 October). Research on best–worst choice finds that "worst" picks usually carry less information than "best" picks, so it counts at half the weight of a pick. That weight is chosen in advance, not fitted: every score records which weight produced it, and the weight is refitted on real answers and checked on players it was not fitted to. - Afterwards never shapes the next session
(
afterwards_never_seeds_next_session). It closes this story; the next story is not told what was picked.
Options say what they measure
Since 5 October 2026. The model that words each option receives a short
statement of the act the option is scored as, and must keep it. In a blind
reading of 24 real menus, the share of options that still expressed their
scored behaviour rose from 46 of 96 to 86 of 96; a model-based meaning
check on the same kind of menus rose from 52% to 76%
(calibration/phaseA/wp31/label_core_ab/). The blind reading had one
reader, a language model. The fix matters for accuracy as well as
information: an option that has lost its meaning still gets chosen, and the
choice is credited to the wrong behaviour.
Typed answers
Every action scene offers "Speak freely": the player may type what they do instead of tapping. Since 9 October a typed answer is treated as its own option, not matched to one of the four on screen. A language model (the one that writes the story: Anthropic's Claude since 28 October 2026, Gemini 3.8 Flash before) maps the typed act onto the same server-held catalogue the options are written from — the reverse of writing an option — and maps any small sign in the words (a glance, a tone, a passing thought) onto the tell catalogue. A typed answer is offered no ending; the small sign, if there is one, is read from the words themselves.
The typed act is scored as a fifth alternative beside the four on screen,
its loadings scaled to the menu's average option strength (a declared rule,
not a fit). The sign, if there is one, becomes a fourth alternative beside
the three tell signals that menu dealt. Both rows are weighted down —
0.55 for the act, 0.12 for the sign — because the reading is a model's
interpretation of a sentence and it has not met its pre-registered
validation bar. Scoring it anyway is an operator decision for the closed
test, recorded in the contract that guards it
(decoded_free_text_scored_only_after_validation), and one deploy setting
turns it off. Neither weight was tuned on a corpus: each is the lower bound
of measured agreement between the decoder and blind readers. The
validation page carries the numbers.
Safety does not depend on that reading. Typed words are also checked for signs of crisis by a separate, previously validated call that runs in parallel; the more severe of the two verdicts wins, and a turn flagged for crisis or concern is never scored.
The choice model
The unit of analysis is one dealt menu and the observed pick: four shelf-backed options in an action scene, three dealt signals for an ending or an Afterwards row, and a typed answer as an extra alternative of its own (above). The measurement model is a conditional logit. The utility of option j for person i is:
U(i, j) = Σ over families f of λ_f · s(j, f) · θ(i, f)
where s(j, f) is the shelf's signed loading of option j on family f and θ(i, f) is the person's trait score on that family. In the served configuration every λ_f = 1 and there are no nuisance terms; choice probabilities follow by softmax over the alternatives. A person's θ is the posterior estimate under a standard-normal prior given their banked choices: each session's evidence on every family is added up and the prior is applied once.
With every λ pinned to 1, the model's own standard error claims more
certainty than the instrument delivers — at fifty sessions it implies a
reliability near 0.58 where about 0.2 is measured. So before it becomes the
interval and the confidence word beside a reading, it is calibrated family
by family to the split-half reliability measured through the served path
on the confirm corpus (served_calibration_v1.json). The calibration sets
how far a score may sit from the mean and how wide its interval is; it does
not change how the choice model ranks people. A family with no measured
consistency gets no weight and never shows a number — seven families on the
current calibration. Since 5 October consistency alone is not enough: a number
is shown only for a family whose readings also agreed with the planted
personas (calibration/phaseA/wp30/validity_goal/, rule A) — honesty–humility
and psychopathy on the current evidence. Every other family shows its range.
Like every figure here it is synthetic-persona evidence, and it is re-measured
with each depth run.
Two things the served model deliberately does not do, and why:
- No fitted discrimination in the choice model. Measured to add nothing on the current engine (0.233 versus 0.234 above); on earlier corpora it measured slightly worse. Pinning λ to 1 also removes a dependency on any fit artifact. (The interval calibration described above is separate: it rescales each family's score and uncertainty from measured consistency and leaves the model's ranking of people unchanged.)
- No position or difficulty terms. Real position bias exists in synthetic choice; the served model absorbs none of it. This is a known simplification, and the in-run comparison above says it costs nothing measurable at present.
Seven broad areas
Since 5 October 2026. The profile opens with seven broad areas and nests
the individual traits inside them. An area is not a new measurement. It pools
the evidence its member traits already carry, weighting each by how much
information it holds, with traits that point the other way counted in reverse
(functions/synthesis/broad_areas.py). A member trait with no measured
consistency carries no weight, but it stays in the area's definition.
Pooling related traits pools their evidence, so an area firms up sooner than
any one of its traits; short questionnaires work the same way, with broad
scales markedly more reliable than the facets built from the same items.
| area | built from (− counts the other way) | nearest standard construct |
|---|---|---|
| Honesty and ethics | honesty–humility, fairness, self-transcendence values, − Machiavellianism, − narcissism, − psychopathy | HEXACO Honesty–Humility, broadened |
| Emotional reactivity | neuroticism, attachment anxiety, − stress tolerance, − resilience | Big Five Neuroticism / Negative Emotionality |
| Agreeableness and care | agreeableness, care/harm, empathy, − attachment avoidance | Big Five Agreeableness |
| Conscientiousness and self-control | conscientiousness, impulse control | Big Five Conscientiousness |
| Extraversion and expressiveness | extraversion, emotional awareness, emotion granularity | Big Five Extraversion, with emotional expression |
| Openness and flexibility | openness, openness-to-change values, cognitive flexibility, emotional flexibility | Big Five Openness |
| Tradition | loyalty, authority, sanctity, − liberty | none in the Big Five or HEXACO; the "binding" moral foundations |
An area follows the trait rule. It always shows its range, and it shows a number only when it is both consistent and accurate against the planted personas. Four areas pass on the current evidence; two of those carry an "early reading" tag in the product. Openness passes only under the definition above, which counts openness-to-change values inside openness, and that definition was settled after the results were seen. Tradition passes narrowly and fails a sensitivity check on an older, deprecated corpus. A choice made after seeing the data is exactly the kind that fails to replicate, so both are re-tested on fresh data before they are treated as established. The validation page carries the figures.
The synthetic-persona programme
Ground truth. Personas come from a library sampled by a Gaussian-copula
forge with no language model involved: 3,280 personas in the v5 library
(persona_library_v5.jsonl plus a 400-persona mid-grid supplement), each
with 29 generating trait scales that project onto 27 measurement axes — the
only two-to-one collapses are the two Schwartz bipolar contrasts
(family_map_v4.json, the axis map's version label; unrelated to the
deprecated library v4). A persona's θ on a family is known exactly, which
is the entire point of synthetic ground truth: recovery is scored against
truth, not against another questionnaire. The trait dimensions are named
after public instruments; no instrument was administered to anyone.
The persona instruction. A deterministic sheet of banded, categorical
trait renderings — never the raw trait vector. The v2 sheet names every
construct (an earlier sheet named only the top two moral foundations and one
attachment word, and its under-mention was measured), reads the trait
ladder in a randomised order per persona that is stamped on every row, and
had every band's wording placed by blind readers at the level it claims —
fourteen of fifteen upper emotional-intelligence bands place exactly
(calibration/phaseA/wp24/). Age and gender condition the persona sampling
but are never conveyed to the driver.
The driver. One model family plays every persona on the confirm run. Earlier multi-driver corpora are deprecated, so nothing on this page separates "the trait" from "how this model plays it"; a second driver family is the named reopening condition for the programme's refuted sign-correction lever.
Run integrity. 40 personas × 50 planned sessions in chains of 20, so a
session begins from the faded state its predecessor left. 1,831 of 2,000
completed (8.45% attrition against a 15% gate; wave_summary.json);
completed sessions per persona median 46. The failure that stopped every
persona short of 50 was located to the resume path of chained runs,
instrumented, fixed and tested (the calibration
page has the account).
The testing discipline
Pre-registration. The confirm run's primary question, statistic, null
construction and threshold were filed before the run
(calibration/phaseA/prereg/WP26_instruction_remediation_confirm.md), and
the analysis was run once on the finished corpus with a fixed seed. Two
earlier registered tests in this programme missed their thresholds and were
published as misses; that is what the discipline is for.
The model-free statistic. To test whether a family expresses — whether the persona's choices lean the planted way at all, before any estimator is involved — the programme uses expression: for each persona and family, how far the chosen option's loading sits above its own menu's mean, on menus where the family was genuinely contrasted, correlated with the planted trait across personas. It touches no fitted parameter, so an estimator cannot flatter it.
The shuffled null. "Could this be chance?" is answered by keeping every scene, option and choice exactly as it was and relabelling only which persona each set of choices belongs to — 5,000 times, one relabelling shared across all twelve families so their real between-family correlation survives. Relabelling each family separately would produce a fictitiously tight bar the corpus clears by construction.
Bootstrap intervals per family. Recovery per family carries a 95% interval from resampling the 40 personas. At this roster the intervals are about ±0.27 wide, and they did not narrow between an interim read at 27 sessions and the finish: their width is set by the number of personas, so per-trait certification is a persona-count question (~170 personas for ±0.15, ~380 for ±0.10), not an engine question.
Split-half consistency. Score each persona from half their sessions and from the other half, correlate the two, and apply the Spearman–Brown correction: does the engine describe the same person the same way twice? Only meaningful at depth — at 27 sessions the halves are too thin — which is why the run was not cut short of 50.
Refutations, on the record. Levers the programme has tested and found
dead are recorded with the evidence and a reopening condition
(graph/refuted.yaml): adaptive item selection, item purity, item
retirement, loading sharpening, statistical sign correction, and content
re-authoring for families the persona does not express. Three of them
looked large in-sample and reversed sign held out; every gain is
cross-validated before it is believed.
What the discipline measured
Expression of the twelve benchmark families against a bar of 0.082 fixed in advance; p = 0.0042 against the run's own shuffled null — confirm_result_v1.json
Median across 27 families; 21 of 27 read in the planted direction; 10 confirmed at the group level; 1 confirmed backwards and withheld
Median per-person consistency at ~46 sessions on the registered split (0.168 averaged over 400 splits — served_calibration_v1.json); honesty–humility is the one family above the 0.70 bar
Smallest per-family effect the 40-persona design can resolve (0.75 for the hardest family); per-family figures are descriptive by registration
Detection and measurement are different bars, and this instrument is much stronger at the first. Reading a population is well established on the strong families; reading an individual steadily is established on one. The validation page carries the full per-family table.
What this methodology does not claim
- No human data exists anywhere in the programme. Every θ, every interval, every reliability figure was computed on synthetic personas.
- No per-family significance. The design was powered for one pooled question; the per-family table is description with its detection floor stated.
- No before/after. The current engine is not compared with the deprecated corpora, because the engine, the persona generator and the instruction changed together.
- No conformal coverage. No coverage guarantee has been measured for the
served intervals; they are posterior intervals under a standard-normal
prior and are labelled as such. What has been checked is the arithmetic:
on simulated players who choose exactly as the model assumes, 95% of
served intervals contain the true value (
live_path_sbc_v1.json). That says the pooling is correct, not that the model fits real people. - No measurement invariance has been tested. It cannot be, before a real-user cohort exists.
- No measured gain from the October session on real or persona players. What it adds has been simulated for players who choose exactly as the scoring model assumes; that is a ceiling, and the validation page says what is and is not known.
- No validated reading of typed answers. They are scored at reduced weight by an explicit decision, below their pre-registered bar.
- The persona population's Σ-fidelity check missed its registered bar — relative Frobenius distance ≈0.12 against a 0.10 tolerance — and the measurement stands as a miss.
The calibration page carries the run record; validation carries the results and the withdrawn-figure watchlist.