essay 001 · jan 2026 · lab notebook cut

ANIMUS — the user-model layer.

I gave LLMs my journal. An experiment in scoring my psychological profile from open questionnaire data. The models diverge.

I ran the same open-ended questionnaire, twelve questions ~3000 words — through Claude Opus 4.5, Gemini 3-Pro and Gemini 3-Flash. Comparing model scoring of BFI trait matrix with standardised psychometric tests. The findings point to the importance of the data layer (context) for agent efficacy.

method · ANIMUS-Methods n · 6 chats / 3 models source · animus vault

animus — full writeup

1.0

the itch

why bother building a profile at all.

I'd been journaling for a while, compiling. Notes on projects, internal arguments with myself about direction, looped drafts of who I want to be in five years. The thing about that volume of self-text is that it sits there, structured for a reader who is also you, and absolutely no one else.

Around the same time I'd been building agentic stuff for clients. Internal agents, tool-use loops, MCP servers — the boring middleware of LLM applications. The pattern that kept showing up was that the agent's quality is the context's quality. A capable model with bad context will produce confident slop. A modest model with excellent context can be genuinely useful. RAG, skills, tools — they're all variations of the same idea: better-structured context yields better outputs.

Which led to the obvious question. The most important slice of context for a learning system is the user themselves. Their patterns, their resistances, the way they tend to fall off habits, the conditions that produce flow. If I want an agent to actually help me learn ML fundamentals — not the way it would help anyone, but the way it would help me — it needs more than my prompt. It needs a model of me.

That's ANIMUS. The user-model layer that the rest of BASAIRA — my broader learning-system project — would sit on top of. This article is the first experiment in that direction: can frontier LLMs build a usable psychological profile from journal-style writing, and what does the variance between them tell us about how the models themselves "think"?

The framing came from a different place too. I'd been burning out on the standard productivity-tool template — habit trackers, schedulers, dashboards. Things you have to feed before they feed you. The ones that survived in my life weren't structured. Journaling was a notebook and a pen. Reading what interested me and stripping away external inputs such as social media content. The pattern was that the things that worked were the things with low operational friction and high contextual fit. Most learning systems fail to elicit deep, cross-domain learnings — because most systems aren't designed around who you are and how you actually learn.

2.0

what i tried

three methods, layered.

2.1 baseline · standard bfi-2 questionnaire

I sat the standard 60-item BFI-2 from the Colby personality lab. Likert items, 1–5 disagree-agree. The questionnaire returns a 5-trait summary plus 15 facet subscores. This is the reference point — the format that decades of personality research has validated. It also has known biases (social desirability, item framing, the surface-level read of self-perception).

2.2 open-ended depth questionnaire

I gave Claude Opus 4.5 a bunch of context of the ANIMUS / BASAIRA ecosystem I had in mind, and the role of a psychometric profile within it — and got it to create a journal-prompt-like question set, the answers of which would form the data for the model scoring. The questions were split across five domains:

  • learning & cognition
  • motivation & resistance
  • structure & spontaneity
  • risk & decision-making
  • energy & rhythm

I answered conversationally, with limited (ideally zero) filter, in one sitting. The answers came to ~2,800 words — forming my test dataset. The full question set can be found at the depth questionnaire.

Open-ended question prompts invite evidence, as opposed to self-rating (as found in standardised tests). Instead of scoring your own behaviour, you describe your own cognitive and behavioural patterns — that itself is the data.

2.3 multi-model scoring

I dropped the entire questionnaire-and-answers document into a fresh chat and asked each model to score me on the BFI matrix. No system prompt. No structured output. T=1 where exposed (the Gemini console). I ran two independent chats per model across three models — six scorings in total — to capture both intra-model consistency (same model, two runs) and inter-model variance (different models, same input). The exact prompt:

Figure 2.3 · the scoring prompt, copied verbatim from BFI self-evaluation.md. no system prompt, no constrained output, no temperature override beyond the chat UI default. minimal scaffolding by design — the experiment is partly about what the model brings on its own.

The constraint is intentional. I wanted to measure what each model does when handed an unstructured task and an unstructured input. Adding skills, rubrics or system prompts would tighten the scoring; it would also smother the model-personality signal I was hoping to detect.

3.0

what surprised me

findings, in order of interest to me.

3.1 the journal beat the likert — but only on certain traits

The first finding was specific. On well-defined, internally-rated traits — intellectual curiosity, anxiety, productiveness — the Likert and the LLM-of-journal converged. Same profile, same broad picture. On context-dependent traits — respectfulness, energy, organisation, emotional volatility — they diverged dramatically.

Pick organisation. The Likert returned a mid-high score. The LLM average came in markedly lower — not because one is wrong, but because they were measuring different things. Self-assessment captures your sense of how organised you are; the journal captures how organisation actually plays out under real conditions — the project-switching, the variable momentum, the structure that builds and dissolves across different cognitive states. Two different reads. Both true. The format chose what to look at.

Same shape on energy levels: Likert 94, LLM average 44.5. The Likert measures potential energy — how I would describe myself on a good day. The journal surfaces the actual pattern: state-dependent, variable, structured around sleep and the conditions that produce flow.

3.2 the models disagreed in legible ways

The second finding was less expected. The six scorings did not look like draws from a single noisy distribution. They looked like draws from three distributions — one per model — each with its own mean, its own variance, and its own narrative voice.

Opus was conservative. Across both chats, the average absolute drift between its two scorings was 4.45 points. It avoided extremes. It stayed close to 50 where evidence was thin. It used balanced language: "moderate-elevated", "low-average", "this is where the internal friction lives". It refused to pathologise. The Anthropic model persona they've RL trained into their models — measured, evidence-grounded, careful with downside — is visible in the conservative scoring patterns (relative to the Google models).

Gemini 3-Pro was dramatic. Average intra-model drift: 8.0 points. It produced archetypes: "The Chaos Pilot", "The Perfectionist's Burden", "The Visionary Architect". It moved further from 50 in both directions. Chat 2 was systematically more severe than Chat 1 — depression jumped +20, organisation dropped -15. Same model, same input, same temperature, two different chats: one mood.

Flash was technical. Average drift: 5.5 points. It anchored tightly on Open-Mindedness facets (perfect agreement across chats: 98, 88, 96) but swung wildly on ambiguous ones (depression +25 between its two chats, the largest single intra-model swing in the dataset). Its language was clinical: "psychometric analysis", "high-volatility conscientiousness", "neurodivergent (ADHD) high-performers", "cognitive exoskeleton". Frameworks were its handle. When the data was clean, it locked on. When the data was ambiguous, it defaulted to whichever framework matched first.

—
cross-model bar chart

anthropic · opus google · gemini-pro google · gemini-flash baseline · likert bfi-2

Figure 3.0 · inter-model agreement on a single facet. n = 6 chats across 3 models; baseline shown for reference. the standard deviation under the title updates with the selected trait. the same person; the spread is the model.

3.3 a cross-provider directional bias

The Gemini-Pro and Gemini-Flash models (both from Google DeepMind) largely agreed with one-another — a mean inter-Google difference of 4.4 points across dimensions. Disagreed with Opus in a consistent direction:

  • open-mindedness scored +9.0 higher across Google models
  • extraversion scored -9.7 lower
  • emotional volatility scored +17.0 higher
  • creative imagination scored +11.3 higher

This is clearly a calibration difference. The Google models appear to reserve different anchor points — what counts as "very high" intellectual curiosity for Opus (88-92) is "almost certain" for the Gemini family (98). Whether that's training data, RLHF reward shape, or product positioning is speculation. But the effect is consistent enough that, if I were running this for production, I'd note which model produced any given score, the same way you'd note the brand of a calliper.

3.4 some traits are simply unstable

The most legible diagnostic I have isn't the average — it's the standard deviation. Across the six LLM scorings, intellectual curiosity sat between 88 and 98 (SD 3.95); anxiety sat between 65 and 75 (SD 4.03). On those traits, the data was clean, the evidence was direct, and all three models converged. On other traits, the spread was huge: respectfulness 15-40 (SD 9.57), organisation 10-38 (SD > 11), depression 40-65 (SD 9.67), with two of the three models swinging by 20-25 points between their own two chats — the largest intra-model swings in the dataset.

The hypothesis that fits the data is that variance is a property of the question, not the model. When the journal text contains direct evidence (intellectual curiosity is a parade of "I read X, I synthesise Y, I dive deep into Z"), all three models pick it up and score similarly. When the evidence is inferential — depression has to be guessed at from "negative spirals" without frequency markers — each model fills the gap differently. Models extrapolate to fit their own biases when the data is thin. If you want stable scoring on those traits, the right move isn't a better model. It's a better question.

3.5 the reasoning was the data

Each model also wrote a paragraph of reasoning per facet — its evidence column. Reading them side by side is the most useful diagnostic I have. The facet score is a number; the reasoning trace is the model showing how it got there. These notes reminded me that the utility was not in the output data itself, but the models' ability to interpret the data — and reason through tasks with these findings in their working context. Various interpretations of the same sentence, weighted differently, foregrounded differently. Pick a facet below and step through the chains. The numbers may converge or diverge; the language always says something specific about what the model decided to look at.

—
—
Figure 3.2 · the evidence column. each cell is verbatim from BFI self-evaluation.md — the actual reasoning text the model produced alongside its numerical score. no editing.
4.0

what i now believe

three working hypotheses.

4.1 models are instruments, not oracles

A frontier LLM scoring a person from 3,000 words of journal is, methodologically, much closer to a clinician's interpretation than to a psychometric instrument. The number is the output, but the reasoning is the meaningful part. Treating any single score as the answer ignores the variance you can see in Figure 6.0 — and variance is information. If three independent models converge on a value, that value is well-evidenced in the source text. If they diverge, the source text is ambiguous and the divergence is the model writing its own bias into the gap. Use ensemble averaging, hold the variance loosely, and treat any single score as a working draft, not a measurement.

4.2 question design is a methodological variable

The Likert/journal divergence on respectfulness, energy, organisation and emotional volatility wasn't an isolated error. The two formats were measuring different things. A future ANIMUS questionnaire should be deliberate about which mode each item is in. Aspirational vs. observed. Stable trait vs. state. Internal-narrative vs. social-presentation. The questionnaire should be positioned as an instrument; define closely what is to be measured by the questions and the specific phrasing.

4.3 profile-as-calibration, not constraint

The point of the profile isn't to create a box around the user. It's to calibrate agent behaviour. Given a task — "help me gain a deeper understanding of ML fundamentals" — a learning agent without profile context may create a structured curriculum and working framework. With profile context, it knows that I'll bounce if the gap is immediately too large, that the WHY and underlying concepts should be articulated first, that self-directed project-based learning endeavours suit my cognition. The same agent, the same request, different default model behaviour. Context engineering is a core tenet of agentic use-cases. For learning, user-profiling should comprise a core component of agent context.

That's also the answer to "why bother running 3,000 words through three models". A single Likert-scored summary won't do this. The number 47 for conscientiousness is unhelpful — does that mean rigid-but-tired, or chaotic-but-trying? The journal text says chaotic-but-trying explicitly. The model reading the journal can pick that up; the agent inheriting that profile knows to offer scaffolding rather than schedules. The numbers compress the picture for storage and retrieval, but the picture itself is the writing.

5.0

where this points

into BASAIRA.

Next steps are unglamorous and largely engineering. I want the experiment off the chat UIs and into a structured pipeline — programmatic API calls, T=0 runs alongside T=1, more models per provider, more passes per model, a wider questionnaire. I want to test whether a layered question design — pass one to establish the broad shape, pass two targeted at the high-variance facets — can shrink the standard deviations on respectfulness and depression. I'd also like to add models from outside the US-Anthropic / US-Google duopoly; the inter-provider patterns I saw in this run are too small a sample to claim much, but they are the most interesting thing the experiment surfaced.

A second thing I haven't explored is whether the profile changes meaningfully over time. ANIMUS as I've described it produces a static snapshot. A useful version is longitudinal — same questionnaire, same models, monthly. A profile that drifts is more interesting than a profile that holds, and a profile that drifts in legible directions is more interesting still. The experiment to set up that loop is small; the discipline to actually run it for a year is the real cost.

Above the experiment, the bigger thing it points at is BASAIRA — a learning system that uses ANIMUS-style profiles as calibration for a system that helps the user learn. Not a curriculum, not a tracker. An ecosystem of tools and software built around the user. Personal software is now my default mode of operation. Building tools to bridge gaps in my knowledge, or demos to represent abstract concepts is now commoditised. I'm looking to build the data layer which sits between the user and the surrounding systems and software which serves as calibration. I'm working on a parallel concept at Makaira.ai, where I'm building the data operations layer for the Property Management industry.

go deeper

this article is the finished cut — the field notes, reflection, thinking and process driven writings and raw data are here — in the animus vault.

6.0

the shape

five dimensions, seven scoring methods overlaid · or the full matrix as a table.

The artifact below articulates the scores of the models, as well as from the baseline test. The white line and nodes represent the scores from the standardised BFI questionnaire, the orange lines and nodes represent the scores from Opus, and the teal lines and nodes are the scores from the Gemini models. Hover and click to explore.

view:

BFI radar — five dimensions

five dimensions · 0–100 scale

five dimensions broken into their three facets each. click any panel to drill the main radar into that dimension's facets.

Figure 6.0 · the shape of the data, two views. shape is a five-axis radar with seven scoring methods overlaid (six LLM chats + likert baseline) and small multiples for facet drilldown. table is the full sortable matrix, colour-tinted by absolute distance from 50, dimension rows bolded, facet rows indented. SD is across the six LLM scorings only (baseline excluded). all numbers extracted verbatim from Inter-Model_Analysis.md and BFI self-evaluation.md.
7.0

colophon

  • stack vanilla html/css/js · es modules · pretext for text-aware layout
  • data n=6 LLM chats · 1 baseline likert · 60-item BFI-2 · one author
  • models claude opus 4.5 · gemini 3-pro-preview · gemini 3-flash-preview · all jan 2026
  • type iowan old style · cairo · space mono
  • nav press j and k to navigate · click the section nav at the top to jump · the moon toggles the theme
  • series essay 001 of n · next: ANIMUS v2 — programmatic, ensemble-weighted