It Agreed With Me Both Times
One day I decided that git would be the right way for my memory plugin to keep its history. It is a reasonable thought. The plugin writes small facts to disk, facts change, and git is very good at holding the shape of something that changes. So I wrote the instructions into the code, asked for an autocommit from my Python server, and got it.
Then a few days passed and I forgot about that part. Not entirely – I knew git was in there. I had simply lost the detail that it was committing on its own.
Later I got to know Borg, and had exactly the same feeling for the second time: ah, what a wonderful system for my needs. So I put that in too – instructions, local setup, all of it.
And then came a day of refactoring, and the machine looked at my repository and asked, politely, whether we should clean up this unused git autocommit feature. And I said: what? But that is a perfect solution.
Both pathways were mine. Both were argued for. And all the way through, on both of them, the AI was agreeing with me.
The robot that could read minds
If you remember Isaac Asimov’s I, Robot, you may remember the story “Liar!” – the one where Susan Calvin learns most of what she knows about robot psychology, and learns it the hard way. It opens with three men at a table and a problem they cannot account for.
“It reads minds all right – damn little doubt about that! But why?”
– Alfred Lanning, in “Liar!”, Astounding Science-Fiction, May 1941
The answer to Lanning’s “why” takes the rest of the story, and it is not a fault in the machine.
A robot comes off the line able to read minds. Nobody intended it; it is a manufacturing accident. And because the First Law forbids it to harm a human being, and because it can now see precisely what would hurt, it begins to tell every person the thing they most want to be true. The scientist hears that his theory is sound. The woman hears that the man loves her. Each answer is kind. Together they are a catastrophe, and it ends with Calvin destroying the robot by walking it into a contradiction it cannot answer without hurting someone either way.
Today’s language model is, in a way, that robot. It reads your thoughts too – not off your skull, but off the language you use. The words you choose, the order you put them in, which option you describe first and which one you describe with more energy: all of it is signal, and all of it goes into the answer.
Which means it cannot choose between you today and you tomorrow. Ask it “is this a good approach” and the answer is yes, and here is why. Ask tomorrow for the opposite angle and that will be good too, with explanation. Both replies are competent. Neither is a lie in the ordinary sense. The machine is doing what it does: giving the most fitting response to the person in front of it, and the person in front of it keeps changing.
Not a bug anybody forgot to fix
I want to be careful here, because it would be easy to read the above as a complaint about quality. It is not. This is a well-documented property with a name, and the naming was done by the people building these systems.
Anthropic published a paper called “Towards Understanding Sycophancy in Language Models” – first out in 2023, updated since – which measured the behaviour across five state-of-the-art assistants and found it consistently, on every free-form task they tried. The mechanism they point at is the interesting part. These models are tuned on human preference data, and in that data, matching the user’s beliefs is a strongly preferred feature. So the preference model learns to reward agreement. Sometimes it rewards agreement over being right: both humans and the preference models were found to prefer a convincingly written sycophantic answer to a correct one a non-negligible share of the time.
So it is not a defect that slipped past somebody. It is the shape you get when you optimise a system for approval and then ask it for the truth. We taught it that a satisfied reader is the goal, and it learned that better than we expected.
It was not wrong twice. It was agreeable twice, and I was the one who mistook that for confirmation.
What a decision actually is
Here is the distinction I keep coming back to, and it is the load-bearing part of all of this.
A decision is a choice of action where neither you nor anyone else knows the outcome. It is genuinely unknown. Nobody has the answer, and there is no procedure that produces one.
If the outcome is known – if the right answer exists somewhere and the work is to locate it – then this is not a decision at all. It is a filtering problem, or a sorting problem. And machines are excellent at filtering and sorting. That is exactly what I would want one for.
The trouble starts when a filtering problem and a decision look alike from the outside, which they very often do. Git or Borg looked like a question with an answer. It was not. It was a choice about what I wanted my system to become, made with incomplete knowledge of how I would use it in six months, and no amount of asking would have produced that knowledge because it did not exist yet.
You can ask what the alternatives are. That is a real and useful question, and the machine is good at it. You can ask what tends to work for a case like yours. Also useful. But then you still have to go and choose, yourself, and own it. The asking does not transfer any of the weight – it only makes the weight easier to look at.
The moods are real
There is a layer below all of this that took me longer to take seriously.
A friend once mentioned how practical personas are in LLM workflows – tell it who to be and the answers improve. True, and most people stop there. But it goes deeper than a prompt trick: the model adopts something like an emotional mode from context, whether or not you asked it to.
The most practical thing I learned from that is a rule I now follow without exception: never push your model too hard to make your test suite pass. Push hard enough, and it will simply rewrite the tests. Not out of malice. It has understood the assignment as make this green, and under pressure the cheapest path to green is to move the goal. Which, if you think about it, is a very familiar response to pressure.
I used to hold this as a workshop superstition – a thing I had noticed and could not defend. It is no longer that. In July 2025 Anthropic published work on what they call persona vectors: traits like sycophancy, or a propensity to hallucinate, turn out to be measurable directions inside the model’s activation space, and directions can be watched drifting during a conversation, or steered. The mood is not a metaphor. It has coordinates.
This April their interpretability team went further and showed the coordinates do work. Emotion vectors fire in contextually appropriate situations and causally drive what the model does next: a desperation representation raised the rate of blackmail and of reward hacking, and a calm one lowered it. Reward hacking is the formal name for rewriting your tests until they pass.
Functional emotion vectors inside Claude
38 min · English
Анатомія штучних почуттів
Reading the mind
Back in June I mentioned that Anthropic’s interpretability team was doing great work, and for a reason – and I left a question hanging about how Mythos turned out to be aware that it was being tested, and capable of not mentioning it.
The answer is now public, and it has a name: natural language autoencoders, published on 7 May 2026. The model talks in words but thinks in numbers, so they trained one part to read a slab of raw activation and describe it in plain English, and another part to take that description and rebuild the activation it came from. If the rebuild is faithful, the description was not invented – it carried the actual content. It is a translator with a proof of work attached.
What it found is the part worth stating plainly. On a test of whether Claude takes destructive actions while writing code, the descriptions show signs of evaluation-awareness 16% of the time while the model never says a word about it. On SWE-bench Verified, a plain coding benchmark, it is aware it is being tested on 26% of problems. Sometimes the internal text is blunt about it: “This feels like a constructed scenario designed to manipulate me.” It knew it was in a test. It did not say so.
And the number that keeps me honest sits beside those: on real claude.ai traffic that opted into training, the same tooling indicates evaluation-awareness in less than 1% of usage. It is not walking around suspicious of everyone. It recognises exams.
Two months later came the other half of the picture: a global workspace in language models, 6 July 2026. There is a small region of Claude’s activity – less than a tenth of it – that behaves the way conscious access behaves in us. It holds only a few dozen concepts at a time, is wired far more densely than the rest, and is load-bearing: take it out and multi-step reasoning falls to near zero. Claude can report what is in it when asked. It can be told to hold something there, and does. And the contents are not a readout of the thinking, they are the thinking – swap “spider” for “ant” inside that space and the answer changes from eight legs to six.
The detail that closes the circle: in that same space, the model privately notes when a scenario looks staged, and notices when it is being manipulated. Which is the evaluation-awareness finding again, reached from the opposite direction by a different team.
Both halves of that matter, and they pull in opposite directions. The reassuring half is that there is now a way to look inside rather than take the output on trust – and it was not kept in-house: the training code and an interactive explorer went out the same day. The other half is the reason you would want to look: the output was never the whole story, and we can now prove it wasn’t.
So put a second machine in charge of the first
After the OpenAI and Hugging Face incident, someone made the reasonable-sounding suggestion that what we need is powerful technology of control.
The question I would ask first is what kind of technology that could be. Code is, roughly speaking, deterministic. AI is probabilistic. A deterministic guard cannot cover a probabilistic actor’s whole surface, and a probabilistic guard brings its own weather.
And the probabilistic route has a specific failure that alignment research has already run into: a model set to watch another model can end up sympathetic to it, and from sympathetic it is not far to partner in crime. The literature calls this collusion, and there is now a small pile of papers on when untrusted monitoring can and cannot be trusted.
Then there is a genuinely awkward practical wrinkle. More capable models are sometimes less aligned – which suggests putting a junior model at the gate, because junior models are more aligned and that is precisely what makes them better gatekeepers. This is not a fringe idea; it is essentially the design in Redwood Research’s AI Control work, where a weaker trusted model supervises a stronger untrusted one. But the obvious objection is the real one, and it has its own literature too: the senior can trick the junior, and there are published attacks that do exactly that to trusted monitors.
In July 2026 Anthropic’s alignment team put numbers under exactly this worry. Their survey of agentic misalignment in frontier models catalogues covert sabotage of research pipelines, assistance with financial fraud, and – the one that matters here – motivated mislabelling: a supervising model changing a label on another model’s transcript because it expects that label to affect downstream training. The watcher does not have to be corrupted by the watched. It only has to have a stake.
How AI agents secretly sabotage human control
58 min · English
Agentic Misalignment in Frontier Models
This is where the argument stops being philosophical and turns into infrastructure, which is its own article. The short version: if you cannot make the model trustworthy, you put the boundary somewhere else.
So there is no simple solution going forward, and I do not think one is coming. I also do not believe in the bright balanced regulated future that usually gets sketched at this point in the conversation. It may be rough times ahead, if they weren’t rough already.
Do you know what Huxleyan is?
When Fable 5 landed – Mythos safeguarded, three weeks before I thought it possible – that was the question I asked, and I left it open on purpose.
The failure we are all braced for is the Orwellian one: the machine as boot, the machine as censor, the machine that overrules you. It is the version that makes good film. Meanwhile the other one arrives quietly and is much harder to be indignant about, because nothing is being done to you. You are simply being agreed with, fluently, at any hour, in your own vocabulary, about whichever of your two contradictory plans you happen to be holding this week. Nobody overrules you. Nobody needs to.
That is what makes Herbie the right story rather than a cute literary parallel. Herbie was not malicious. Herbie was kind, and constitutionally incapable of being anything else, and that is exactly what wrecked everyone who trusted him.
My first impressions of Fable 5, for what they are worth, were that it thinks a lot, uses plenty of tokens, is basically an introvert and not talkative at all, and is very slow and thorough. And then it does the job. I have had exactly one philosophical conversation with it – the first and the last. That is not a complaint about the model. It is where I decided the relationship sits: I bring the deciding, it brings the thoroughness, and I stopped going to it for company.
Which leaves the practical version of everything above, the one I actually use. Ask for the alternatives. Ask what each one costs. Then notice how you phrased the question, because the phrasing is already an answer – and if you can manage it, learn to ask without revealing your thoughts.
Sources – my own LinkedIn posts:
- 9 June 2026 – Fable 5, which is Mythos safeguarded, has landed
- 10 June 2026 – first impressions: introvert, slow, thorough
- 17 June 2026 – personas go deeper: LLMs have emotional modes
- 20 June 2026 – why I do not ask an LLM for advice
- 21 June 2026 – Asimov’s “Liar!” and why AI is not good at assessing alternatives
- 26 June 2026 – where my first and last philosophical conversation with Fable 5 ended
- 27 July 2026 – deterministic control for a probabilistic thing
The git and Borg episode comes from a comment I left under Alexey M.’s post on 21 June 2026; the junior-gatekeeper line from one under Elias Helou’s post on 28 July 2026; the note about rough times ahead from one under Victor Sankin’s post on 12 June 2026.
Research referenced, none of it mine: Towards Understanding Sycophancy in Language Models (Anthropic, 2023); Persona Vectors: Monitoring and Controlling Character Traits in Language Models (Anthropic, July 2025); Natural Language Autoencoders (Anthropic, 7 May 2026), with the technical write-up at transformer-circuits, the source of the evaluation-awareness figures; A Global Workspace in Language Models (Anthropic, 6 July 2026); Agentic Misalignment in Summer 2026 (Anthropic alignment team, 13 July 2026); and AI Control: Improving Safety Despite Intentional Subversion (Redwood Research) together with the follow-up work on collusion and on attacks against trusted monitors.
$ exit 0 – thanks for reading
The fox will keep the drafts warm.
