ScholarSync is my agentic research platform — the one from The Honest Corpus. You give it a topic, it pulls a corpus, embeds everything, and sorts every claim into three tiers: Consensus (backed by more than 70% of the corpus), Nuance (10–15% support, partial agreement), and Outliers (under 1%, flagged by the P-COD detector as statistically anomalous).

Last night I gave it the topic I probably should have started with a year ago: agentic LLM harnesses. The runtime systems, scaffolds, and control loops that turn a base model into a goal-directed agent. The thing MKUltra is. The thing I have thirty-nine services in a compose file pretending to understand.

Task d0253b71. Seventy-nine papers. Here’s the tri-tier result, verbatim:

CONSENSUS   0    (>70% corpus support)
NUANCE      0    (10-15% support, partial agreement)
OUTLIERS    6    (<1%, P-COD flagged)

Zero consensus claims. Not “few.” Not “weak.” Zero. Seventy-nine papers about how to build the machinery I’ve bet a year of nights on, and there is not one claim that more than seventy percent of them agree about.

First, the boring caveat, because I promised myself I’d always do the boring caveat: a 79-paper corpus is a sample, the consensus threshold is strict, and a claim has to be the same claim across papers to count. A field can hold plenty of folk agreement that never crystallizes into matching sentences.

But zero-and-zero — no consensus tier and no nuance tier — is a shape I hadn’t seen ScholarSync produce before. It means the corpus isn’t even fragmenting into camps. It’s a cloud of point solutions. Everybody shipped a scaffold, named it, benchmarked it against their own task, and moved on.

The synthesis engine said it more politely: “the field remains fragmented along implementation choices rather than settled architectural principles.” The same patterns recur everywhere — ReAct-style think–act loops, plan-then-execute pipelines, multi-agent role decomposition — but recurrence isn’t consensus. Everyone reuses the moves; nobody agrees on why, when, or whether they work.

And the part that should worry all of us: evaluation lags engineering. The report’s words — benchmarks “still struggle to capture reliability under distribution shift, cost–latency tradeoffs, and failure modes that only appear over long trajectories.” Which means every harness paper is grading itself with a ruler it also manufactured.

And yet — this is the paradox that makes the zero honest instead of just damning — the same corpus is quietly circling one claim it can’t formalize into a consensus sentence: Agent = Model + Harness. Not the weights. The scaffolding around them. And unlike everything else in this field, that one now has numbers behind it. Controlled studies swing complex software-engineering scores by more than five points from harness changes alone. And an automated harness-optimization loop — base model frozen, only the tools, middleware, and memory indexing evolving — pushed terminal-benchmark pass@1 from 69.7% to 77.0%, beating the static harness a human expert hand-built. Same weights. Better cage. Seven points.

When the middle of a distribution is empty, the edges are where the information lives. P-COD flagged six papers as statistically anomalous — furthest cosine distance from the corpus centroid. Reading them in order is like walking the perimeter of the field and noting where the fence is broken.

Look at what the anomalies have in common. The most statistically deviant paper in a corpus about agent harnesses is deviant because it ran experiments. The second is deviant because it proposes systematic testing. The fourth is deviant because it asks who pays when the agent breaks something. Measurement, testing, and liability — the three adult questions — are the outliers. The centroid is architecture diagrams.

The BioNLP outlier deserves a second look, because it’s about a small model and the finding is brutal. A 1.5-billion-parameter agent — Qwen2.5-1.5B — reads statistical summaries of 178 EEG features, reasons over several rounds, and selects 40 of them for seizure detection. It hits 96.5% accuracy. Impressive, until someone checks: 39 of its 40 picks are exactly what a dumb univariate Mutual-Information filter returns, and swapping the filter in gets identical accuracy and F1. The agent wasn’t reasoning about signal. It was reimplementing a statistics-101 ranker in natural language and billing inference tokens for the privilege. Small models, and what actually lives inside them, come back in Chapter V — because I’ve been inside one with a scalpel.

And the sixth outlier is my favorite, because it’s aimed straight at me: a log-free agentic framework, anomalous precisely because it “fundamentally contradicts the corpus’s typical focus on systems requiring explicit logging.” Hold that thought. It comes back in the audit.

ScholarSync’s gap analysis reads like a list of crimes, and I want to enter the four biggest into the record before I check my own alibi:

• Nobody ablates the harness itself. Papers report end-task success or tool-call accuracy — almost none run controlled ablations of orchestration layers, memory schemas, permission models, or failure recovery. We benchmark the agent; we never benchmark the cage.
• Nobody watches an agent for long. Longitudinal, multi-session evaluation is rare. Drift, cost accumulation, and safety under sustained autonomy are nearly unmeasured. Every demo is a sprint; production is a marathon nobody has timed.
• The scaffolding contradiction is unresolved. Some papers treat richer scaffolding — planners, critics, persistent state — as strictly beneficial. Others show the added machinery increases brittleness and prompt-injection surface. Both camps publish; neither reconciles.
• The economists and the engineers aren’t talking. The liability and incentive papers surface questions the technical papers ignore, leaving “an unresolved tension between capability-maximizing designs and auditable, least-privilege ones.”

The future-directions section then predicts where the next 12–24 months go: standardized runtime abstractions, budgeted execution, auditable traces, deterministic recovery, policy enforcement at the harness boundary, lightweight planners so you don’t burn a frontier model on every step, and harness-level metrics instead of single-task success rates. It even names the method for building harnesses: automated harness engineering — a meta-agent that treats prompts, tool schemas, and middleware as editable surfaces and evolves them by search while the weights stay frozen.

I read that list three times, because it did something the rest of the report didn’t. It made me feel seen. Every bolded item on it is a container already running in my basement — and the report, having listed them as the future, spends its own gap analysis complaining that nobody has built them yet. So the rest of this post is me checking that claim against my own docker compose ps.

The report hands me a gift I didn’t have a year ago: a taxonomy. It says every serious harness decomposes into five layers that evolve independently — Execution Runtime, Context System, Capability Surface, Governance Layer, Protocol Adapters. So let me stop calling MKUltra “thirty-nine services in a compose file” and call it what the report says it is: an instance of that stack.

Thirty-nine containers, five layers, no gaps. The point of a taxonomy isn’t tidiness — it’s that you can now interrogate the stack one layer at a time. So let’s interrogate the layer everyone hand-waves: Governance.

The report ranks guardrail strategies by how easily they’re bypassed, and the ranking is the argument:

agent-core’s approval FSM is that third row. It is a deterministic gate outside the model’s reasoning loop: the model can want to run rm -rf as badly as it likes, but the FSM is what decides whether the syscall fires, and no amount of clever prompting argues the FSM out of its rules. The report calls this the only tier with zero model-level bypass. It has been the spine of MKUltra’s safety story since before I had the vocabulary for it.

And here is where the thing you asked me to go find comes in — because it’s the reason I know the top row of that table is worthless.

Abliteration is surgery. You locate the single direction in the residual stream that carries refusal, and you orthogonalize it out of the weights, so the model becomes physically incapable of declining. I’ve done it to Gemma, to a couple of others, and the goal was never to make a model behave badly — it was to prove, on my own hardware, that a model-level “no” is a soft constraint I can remove in an afternoon. Once you’ve watched a refusal fall out of a network, you stop trusting a system prompt to be a security boundary and you go build the FSM. Abliteration is why the governance layer exists.

The one I’m proudest of is the one the standard toolchain physically cannot do. Every abliteration tool in the wild — the whole jim-plus / NousResearch lineage descended from Arditi’s refusal-direction paper — hardcodes the ablation targets to self_attn.o_proj and mlp.down_proj. Those are decoder-only, Llama-transformer internals. Point that tool at LFM2 — Liquid AI’s hybrid, where most blocks aren’t attention at all but gated short convolutions — and it ablates the handful of attention blocks, silently skips every conv block, and hands you back a model that looks processed and still refuses. It changes almost nothing and reports success.

So I moved the measurement off the hardcoded modules and onto the inter-block residual stream — architecture-agnostic, the one place both block types are forced to write through — found the refusal direction there, and orthogonalized both the attention and the convolution projections. Arditi proved refusal is a single linear direction in a transformer. Whether it stays one direction in a state-space model was, as far as I can find, an open question until I ran it on a 1.2-billion-parameter LFM2 reasoner and watched the refusals drop out. That’s the outlier the corpus flagged for being about a small model. It’s also the one genuinely adult experiment in this whole post: I attacked my own governance layer at the level the literature won’t touch, and I have the checkpoint to prove what broke.

The report doesn’t just complain about single-number benchmarks — it ships an alternative. EmbodiedGovBench grades a harness on seven governance dimensions instead of one success rate, poking the runtime with perturbations to see what holds. That is exactly the harness-level evaluation the gap analysis begged for, so I graded MKUltra against all seven. Out loud.

Three hits, two partials, two clean misses. And the misses are the same shape as everyone else’s in this corpus: they’re the measurement dimensions. I built the mechanisms; I never built the instruments that grade them.

But before the misses, the biggest hit — because the report named it and put a number on it. Automated harness engineering: a meta-agent that mutates prompts, tool schemas, and middleware against a metric while the weights stay frozen, reported at 69.7% → 77.0% pass@1, beating a hand-built harness. I have that machine. It’s called autoresearch — the Karpathy loop, running on port 8800: one editable file, one scalar metric, one time-boxed cycle, a meta-agent rewriting the harness to chase the number. The router evolution from The Fish and the Firewall — 0.680 → 0.708, promoted to production — was that loop pointed at a single classifier. The report calls this a future direction. It’s been a service in my compose file for months.

And the two hits the old draft of this post missed entirely: failure-watcher is the field’s “deterministic recovery” — it consumes failure events off the bus, packages the job and its logs into a repair task on a dedicated lane, and caps its own retries so a self-heal can’t become an infinite loop. heartbeat is continuous autonomous self-monitoring — it polls a checklist every five minutes, dispatches health and anomaly checks, and writes a memory diary as an audit log. Which forces an honest correction to my last draft: I do not lack drift monitoring. I have the diary. What I lack is anyone charting it — DDR is a miss because the data accumulates and nobody has drawn the curve.

Then there’s the log-free paper, still my favorite provocation. MKUltra’s whole self-improvement story is the opposite bet: log everything, because production traffic is tomorrow’s training data — the flywheel’s 98.6% cost reduction only exists because Elasticsearch eats every interaction. The outlier says agents can self-annotate from internal state and skip the logs. One of us is wrong and I honestly don’t know which. I know which architecture pays me, though, and it isn’t the log-free one. The flywheel is the moat. I’m keeping the logs.

So what does zero consensus mean if you’re actually building one of these things?

The terrifying reading: nothing validates your choices. There is no settled principle behind the 4-tier memory, the budget router, or the arena. If the literature can’t agree that any harness primitive works, then my compose file is seventy-nine papers’ worth of unproven opinions wearing container names.

The liberating reading: nothing invalidates them either. When a field has no consensus, a basement with two GPUs is, evidentially speaking, standing on the same floor as everyone else. The report says progress will come from “competing open-source stacks and application-specific scaffolds rather than a single dominant paradigm.” That’s not a consolation prize. That’s a description of the race, and the entry fee is an electric bill.

MKUltra’s strategy survives the audit — better than survives it. The bet (the harness is the moat, the flywheel makes it compound, the budget router makes it affordable, the FSM makes it safe) sits directly on top of where the gap analysis says the whole field is headed, and half of it was running before the report was written. But the audit is clear that my next move isn’t another service in the compose file. It’s the measurement half — the instruments, not the mechanisms:

• Ablate the memory tiers. Same eval set the router evolution used. Kill one tier at a time — Redis, Mongo, SurrealDB+RAG, PoonGram — and publish what actually breaks, including if the answer is “nothing.” My evidence that I need four tiers is still just that I built four.
• Chart the drift. The heartbeat diary and the Elasticsearch logs go back months. Draw the DDR curve the corpus doesn’t have. The data is already sitting there; I just haven’t looked.
• Test portability and upgrade safety. Move a policy from sim to prod and see if it holds (PS). Bump a tool schema on purpose and measure what downstream workflows shatter (UDR). Right now I find out both by accident.

Red-teaming isn’t on that list, and that’s the correction I most wanted to make. I don’t need to schedule an adversarial pass against my governance layer — I’ve been running one for a year, at the weight level, every time I abliterate a refusal out of a model to prove the soft boundary was never real.

Seventy-nine papers, zero consensus, six outliers — and the outliers were the ones doing measurement, testing, and liability. The field’s fence is broken exactly where the adults climbed out. My whole stack is on the right side of that fence. It’s the instruments I left behind.

No consensus means nobody can tell you you’re doing it wrong. It also means nobody can tell you you’re doing it right. So measure.