Most of my projects start with a feeling and end with a repo. This one started with a document I wasn’t allowed to edit.

instructions.md — the original brief for ScholarSync, frozen on purpose. Scrape academic papers across three sources. Run a Map-Reduce analysis that splits findings into Consensus, Nuance, and Outliers. Let a human chat with the corpus. Draw the semantic clusters. Render a report a person would actually read.

Then I did something I normally skip: I built it spec-first, with GitHub’s Spec-Kit, which forces you to write a constitution before you write a function. Seven binding principles. Async-first. Library-first decomposition. Type safety end to end. Modularity by service boundary. Observability. Spec-driven evolution.

And principle number six, which is the reason this post exists:

Write that down before you write the scraper and it changes what you’re allowed to ship. I did not fully understand what I’d signed until the thing ran.

The design says 120 papers. Forty from Semantic Scholar, forty from ArXiv, forty from Crossref. Three sources so no single index gets to decide what the literature says.

First real smoke test, the corpus came back at 70.

ArXiv gave me 40. Crossref gave me 40. Semantic Scholar gave me nothing at all — anonymous tier, 429, rate limited into silence.

healthz: ok
corpus:  70   (40 arxiv + 40 crossref)
s2:      0    (429 — anon tier rate limit)

There are two things you can do here. You can quietly relabel the feature “up to 120 papers” and let the marketing carry the shortfall. Or you can write down that a third of your evidence base didn’t show up, put the number on the health endpoint, and let the user decide whether a 70-paper consensus is worth anything.

I’d already signed the constitution. The corpus count ships as a fact, not a target.

The pipeline is Map-Reduce with opinions.

Map: every paper gets summarized independently. No cross-talk. Each one gets read on its own terms so a loud abstract can’t colour its neighbours.

Reduce, three ways. This is the part I actually care about. Instead of collapsing seventy summaries into one authoritative paragraph — the thing every “AI research assistant” does, and the thing that quietly launders disagreement into false confidence — the reducer runs three separate passes over the same evidence:

• Consensus — what does the corpus broadly agree on?
• Nuance — where does it agree with caveats, conditions, contested mechanisms?
• Outliers — which papers disagree with the pack, and are they cranks or early?

Outliers get detected structurally, not vibes-based, and they come back with paper IDs attached so you can go read the dissent yourself.

await _emit("reduce.consensus", 0.55, "synthesising consensus")
consensus = await reduce_consensus(reduce_llm, summaries)

await _emit("reduce.nuance",    0.65, "synthesising nuance")
nuance    = await reduce_nuance(reduce_llm, summaries)

await _emit("outliers",         0.78, "detecting outliers")
outliers  = await detect_outliers(...)

The whole thing streams progress events as it goes, because a ninety-second silent spinner is how you teach a user that your software is broken.

Then projection.py drops the corpus into 2D so D3 can draw it as a cluster map. Seventy papers as a constellation instead of a list. You find the lonely dot on the edge and that’s usually where the interesting argument is.

Default LLM provider: LiquidBrain, my own thing, port 7778.

At bootstrap, LiquidBrain was not listening. (Regular readers will recall what was wrong with LiquidBrain around this era. It was busy being a parrot.)

So the provider abstraction did its job: probe failed, connection refused, fall through to Ollama on 11434, keep going. And then — the part that matters — it said so:

llm_provider: "ollama (fallback)"

Not "ollama". Not a silent swap. The health endpoint tells you the primary died and you’re running on the understudy.

Every silent fallback I’ve ever written has eventually cost me a week. A system that degrades gracefully and doesn’t mention it isn’t resilient — it’s just lying with extra steps.

Small chapter. Purely for anyone who runs too many services on one box and thinks it’s only their problem.

Backend was specced for :8000. Port taken — a nginx container from an entirely unrelated healthcare project sitting on it.

Frontend was specced for :3000. Also taken — by the MKUltra frontend.

ScholarSync lives on :8088 and :3088 because my own past projects are squatting the default ports on my own machine like pigeons on a statue.

Semantic Scholar rate-limited the scraper. Then the LLM tier started rate-limiting the analysis, because seventy map-stage summaries is seventy calls before you’ve reduced anything.

Three fixes, in order of increasing shamelessness:

Split the models by job. The map stage is bulk work — seventy cheap summarizations where quality per call barely moves the final answer. The reduce stage is where the actual thinking happens. So they got separate model configs: a cheap fast one for bulk, an opt-in heavy override for reduce. Spend your good tokens on the synthesis, not the grunt work.

Rate-gate the provider. Requests get paced so the pipeline degrades into slow instead of failed.

Rotate the free-tier keys. I wrote a Gemini key rotator. It is exactly what it sounds like. I’m not going to pretend it’s elegant — it’s a pile of free-tier keys in a trench coat — but a research tool you can’t afford to run is a research tool you don’t run.

Every honest local-AI project has one component like this. Mine rotates API keys.

The reason this project matters to me isn’t the stack. It’s that the interesting engineering all landed in the same place: refusing to smooth things over.

Three sources instead of one, so no index gets to be the truth. Three reduction tiers instead of one summary, so disagreement survives contact with the synthesizer. A corpus count that reports 70 when it hoped for 120. A health endpoint that admits it’s running the backup model. Outliers that ship with IDs so you can go check whether the weirdo paper is wrong or early.

Every one of those is a decision to make the tool less impressive and more useful. And every one of them came from writing the constitution first, when I was still theoretical about my own integrity and hadn’t yet met the demo where the number came back low.

That’s the actual argument for spec-driven development, and it’s got nothing to do with planning. You write down what you refuse to do while refusing is still free.

The failure mode of every AI research assistant is the same: it reads a contested literature and hands you one confident paragraph. It launders seventy arguments into a voice with no doubt in it. That’s not synthesis. That’s a machine for producing the feeling of having read something.