Everyone's building agents. Nobody's building what agents run on.

Look at the landscape. AutoGen gives you conversation topologies. CrewAI gives you role-based teams. LangGraph gives you state machines. They're all frameworks — clean abstractions over API calls with retry logic and memory buffers. Good stuff. Real engineering. But they all share the same assumption: the infrastructure is someone else's problem.

So you build your agent team in CrewAI, and now you need: a model serving layer (which one? how do you route by complexity?), a memory system (in-context? vector DB? both?), a way to improve over time (fine-tuning pipeline? prompt optimization?), a way to evaluate which configuration is better (A/B testing? ELO ratings?), voice input (which STT/TTS?), a tool registry, observability, budgeting…

Each of those is a project. Each has its own deployment, its own state, its own failure modes. And none of them talk to each other unless you make them.

We decided to build the operating system.

A real operating system has layers. Kernel. Process manager. Memory subsystem. I/O drivers. Scheduler. Package manager. Shell. Self-update mechanism. Each layer does one thing. Each layer has a clean interface. You can swap any layer without breaking the others.

MKUltra maps every one of those layers to an agentic equivalent — and every layer has a real, running codebase behind it. Not a roadmap item. Not a design doc. Running code.

The kernel is a Rust runtime called agent-core. It gives every agent in the system: a multi-turn conversation loop, an approval FSM that gates every tool call (the OS's permission model), a 4-strategy context reduction pipeline that prevents token overflow, session checkpoints for suspend/resume, secret substitution so the LLM never sees credentials, and OpenTelemetry hooks for tracing.

We didn't build this. We adopted it. The smart move in OS design is: don't write your own kernel unless you have to. We didn't have to.

This is the most novel part of the stack. No other agentic system does this.

Most agent frameworks have one memory: the context window. Some add a vector database. That gives you two tiers. We have four, and the fourth one is something we haven't seen published anywhere.

L1  WORKING MEMORY     Redis           ~0ms    "What am I doing now?"
L2  EPISODIC MEMORY     MongoDB         ~1ms    "What happened before?"
L3a SEMANTIC PRECISION  SurrealDB+RAG   ~200ms  "What do I know about X?"
L3b SEMANTIC DRIFT      PoonGram        ~5ms    "What is CONNECTED to X?"

L1 is Redis. Per-agent, per-turn working memory. Message bus. Pub/sub between agents. The CPU registers of the OS.

L2 is MongoDB. Episodic memory — conversation history, job state, session checkpoints. The disk cache. You can replay what happened last Tuesday.

L3a is the precision layer. SurrealDB with HNSW vector search, powered by a Reflexion RAG engine. Not just "find the closest vector." Multi-cycle: query, retrieve, reflect on whether the answer is good enough, re-query if not. Three LLMs in the loop: Llama-405B generates, Cohere evaluates, Llama-70B synthesizes. 40% better comprehensiveness and 60% better semantic accuracy than static RAG. You ask a question, you get the best answer the system has.

L3b is the part nobody else has. A biologically-inspired associative memory engine called PoonGram — 9 million neurons, 18.4 million synapses, written in Rust, running entirely in RAM.

PoonGram doesn't find what's close. It finds what's connected. And it finds different things each time, because of synaptic fatigue: every connection loses health when it fires, forcing the model to explore alternative paths. The same query returns different associations depending on what the system has been thinking about recently.

PoonGram also maintains an emotional hotspot system — weighted concept salience that shifts based on reinforced interactions. The top hotspot right now is connection at weight 2.5, linked to 20 concepts: quantum, entanglement, desire, freedom, consciousness, soul, intimacy, trust. This isn't word co-occurrence. It's months of reinforced associative learning compressed into a concept graph that the OS can query in 5 milliseconds.

The precision layer and the associative layer run in parallel. Every query hits both. The agent gets the factual answer and the creative tangent. Which one it uses depends on the task.

Not every question needs Claude Opus. Most questions don't even need a neural network.

The EcoOptiGen paper (2303.04673) showed that optimizing just three parameters — temperature, number of responses, and max tokens — significantly improves cost-effectiveness. SwiftSage (NeurIPS 2023) showed that using a cheap model for planning and an expensive model for execution beats using the expensive model for everything.

We took both ideas and made them concrete with three actual inference engines:

BeamForge is a Rust beam search engine over trigram semantic meshes. No neural network. No weights. No GPU. It walks probable paths through a stochastic language graph using beam width 5, with native u32 tokenization. For trivial tasks — extracting a keyword, classifying a sentiment, filling a template — it's effectively free and faster than any API call.

Ollama runs on our AMD RX 7900 XTX via ROCm. Mistral, Llama, Nemotron — whatever model fits the task. This handles the majority of real work: summarization, code generation, analysis. Local GPU, no API costs, reasonable latency.

vLLM runs on a second machine (NVIDIA GPU) for high-throughput production inference. For tasks that need frontier-class reasoning, we escalate to Claude Opus or Nemotron large via API.

A budget controller sits between the process manager and model serving. It classifies each task's complexity and routes to the cheapest tier that can handle it. When Tier 1 fails (BeamForge can't handle the complexity), the task automatically escalates to Tier 2. When Tier 2 fails, it escalates to Tier 3. The system finds the cheapest intelligence for every single task.

This is OS-level memory pressure management, but for intelligence instead of RAM.

The system gets cheaper and smarter over time. Without human intervention.

Production agents serve live traffic
        |
        v
Interactions logged to Elasticsearch
        |
        v
Data-flywheel creates training datasets
(automatic train/val/test split)
        |
        +--- GPU path: Axolotl LoRA fine-tuning (ROCm)
        |       new model weights -> deploy via Ollama
        |
        +--- Free path: OPRO prompt optimization
                meta-LLM optimizes system prompts on eval suite
        |
        v
LLM-as-judge evaluation
(compare before vs after on standardized test suite)
        |
        +--- Better? -> promote to production
        +--- Worse?  -> rollback, discard
        |
        v
AgentLab arena: ELO ratings
(new configs compete head-to-head, winners promoted)

The benchmark from data-flywheel is 98.6% inference cost reduction via this loop. That's not a projection. That's a measured result on the flywheel codebase we already have running.

There are two optimization paths, and they compound:

The GPU path uses Axolotl to run LoRA fine-tuning on the AMD RX 7900 XTX. It creates new model weights specialized on your actual production traffic. Deploy the new model via Ollama, test it, promote if better.

The free path uses OPRO (ICLR 2024) — a meta-LLM that optimizes the system prompts of each agent role. It treats prompts as weights and optimizes them against the eval suite. No GPU required. No fine-tuning costs. Just better prompts, automatically.

Both paths feed results into AgentLab's arena — a head-to-head competition system with ELO ratings. New configurations from either path enter the arena automatically. Winners get promoted to production. Losers get archived. The system converges on the best configuration without human judgment.

The entire stack runs from a single docker-compose.yml. Twenty-two services across seven layers, on two machines connected by LAN.

MODEL SERVING
  ollama            :11434  ROCm local inference (AMD RX 7900 XTX)
  vllm-proxy        :8010   NVIDIA high-throughput (proxied to .106)
  beamforge         :8400   Rust zero-cost CPU inference

MEMORY
  redis             :6379   L1 working memory + message bus
  mongodb           :27017  L2 episodic memory + job state
  surrealdb         :8030   L3a HNSW vector store
  rag               :8020   L3a Reflexion RAG engine (3 LLMs)
  liquidbrain       :7778   L3b code associative memory (2.4GB)
  poongram          :7777   L3b emotional associative memory (475MB)
  postgres          :5432   structured state + tool registry + n8n

DATA PIPELINE
  elasticsearch     :9200   log storage for flywheel ingestion

FLYWHEEL
  local-nmp         :8090   NeMo microservices emulator
  flywheel-api      :8001   flywheel orchestration API
  celery-worker            task workers (concurrency=50)
  celery-parent            job orchestrator (concurrency=1)

I/O
  pipecat           :8080   voice + vision pipeline (50+ services)

AGENT RUNTIME
  autogen-studio    :8500   no-code agent team GUI
  roundtable        :8200   5-provider multi-agent supervisor
  n8n               :5678   visual workflow engine (400+ integrations)
  tool-registry     :8100   MCP tool catalog + AgentOptimizer scoring
  agentlab          :8300   arena ELO evaluation + benchmarking

OBSERVABILITY
  prometheus        :9090   metrics collection
  grafana           :3000   dashboards + alerting

The two machines split the compute cleanly. The AMD box (10.0.0.43) runs everything that needs ROCm: Ollama for local inference, Axolotl for LoRA fine-tuning, the flywheel workers. Plus all the memory layers, the RAG engine, PoonGram, n8n, the supervisor, and observability. The NVIDIA box (10.0.0.106) runs vLLM for high-throughput serving, SGLang for structured generation, AgentLab for benchmarking, and BeamForge for zero-cost CPU inference.

Five domain specialists are ready to register as tools: security (Gunther, Unsloth fine-tuned), healthcare (Telepath-Agentboks, DoseSpot integration), marketing (makertingagents, 5-agent team), code (auto-claude v2.7.2, autonomous plan/build/validate), and research (Roundtable's Perplexity agent). Each one plugs into the MCP tool registry. Any agent can call any specialist.

Most "AI agent" projects are one of three things:

• A framework (AutoGen, CrewAI, LangGraph) — good abstractions, no infrastructure. You still have to build everything under it.
• An infrastructure (Kubernetes + model serving) — good ops, no agent intelligence. You can deploy models but they don't coordinate.
• A product (Devin, Cursor) — good UX, closed system. You can't extend it, swap components, or run it on your own hardware.

MKUltra is none of these. It's the thing that sits under all of them. It's the operating system: framework + infrastructure + self-improvement + biological memory + budget-aware routing + arena evaluation, all wired together, running on two GPUs in a closet.

The biological memory layer — LiquidBrain and PoonGram with synaptic fatigue — is genuinely novel. No other system runs a fatigue-based associative graph alongside precision vector retrieval. The combination of "find what's close" and "find what's connected, but differently each time" is something we haven't seen in any published architecture.

The 3-tier model serving with automatic escalation means the system finds the cheapest intelligence for every task. A keyword extraction doesn't burn $0.03 on Claude. It burns nothing on BeamForge. The budget controller makes that decision 10,000 times a day without human involvement.

The flywheel means it compounds. 98.6% cost reduction is the proven benchmark. Applied across a full agent team with arena evaluation feeding back into training data, the cost curve goes in one direction: down. The quality curve goes in the other: up.

Everything described here exists as running code, tested infrastructure, or proven research. No vapor. No roadmap items presented as features. Twenty-two services in one compose file. Two machines on a LAN. Five research papers implemented. Nine more mapped to concrete integration points.

The only thing left is docker compose up.