Engineering Blog · Architecture

Why We Built a Dual-Engine Architecture (System One + LLM) and Open-Sourced CodeOtter

Agents are writing more code than ever, multiplying pull request volume. Here is why we separated deterministic System One scoring from generative LLM walkthroughs—and released the entire stack for free self-hosting under the Elastic License v2 (ELv2).

Last Updated: September 29, 2026·10 min read

TL;DR

AI coding agents are multiplying pull request volume across engineering organizations, turning human code review into the primary shipping bottleneck.

CodeOtter is built for the AI era of software engineering: a self-hosted Dual-Engine review platform where a System One model owns all typed 0–100 gauges and yes/no merge gates while a Language Model writes the summary, file walkthrough, and inline hunk findings.

We invested $1,387.00 building and validating CodeOtter and released the entire platform as free-to-self-host software under the Elastic License v2 (ELv2), backed by community and enterprise sponsors.

Why We Built a Dual-Engine Architecture (System One + LLM) and Open-Sourced CodeOtter — *CodeOtter — Dual-Engine PR Scoring & Code Review Architecture for the AI Era*

Two AI shifts reshaping code review: coding agents and PR volume

Software development is being rebuilt around AI coding agents. Engineers now dispatch multiple agents in parallel to implement features, refactor legacy modules, and patch bugs—multiplying the number of pull requests opened every single day.

While code generation got 10x faster, code review remained manual. Worse, first-generation AI review bots rely on a single conversational LLM prompt that mixes prose suggestions with hallucinated numeric scores—making them too noisy to trust as automated CI/CD gates.

What CodeOtter shipped: two specialized review engines running in parallel

Instead of asking one chat model to do everything, CodeOtter runs two engines concurrently on every pull request:

System One Rubric & Gate Engine: Speaks POST /v1/systemone with state + typed questions (score / choice / noul). When a System One model is configured, it owns every gauge (Quality, Blast Radius, Risk, Tests, Readability, PR Hygiene) and every one of the 9 pre-merge safety gates (Title, Description, Security, Complexity, Tests, Docs, Scope, Guidelines, Issue Requirements). The language model is not asked for scores at all.
Language & Walkthrough Engine: Generates the executive summary, file-by-file change walkthrough, and hunk-level remediation diffs—powered by Claude, OpenAI, MiniMax, OpenRouter, or local llama-server / CodeReviewer sidecars.

// Single Node.js zero-dependency backend (server.mjs)
// Parallel execution: System One (typed scores & gates) + LLM (walkthrough & findings)
const [s1Result, llmResult] = await Promise.all([
  runSystemOneRubrics(diff, guidelines),
  runLanguageWalkthrough(diff, guidelines)
]);
const record = judge(s1Result, llmResult);

Why we released CodeOtter under ELv2 with a transparent -$1,387.00 ledger

Building a dependency-free Node backend (server.mjs), PocketBase schema migrations, JSON-driven React settings/report renderers, and local C++/Python model sidecars took $1,387.00 in engineering and infrastructure investment.

Rather than locking self-hosted teams behind per-seat SaaS pricing, we released the entire CodeOtter stack under the Elastic License v2 (ELv2) for free self-hosting and published our transparent project ledger of $1,387.00 in engineering and infrastructure investment on /sponsors so service companies worldwide, cloud/GPU hosts, and engineering teams can sponsor continued development and receive direct maintainer channels and custom engineering work.