Connecting My Own Agent to dev-orchestra (Local LLM Edition)

Introduction

Last time, I wrote about rearranging reviewers and roles in dev-orchestra. This time, I connect an AI other than Claude Code, Codex, or Antigravity with an adapter I wrote myself. As an example, I add the local LLM I run on my own machine as a reviewer.

I’ve written two posts about connecting a local LLM before.

• Adding a local LLM as a dev-orchestra code reviewer: the trial and error of getting it connected, and how the official mechanism came in with 0.9.0
• Reviewing confidential code with AI without sending it to the cloud: running reviews with only a local LLM

This post skips the backstory and focuses on the steps to write an adapter from scratch in dev-orchestra 1.0.0.

dev-orchestra is open source on GitHub (MIT license). If it looks useful, a star would mean a lot.

→ github.com/istb16/dev-orchestra

How it works: the adapter just “builds a command”

dev-orchestra doesn’t call AIs directly. It just starts a CLI, passes the prompt on stdin, and reads the answer from stdout. The adapter (provider) is what builds the command for that CLI.

Part	Role
dev-orchestra	Builds the prompt and starts the command the adapter returns
Adapter	Returns “which command to start, with which arguments”
CLI	Passes the prompt from stdin to the AI and writes the answer to stdout

In other words, any AI can be connected as long as you have a CLI that reads stdin and writes stdout.

Steps

1. Prepare a CLI that takes a prompt and returns an answer

Local LLMs (such as llama.cpp’s llama-server) expose an OpenAI-compatible API. I wrote a CLI that just sends the prompt there and prints the answer. The standard library is enough.

"""Send a prompt from stdin to a local OpenAI-compatible server and print the answer."""

import json
import os
import sys
import urllib.request

URL = os.environ.get("LOCAL_LLM_URL", "http://127.0.0.1:8080") + "/v1/chat/completions"


def main():
    # dev-orchestra writes the prompt as UTF-8, whatever the OS locale is
    sys.stdin.reconfigure(encoding="utf-8")
    prompt = sys.stdin.read()
    body = json.dumps({"messages": [{"role": "user", "content": prompt}]}).encode("utf-8")
    request = urllib.request.Request(URL, data=body, headers={"Content-Type": "application/json"})
    # Ignore proxy settings so the prompt never leaves this machine
    opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
    with opener.open(request, timeout=1800) as response:
        answer = json.load(response)["choices"][0]["message"]["content"]
    sys.stdout.reconfigure(encoding="utf-8")
    sys.stdout.write(answer)


if __name__ == "__main__":
    main()

Save this as local_llm_review.py wherever you like.

There were two things I was careful about in this CLI.

What I wrote	Why
sys.stdin.reconfigure(encoding="utf-8")	dev-orchestra writes the prompt as UTF-8. Japanese Windows reads cp932 by default, so diffs containing Japanese comments get garbled or crash on read
Use an opener built with ProxyHandler({})	urlopen follows environment variables like HTTP_PROXY and the Windows proxy settings. Even for 127.0.0.1, your code can leave the machine through a proxy

Make the local LLM server listen only on 127.0.0.1 (localhost). If it listens on 0.0.0.0, anyone on the same network can use it.

2. Write the adapter

Next is the adapter. I based it on the minimal example in the dev-orchestra docs and wrote it for the local LLM.

"""Adapter for a local LLM behind an OpenAI-compatible server. Review only."""

from __future__ import annotations

import sys
from typing import Any, Dict, List, Optional, Sequence

from orchestrator.providers.base import (
    READ_ONLY_MODES,
    ModelCandidate,
    ModelResolutionError,
    Provider,
    ResolvedModel,
)

# The small CLI that sends the prompt to the local server
CLI = r"C:\path\to\local_llm_review.py"


class LocalLLMProvider(Provider):
    name = "localllm"
    display_name = "Local LLM"
    executable = sys.executable

    fallback_models = (ModelCandidate("", "default", "server default", "builtin-fallback"),)
    fallback_updated = "2026-10-12"
    option_keys = ("args",)

    def resolve_latest(self, family: str) -> ResolvedModel:
        if family in ("", "default"):
            return ResolvedModel(self.name, "default", "latest", None, "server default", "cli-default")
        raise ModelResolutionError("localllm: only the default model is supported, not %r" % family)

    def build_command(
        self,
        mode: str,
        resolved: ResolvedModel,
        cwd: str,
        extra_args: Sequence[str] = (),
        options: Optional[Dict[str, Any]] = None,
    ) -> List[str]:
        # The CLI cannot read or write files, so it can only review a diff it is given.
        if mode not in READ_ONLY_MODES:
            raise ModelResolutionError("localllm cannot implement: it has no file access")
        return [self.executable, CLI] + self.option_args(options)


def build_provider(executable: Optional[str] = None) -> LocalLLMProvider:
    return LocalLLMProvider(executable)

There are four key points.

Point	Why
from orchestrator.providers.base import ...	This file is loaded from outside the plugin’s package, so relative imports (from .base) don’t work
name = "localllm"	The name you write in provider: in the config. claude, codex, agy, and mock are not allowed
resolve_latest accepts only default	For a model name it can’t verify, it stops instead of guessing, in line with dev-orchestra’s rule everywhere
Refuse runs that aren’t read-only (implementation)	This CLI can’t read or write files. Even if it’s put in the implementer role, it should never pretend to succeed without changing anything

Change CLI to the actual location of the file you saved in step 1. The r"..." in the example lets you write the \ in Windows paths as is. On macOS / Linux, write something like CLI = "/home/you/local_llm_review.py".

Since executable = sys.executable, what dev-orchestra runs to check the installation is python --version. The CLI doesn’t need its own --version.

3. Put it in providers/ in the config directory

The adapter goes in providers/ in the config directory.

OS	Location
Windows	%APPDATA%\dev-orchestra\providers\
macOS / Linux	~/.config/dev-orchestra/providers/

Give the file a name that works as an identifier, such as localllm.py.

Don’t put it inside the plugin (~/.claude/plugins/cache/...). Anything there is deleted when the plugin updates. The config directory is where the config file (config.yaml) lives, so it survives updates.

On Windows, if you use the Microsoft Store version of Python, files you put in %APPDATA% may not be visible to Python (I lost half a day to this in an earlier post). If doctor detects this, it shows the real location and how to fix it.

4. Check that it loaded with doctor

Once the adapter is in place, run dev-orchestra doctor before anything else (asking the agent to do it works too). Here’s what I got (excerpt, paths omitted).

Local LLM (localllm)
  Installed: yes
  Source: user module <config dir>\providers\localllm.py
  Model selection: supported
  Models (builtin-fallback): server default
  Read-only runs: not reported by this adapter
  Resume: NOT SUPPORTED -- localllm does not resume sessions

User providers
  Directory: <config dir>\providers
  Code in this directory is imported at startup, from outside the plugin.
  Imported: localllm  <- <config dir>\providers\localllm.py

It even shows which file it was loaded from. If it ends with No problems found., you’re done. Even if you make a typo, dev-orchestra as a whole keeps working, and doctor tells you the file and the error.

5. Register it as a reviewer

Add it as a reviewer. Don’t forget --model default.

dev-orchestra reviewer add --provider localllm --model default --role general

At first I forgot --model. It got registered with the family name recommended-coding, and at review time my adapter refused with only the default model is supported, not 'recommended-coding'. That’s exactly the “stop on names it can’t verify” behavior from step 2.

6. Have it review

I tried it on the sample with planted bugs from the previous post. But what I wanted to check here was whether the plumbing works. So instead of the local LLM, I used a dummy server that returns a fixed answer.

ok      localllm-general   localllm server default     1 finding(s)

1 successful, 0 failed

The dummy server’s answer came back through my adapter. What this confirmed is the adapter wiring and the parsing of the answer format. If the answer follows dev-orchestra’s format (## Finding …), it goes through deduplication and triage just like the other reviewers’ findings.

For how well a real local LLM actually reviews, see an earlier post (compared with two commercial ones, 1 of 6 findings was adopted) and the confidential-code post.

7. Confirm it refuses when asked to implement

Just to be safe, I set this adapter as the implementer and ran it.

localllm cannot implement: it has no file access

It stopped before starting the CLI. Even if you put it in the implementer role by mistake, you won’t get the hard-to-notice failure of “it succeeded but nothing changed.”

Roles it can and can’t take

Role	With this adapter	Why
Review	Yes	The diff is passed in the prompt, so it works without reading files
Design	Partly	It runs, but it only sees the prompt it’s given. It can’t look up related files on its own
Implementation / fixing	No	It can’t write files, so the adapter refuses

Claude Code and Codex can follow and investigate related files on their own. This CLI only sees what it’s given, so it works best as a reviewer that adds another perspective.

Cautions

• Files in providers/ run with your permissions every time dev-orchestra starts. There’s no sandbox. Only put in files you wrote yourself, or ones you’ve read and trust
• The local LLM server listens only on localhost (the note in step 1)
• On the CLI side, send to 127.0.0.1 without going through a proxy (ProxyHandler({}) in step 1). If you change LOCAL_LLM_URL, keep it a local address too
• If you mean to keep your code on your machine, don’t call dev-orchestra from a cloud AI like Claude Code, because the calling AI reads the code. Run it directly from the terminal and narrow it to the local LLM with --only (the confidential-code post)
• Your own adapter is not added to presets automatically. It only runs when you name it in the config
• Read-only runs: not reported by this adapter in doctor means this adapter doesn’t report how it keeps runs read-only. This CLI never touches files, so there’s no real harm, but when you connect a CLI that can touch files, you need to check for yourself that it really stays read-only
• The parts adapters rely on (the ones used in this post’s example) are covered by the 1.0.0 compatibility promise. Still, it’s reassuring to run doctor once after updating dev-orchestra

Summary

• dev-orchestra just starts a CLI. With a CLI that reads stdin and writes stdout, any AI can be connected
• The adapter is about 50 lines. Import from orchestrator.providers.base and write resolve_latest and build_command
• It goes in providers/ in the config directory, which survives plugin updates
• Once it’s in place, use doctor to check which file it was loaded from
• Register it with reviewer add --provider localllm --model default --role general
• Make a CLI that can’t touch files review-only, and have the adapter refuse implementation
• Code in providers/ runs with your permissions. Only put in what you trust

The detailed rules are in the providers docs.

All posts in this series are in the list of “dev-orchestra” posts.

GitHub: istb16/dev-orchestra (a star is always appreciated)