Blog
With dev-orchestra 1.2.1, I made one small feature request and measured how long it took (about 6 minutes, not counting my approval) and how many findings were reported and accepted with 1, 2, and 4 reviewers. More reviewers meant more accepted findings, but the extra ones were about tests.
I tried instructions, added in dev-orchestra 1.2.1. Standing instructions for the implementer or reviewers written in the repository go into every prompt. I check, with real output, how they are shown, where they go, and how a change to the instructions is treated.
A roundup of dev-orchestra's changes from v1.1.0 to v1.2.1: running a plan as work units side by side, standing instructions per role, questions before the design, and settings for progress reports. If you capture its output on Windows, note the encoding change.
I tried the work units added in dev-orchestra 1.2.0. I wrote a ## Units section in a plan, had Claude implement two units side by side in separate git worktrees, merged them into the working tree, and ran the final integration unit in place, with the actual commands and output.
I actually updated dev-orchestra from 1.0.0 to 1.1.1 in each of Claude Code, Codex, and Antigravity. In Claude Code, claude plugin update alone did nothing; the marketplace had to be refreshed first. I cover how to check your version, the steps for each CLI, and what to check after updating.
With dev-orchestra 1.0.0, I wrote my own adapter from scratch and added a local LLM as a reviewer. All it takes is a small CLI that takes a prompt and returns an answer, plus an adapter of about 50 lines. I cover where to put it, checking it with doctor, which roles it can and can't take, and the cautions.
How to set the model for each role in dev-orchestra, how to choose reviewer focuses, and config suggest-roles, which proposes domain specialists. I also showed a change with deliberately planted bugs to 9 reviewers to see what changes as you add more of them.
I expanded dev-orchestra's setup presets quality / standard / fast in three environments (Claude Code only, Codex only, and both) and compared them. Here is how the per-role models, the number of reviewers, and how much cost is cut change, and which one to pick.
I installed dev-orchestra in Antigravity too. It works fine as the place you talk to, but handing design and review to the Antigravity CLI (agy) is limited. Here is what works, what doesn't, and how I recommend using it.
I installed dev-orchestra in Codex too. It works with Codex alone, but with both Claude Code and Codex installed, it automatically switches to the recommended setup where AIs from different companies review your code. I also summarized the Codex-specific features and caveats at the end.
I installed dev-orchestra in a Claude Code-only environment and tried everything from the initial setup to a one-sentence request. Installation is two lines, and on first use you just pick a preset; the reviewers are made up of Claude alone. At the end, I also summarize the features and caveats specific to Claude Code.
An introduction to dev-orchestra, a plugin that runs on Claude Code / Codex / Antigravity. I cover what it does, how it differs from the official codex-plugin-cc, why it's worth using even with Claude Code alone, and its downsides.
I wanted code written by one AI to be reviewed by other AIs too. Doing it by hand, copy-pasting between terminals and waiting wore me out. I released version 1.0.0 of dev-orchestra, a plugin that turns that into a system, and this post covers why I built it and how it has felt to use.
How I review code that can't leave the company without sending it to a cloud AI. I register a local LLM as a dev-orchestra reviewer, use only the local model for confidential code, and combine it with cloud AI for code that can be shared. I also cover how good the local LLM's reviews really are, and the pitfall where a single choice of how you invoke it sends your code out anyway.
I put together the steps that make the results trustworthy when measuring the speed or quality of a local LLM. The same conditions vary by 10%, the order changes the result, background work causes 5x swings, and the grader is broken. This is a before, during and after checklist built from mistakes I actually made.
I ran a local LLM on a Ryzen AI mini PC with 96GB of memory and measured how many GB it actually uses. A 35B MoE model takes 24GiB, and even with the context stretched to its 256K limit, it stays around 28GiB. I sort out what 32GB, 64GB and 96GB are each likely to handle, keeping measurements and estimates apart.
All the steps to run a local LLM at around 20 tokens per second on a Ryzen AI 9 HX 370 (Radeon 890M) mini PC, in one post. Install the Vulkan build of llama.cpp, pick an MoE model, add only the settings that help, and end up able to just type llm. Every stumbling point I hit is included.
I put the settings I tried for running a local LLM on a Ryzen AI mini PC on one page, ordered by how much they help. Vulkan gave 5.2x, MoE 5.7x. Some settings did nothing, some backfired, and some were never enabled in the first place. Every number was measured on this machine.
My local LLM server was loading the image mmproj on every start, even though I only use text. I measured whether dropping it makes things faster, following the checklist from my previous post. In the first run the drift during measurement was too large to say anything about speed, and it turned out another AI session was running in the background, so I stopped it and measured again. Memory went down by 1 GiB, and speed stayed the same, within 3%. I made dropping it the default.
While writing 13 posts about building and testing my local LLM setup, the measuring side was wrong 19 times: how I compared, bugs in the grader, interruptions from outside, wrong assumptions, and safety mechanisms that weren't working. Counting how I noticed, more than half came from "impossible values" or "raw data". I turn them into a checklist for before measuring.
I had a local LLM read a 128K-token document and measured whether it could pick up facts planted in 20 places. Nothing was missed because of depth, and for things not in the document it answered "not stated". The only mistake was mixing up a different ID that shared its first two characters, and it went away when I renamed one of them.
Having a local LLM read a 128K-token document takes over 20 minutes. To see whether later requests can be faster, I measured the prompt cache in 6 scenarios. Changing just the question takes seconds, but saving to disk and restarting means re-reading everything. The reason lies in the model's hybrid architecture. I also found a mistake in an earlier post.
I stretched my local LLM's context from 32K to 128K and measured it. The memory shortage I expected never happened, and the reason lay in the model's architecture. The wall I hit instead was prompt ingestion time: just reading 128K takes 21–24 minutes. It also covers how my own auto-start polluted the measurements along the way.
I had turned thinking off by default in my local LLM CLI. To back up a decision I'd made by looking only at speed, I measured accuracy, and it was the same. But on the way there, I hit bugs in the grading script I wrote myself twice.
While making 16 figures across 8 posts, I settled on how to verify colors, a minimum text size, and when to include a figure at all. Here are the rules I use now, including a bar chart whose origin was off and figures so small I remade them all.
I've been running local LLMs on a machine with a 50 TOPS NPU, but I've never used the NPU. When I looked into whether I could, I found there's no guarantee it runs, it's slower than the iGPU even if it does, and its biggest advantage doesn't apply to how I use it. A record of stopping right after downloading the installer.
When I tried to verify a settings decision I'd made earlier, I got the opposite result. Just as I was about to celebrate, I noticed something physically odd, measured again, and the original decision was right. This is about why you shouldn't compare benchmarks from separate launches.
I had three reviewers, Claude, Codex and a local Qwen, review scripts I had written and been running myself. Three were bugs I couldn't have found on my own, and one was in the very script that tallies the review results. There was also a defect all three missed.
I wanted to use the local LLM on my Windows host as a reviewer for development in WSL. There are two routes, and either one can be made to work. But one meant giving up safety and the other meant giving up reliability, so I passed. This is a record of that decision.
Making it fast wasn't enough to get me to use it. How I got to the point where typing `llm` after booting the PC returns an answer, a way to give back 22.7GB when idle, and a bug that was needlessly taking up 48GB.
I added the Qwen model running on my home PC as a third reviewer in dev-orchestra, which has multiple models review code independently. Providers wrap CLIs rather than APIs, so I connected it through a thin CLI. I also added a postscript on the official support that came later and a path trap in the Store version of Python.
I rebuilt the local LLM setup on my freshly wiped Ryzen AI 9 HX 370 machine with llama.cpp and Vulkan: about 7 times faster than CPU inference. Here's how picking the newest model backfired, and measurements showing the bigger file was faster.
To rebuild the local LLM setup I had piled up on my AMD Ryzen AI 9 HX 370 machine with the latest approach, I'm deleting it all first. I cover taking inventory of what's left, the removal steps, and what you shouldn't delete.
I used Claude Code's /insights command to analyze my past 41 sessions. Here's what was in the report, which pointed out what's working, where I get stuck, and even my own habits.
I built a web app that combines weather, earthquake, air quality and heat wave data in real time and shows the risk level of all 47 prefectures as a ranking and a map.
How to install Microsoft's GNU Coreutils on Windows with winget so you can use Unix commands such as cat and rm from PowerShell.
I wanted to switch cookies and localStorage between multiple accounts during development, and couldn't find open-source software I could trust, so I built my own.
On Windows 11 in Japanese, typing "terminal" into Start search doesn't find Windows Terminal. Here's how to fix that by adding a shortcut to the Start menu.
The steps from installing Copilot Keyboard from the Microsoft Store to finishing its initial setup.
Using the C# SDK for the Model Context Protocol (MCP), I built a weather tool and connected it to Claude Code. Here are the steps. It was simpler than I expected.
The steps to run the AI coding assistant opencode with Ollama on a local LLM (Qwen3.6), and my impressions after actually trying it.
What I write about on the tk3.biz blog (local LLMs, development and code review with AI, setting up Windows, and apps I've built), and the rules I follow when writing.
No matching posts.