Introduction
If you only ask “Let tasks have due dates.”, dev-orchestra asks questions before the design. In part 16, with design.ask_questions: auto, three questions arrived together, and I could answer by picking from the options. It is easy, but ideas that are not among the options, and any vagueness in my answers, flow straight into the design.
1.4.0 added grill, which turns these questions into an interview. According to the CHANGELOG, it asks one question at a time and digs deeper when an answer is vague, contradicts an earlier one, or adds a condition. I tried it with the same request as part 16.
To give the result first: it asked one question at a time, eight in all. An answer that added a condition did get a follow-up question. On the other hand, when I deliberately answered vaguely, it interpreted the answer and moved on without probing, and the intent and success criteria listed in the docs were never asked. I tried it only once, so all of this is simply what happened this time. The other changes are covered in the v1.4.0 release post published the same day.
dev-orchestra is open source on GitHub (MIT license). If it looks useful, a star would make me happy.
One question at a time, eight in all
The setting is one line. This time I changed it in a settings folder separate from my usual one.
dev-orchestra config set design.ask_questions grill
With the same small Python todo package as part 16, I gave Claude Code the same request, and the first question came back in 27 seconds.
質問は1つずつ行います。まず最初の質問です。
**Q1. 期限はどの粒度で指定しますか?**
- (a) 日付のみ(例: `2026-12-01`)
- (b) 日付と時刻(例: `2026-12-01 18:00`)
私のおすすめは **(a) 日付のみ**です。この CLI は小さく、時刻まで扱うとタイムゾーンや入力形式の考慮が増えます。後から (b) へ拡張することもできます。
ご回答をいただいたら、次の質問(指定方法、一覧での表示と並び順、期限切れの扱い、既存データの互換性)に進みます。「お任せ」の場合はその旨をお知らせください。私の推奨で進めます。
(It says: questions will come one at a time. Q1 asks how fine-grained the due date should be, (a) date only or (b) date and time, and recommends (a) because the CLI is small and times would bring time zones and input formats into play. Next it will ask how to set the date, how the list shows and sorts it, how overdue tasks are handled, and compatibility with existing data. If I leave it to the AI, it will go with its recommendation.)
Under auto in part 16, the three questions came in one message; under grill, it is one question per message. Each still comes with options and a recommendation, so each question is just as easy to answer. In exchange, there are more round trips. These are the eight questions I was asked.
| Q | Asked by | Topic | My answer |
|---|---|---|---|
| Q1 | Orchestrator | Granularity of the due date (date, or time too) | Answered vaguely, then date only in the next answer |
| Q2 | Orchestrator | How to set a due date | add --due and due <id> |
| Q3 | Orchestrator | Display and order in the list | Sort by due date only with list --sort due |
| Q4 | Orchestrator | Whether a due date can be removed | Remove it with due <id> none |
| Q5 | Architect | How a list line is written | Parenthesized form. I want overdue ones in red |
| Q8 | Orchestrator | Follow-up to Q5: what to color red | The whole line, only in a terminal |
| Q6 | Architect | Whether past dates are accepted | Left to it |
| Q7 | Architect | Where done tasks go under --sort due | Open tasks first |
Q1 to Q4 were the orchestrator’s questions (the calling Claude Code). Once I answered Q4, the architect’s question round, where it reads the code, ran, and Q5 to Q7 followed one at a time. Q8 was a follow-up to my answer to Q5. The numbers follow the order they were recorded, so they differ from the order they were asked.
A vague answer was interpreted, and it moved on
What I wanted to test was the probing. I deliberately answered Q1 vaguely.
うーん、だいたい日付でいいけど、場合によっては時間もほしいかも。
(Roughly: “Hmm, a date is mostly fine, but in some cases I might want a time too.”)
What came back was not a follow-up question.
了解です。「基本は日付のみ、必要なら時刻も付けられる」という方針として記録します。入力は `2026-12-01` と `2026-12-01 18:00` のどちらも受け付け、時刻は任意とします。
**Q2. 期限はどのように指定しますか?**
(It recorded the policy as “date only by default, with an optional time when needed”, accepting both 2026-12-01 and 2026-12-01 18:00, and went on to Q2: how is a due date set?)
It did not ask what “in some cases” meant; it read my answer as accepting an optional time and moved to the next question. When I added “Actually, I don’t need a time. Make it date only.” to my answer to Q2, it took that as “Q1 is settled as date only” and withdrew the time-related decisions. The recorded answer to Q1 is the “date only” I restated at the end.
My restatement was me correcting my own answer to the earlier question, so it does not count as testing what the docs call an answer that contradicts an earlier one. What caught my attention was that the first vague answer was about to be recorded as the policy as it was. In fact, the Q2 question already carried the time-included premise: it said that when the time is omitted, the plan was to treat the date as “valid until the end of that day (23:59)”. This time it was withdrawn because I restated it myself, but had I not, it would have gone ahead on that premise.
Adding a condition brought a follow-up question
For the architect’s Q5 (how a list line is written), I answered with one condition added to an option.
a で。ただ、期限切れの行はターミナルで赤くしたい。
(Roughly: “Go with a. But I want overdue lines in red in the terminal.”)
This got a follow-up question right away.
Q5 は「(a) 括弧付きの接尾辞、期限切れの行は赤で表示」として記録しました。
**Q8. 期限切れの行を赤くする条件と範囲は?**(Q5 の追加質問)
- (a) 期限切れの行全体を赤にする。ただし標準出力がターミナルのときだけで、環境変数 `NO_COLOR` が設定されている場合は赤にしない。
- (b) `OVERDUE` の語だけを赤にする。条件は (a) と同じ。
- (c) 期限切れの行全体を、常に赤にする。
(Q5 was recorded as “(a) parenthesized suffix, overdue lines in red”. Q8, a follow-up to Q5, asks when and how much to color: (a) the whole overdue line, only when stdout is a terminal and not when NO_COLOR is set; (b) only the word OVERDUE, under the same conditions; (c) the whole overdue line, always.)
Behind the simple “I want it red” hides a question: will color control characters get mixed in when the output goes through a pipe? The follow-up broke that point out into options and asked me back. In the record, too, Q8’s title starts with Follow-up to Q5:, as the docs prescribe.
Q8 [answered] Follow-up to Q5: Red overdue lines (asked by the orchestrator)
When and how much should an overdue line be colored red?
It asked about the spec, not the intent
In the docs (Design questions), under grill the orchestrator is to ask up to five questions, most important first, about whatever the request does not answer, from these angles: the intent, how success will be judged, what is out of scope, the alternatives rejected and why, how easy it is to undo, how it could fail, and whether a smaller change or doing nothing would be enough.
This time, Q1 to Q4 were all spec questions: granularity, how to set it, display, and how to remove it. “Why do you want due dates?” and “What counts as success?” were not asked. Perhaps it judged that the subject was small and the intent could be read from the request, but I only tried it once, so I can’t confirm that. If you want the interview to pin down the intent as well, for now I think it is more reliable to write the intent in the request itself.
One more thing: the docs say each question is recorded before it is asked, but this time Q1 to Q4 were recorded together after I answered Q4. The final record had all eight questions and their answers, so the result was no different.
The plan records who decided what
When I answered the last question, Q7, the architect wrote the plan and, with the plan updated for the 12 findings from the design review, asked for my approval. The plan’s ## Decisions lists what I decided and what the architect decided separately (excerpt).
- **Q5 Line format** (user): parenthesized suffix, `(due D)` or `(due D, OVERDUE)`.
- **Q6 Past dates when setting** (me): accepted silently, with exit 0 and no warning. The task shows OVERDUE on the next `list`, which is the feedback. Rejecting would stop users recording tasks that are already late, and would make writes depend on the clock.
- **Q7 Done tasks under `--sort due`** (user): open tasks first, then done; each group by due with no-due last; ties by id.
- **Q8 Red** (user): whole overdue line, only when stdout is a TTY and NO_COLOR is unset.
For Q6, which I left to it, the record kept what the architect decided along with its reason. This format is the same as in part 16. This time I stopped when it asked for approval and did not go on to implementation.
From the request to the approval prompt took about 8 minutes, including waiting time. Of that, the last round trip, which included the design and the design review, took 4 minutes 42 seconds. The cost of the architect and design review recorded by dev-orchestra was $1.69, and the amount reported by the Claude Code session (the running total over the nine round trips) was $0.92, about $2.61 in all. Both are at API prices.
When to use it
With grill, every answer costs a round trip. This time it was eight questions, more effort than the three under auto in part 16. In return, when I added a condition, it broke the problem behind that condition (color and pipes) out into options and asked about it.
If you are counting on it to dig into vague answers, from one try you can’t rely on that yet. If you notice your answer was vague, restate it in the next answer. If the intent or success criteria matter for the change, write them in the request. Cover those two, and I think it works as a tool for reducing gaps in pinning down the spec.
Summary
- Under
grill, the questions before the design come one per message; this time there were eight - An answer that added a condition got a
Follow-up to Q5:follow-up question. A vague answer was not probed this time; it was interpreted and the run moved on - Angles in the docs such as the intent and success criteria were not asked; it was spec questions only. Writing the intent in the request is more reliable
Test conditions
| Item | Details |
|---|---|
| dev-orchestra | 1.4.0, preset standard, design.ask_questions: grill |
| Caller | Claude Code 2.1.327. I made the request with claude -p and passed my answers with claude -p --resume |
| Architect | Claude Fable |
| Subject | A small Python package todo I made for this (a Store that keeps tasks in JSON, a CLI with add, done, and list, and 2 tests) |
| Environment | Windows 11, Python 3.14.10. The settings were in a separate folder (DEV_ORCHESTRA_HOME), not my usual one |
I tried it only once, with a single request. I did not test how it behaves in runs nobody is watching (CI and the like).
The fine print
This section covers the rules written in the docs and the CHANGELOG. What I confirmed in this run was only that questions come one at a time and the Follow-up to Q<n>: title; I did not test the caps or the behavior in runs nobody is watching.
The setting’s values
design.ask_questions has four values: false (default), auto, true, and grill. grill is the same as true in that a question round always happens; only the way of asking becomes an interview. The architect’s question rounds and their cap (design.max_question_rounds, default 2) do not change either. The Design questions: line in status shows grill. mode in status --json is grill too. Since 1.4.0, lines that show the setting’s value show true as true rather than on, but in --json, mode for true stays on as before.
The interview rules
The orchestrator’s own questions are capped at five, most important first. Follow-up questions are capped at two per question. After ten questions including follow-ups, it asks the user whether to continue. A follow-up’s title must start with Follow-up to Q<n>:, and it is recorded as an ordinary orchestrator question. The CLI does not count any of these, and it does not check the link between a follow-up and the original question. It is up to the orchestrator to follow the rules. If the user says “that’s enough”, the rest is left to the architect (--delegate).
When nobody is watching
With design.require_approval: false, or with design.require_approval: auto when CI or DEV_ORCHESTRA_UNATTENDED is set, there is nobody to answer. Then grill behaves the same as true: the questions are just recorded together, and no follow-ups are asked. Questions still open are recorded as left to the architect by the run that writes the plan.
The details are in the docs, Design questions.
GitHub: istb16/dev-orchestra (a star would mean a lot)
All posts in this series are on the “dev-orchestra” list.
Related Posts
I Had dev-orchestra Ask Me Questions Before the Design
I tried dev-orchestra 1.3.0's questions before the design (design.ask_questions) with a vague request and a clear one. Under auto, only the vague request got questions; under true, the architect asked four even about the request I thought was clear. When someone is watching, no plan is written until every question is answered.
I Had Claude Code's /insights Analyze How I Use It
I used Claude Code's /insights command to analyze my past 41 sessions. Here's what was in the report, which pointed out what's working, where I get stuck, and even my own habits.
[Released] dev-orchestra: A Pre-Design Interview, and Less Work in Reviews v1.4.0
A roundup of dev-orchestra v1.4.0. It adds an interview that asks one question at a time before the design (design.ask_questions: grill). In reviews, reviewers are told what was already decided, minor pre-existing problems stay out of triage, and re-review rounds run fewer reviewers. The CLI also picks the light tier itself, and a workflow can continue in a new session.