Created
April 28, 2026 21:13
-
-
Save SgtPooki/dae9efdd32e45e1364f7a52819380f78 to your computer and use it in GitHub Desktop.
Revisions
-
SgtPooki created this gist
Apr 28, 2026 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,75 @@ # Dirac retry-loop bug — full repro and logs Companion to [issue dirac-run/dirac#XXX](https://github.com/dirac-run/dirac/issues). ## Environment - `dirac-cli` 0.3.3, Node v24.11.1, macOS (Darwin 25.3.0) - Provider: OpenAI-compatible (vLLM) via `--baseurl https://my-vllm-endpoint/v1` - Failing model: `qwen36-27b-aeon` (27B, AWQ-quantized Qwen 3, served by vLLM) - Passing model: `qwen3-32b` (32B, served by vLLM) - Mode: `--act --yolo --reasoning-effort medium` ## Reproduction ```bash mkdir -p /tmp/dirac-repro/src && cd /tmp/dirac-repro echo '{"name":"repro","version":"0.0.1"}' > package.json echo "# repro" > README.md echo "export const x = 1" > src/index.ts echo "export const y = 2" > src/util.ts git init -q && git add -A && git commit -qm init dirac auth -p openai -k EMPTY -b <vllm-base-url> -m <model> dirac -y -a -c /tmp/dirac-repro --max-consecutive-mistakes 6 \ "Investigate this repo. First list all files. Then read each TypeScript file. Then summarize the structure and any quality issues." ``` ## Result with `qwen36-27b-aeon` ``` Task started: 1777410056605 Error: Dirac tried to use attempt_completion without providing a value for 'result'. Retrying... Error: Dirac tried to use attempt_completion without providing a value for 'result'. Retrying... Dirac tried to use attempt_completion without providing a value for 'result'. Retrying... Task Summary: Tokens: 37,419 in, 1,089 out ``` Aborted after 6 consecutive mistakes. Zero file reads, zero useful output. ## Result with `qwen3-32b` (same prompt, same repo, same flags) ``` Repository investigated. It's a minimal 4-file TypeScript project: Files: package.json, src/index.ts, src/util.ts, README.md TypeScript files analyzed: - src/index.ts: exports constant `x = 1` - src/util.ts: exports constant `y = 2` Quality issues identified: 1. No tsconfig.json for TypeScript configuration ... Task Summary: Tokens: 29,575 in, 984 out ``` Completes cleanly. ## Second observation (real repo, separate task) Driving Dirac on a multi-package monorepo (CI-fix task on a dependabot PR) with `qwen36-27b-aeon`: - v1 invocation: hit the same loop on `list_files` missing `paths`. Aborted after the retry budget. 53,296 input / 715 output tokens consumed, no work done. - v2 invocation with the prompt explicitly steering away from `list_files` ("use `execute_command` + `ls` instead") and `--max-consecutive-mistakes 10`: completed end-to-end. 167,664 input / 1,632 output. Single working commit, all CI commands green locally. So the retry loop is reproducible on real workloads, and the workaround (prompt the small model away from the problematic tool) actually works. ## Possible explanation (hypothesis) Dirac's tool docs in the system prompt are large (line-hash protocol + `edit_file` examples). The `Missing value for required parameter '<x>'. Please retry with complete response.` feedback IS injected as a tool error on each retry — verified by reading `dirac-cli/dist/cli.mjs` (`missingToolParameterError`). At 27B, the model appears to not act on that feedback, possibly because: - The original tool docs still dominate attention on resampling. - The repair message is generic; it doesn't include the malformed call or an example of the correct shape. - The error is delivered as a tool result, not as a strong steering directive. This is hypothesis, not diagnosis — would need to instrument the chat history or test multiple repair-message phrasings to confirm.