Skip to content

Instantly share code, notes, and snippets.

@SgtPooki
Created April 28, 2026 21:13
Show Gist options
  • Select an option

  • Save SgtPooki/dae9efdd32e45e1364f7a52819380f78 to your computer and use it in GitHub Desktop.

Select an option

Save SgtPooki/dae9efdd32e45e1364f7a52819380f78 to your computer and use it in GitHub Desktop.
Dirac small-model retry-loop bug — full repro

Dirac retry-loop bug — full repro and logs

Companion to issue dirac-run/dirac#XXX.

Environment

  • dirac-cli 0.3.3, Node v24.11.1, macOS (Darwin 25.3.0)
  • Provider: OpenAI-compatible (vLLM) via --baseurl https://my-vllm-endpoint/v1
  • Failing model: qwen36-27b-aeon (27B, AWQ-quantized Qwen 3, served by vLLM)
  • Passing model: qwen3-32b (32B, served by vLLM)
  • Mode: --act --yolo --reasoning-effort medium

Reproduction

mkdir -p /tmp/dirac-repro/src && cd /tmp/dirac-repro
echo '{"name":"repro","version":"0.0.1"}' > package.json
echo "# repro" > README.md
echo "export const x = 1" > src/index.ts
echo "export const y = 2" > src/util.ts
git init -q && git add -A && git commit -qm init

dirac auth -p openai -k EMPTY -b <vllm-base-url> -m <model>
dirac -y -a -c /tmp/dirac-repro --max-consecutive-mistakes 6 \
  "Investigate this repo. First list all files. Then read each TypeScript file. Then summarize the structure and any quality issues."

Result with qwen36-27b-aeon

Task started: 1777410056605
Error: Dirac tried to use attempt_completion without providing a value for 'result'. Retrying...
Error: Dirac tried to use attempt_completion without providing a value for 'result'. Retrying...
Dirac tried to use attempt_completion without providing a value for 'result'. Retrying...

Task Summary:
Tokens: 37,419 in, 1,089 out

Aborted after 6 consecutive mistakes. Zero file reads, zero useful output.

Result with qwen3-32b (same prompt, same repo, same flags)

Repository investigated. It's a minimal 4-file TypeScript project:
Files: package.json, src/index.ts, src/util.ts, README.md
TypeScript files analyzed:
- src/index.ts: exports constant `x = 1`
- src/util.ts: exports constant `y = 2`
Quality issues identified:
1. No tsconfig.json for TypeScript configuration
...

Task Summary:
Tokens: 29,575 in, 984 out

Completes cleanly.

Second observation (real repo, separate task)

Driving Dirac on a multi-package monorepo (CI-fix task on a dependabot PR) with qwen36-27b-aeon:

  • v1 invocation: hit the same loop on list_files missing paths. Aborted after the retry budget. 53,296 input / 715 output tokens consumed, no work done.
  • v2 invocation with the prompt explicitly steering away from list_files ("use execute_command + ls instead") and --max-consecutive-mistakes 10: completed end-to-end. 167,664 input / 1,632 output. Single working commit, all CI commands green locally.

So the retry loop is reproducible on real workloads, and the workaround (prompt the small model away from the problematic tool) actually works.

Possible explanation (hypothesis)

Dirac's tool docs in the system prompt are large (line-hash protocol + edit_file examples). The Missing value for required parameter '<x>'. Please retry with complete response. feedback IS injected as a tool error on each retry — verified by reading dirac-cli/dist/cli.mjs (missingToolParameterError). At 27B, the model appears to not act on that feedback, possibly because:

  • The original tool docs still dominate attention on resampling.
  • The repair message is generic; it doesn't include the malformed call or an example of the correct shape.
  • The error is delivered as a tool result, not as a strong steering directive.

This is hypothesis, not diagnosis — would need to instrument the chat history or test multiple repair-message phrasings to confirm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment