feat(agent): no-tools-called retry for local Ollama models (complements #210) - #211
Open
mike-011 wants to merge 2 commits into
Open
feat(agent): no-tools-called retry for local Ollama models (complements #210)#211mike-011 wants to merge 2 commits into
mike-011 wants to merge 2 commits into
Conversation
When a local Ollama model (12-14B) returns a final answer without calling any tools, the adapter now retries with an escalating nudge message: Attempt 1: 'You answered without reading wiki files. Call read_file with path summaries/index.md first.' Attempt 2: 'You MUST use the read_file tool. Do NOT answer without reading files first.' Attempt 3: 'IMPORTANT: Your previous answer was not grounded. Call read_file now. Do not answer until you have called a tool.' Detection: _has_tool_calls() checks result.new_items for ToolCallItem. If none found and agent has tools, nudge and retry (up to tool_call_retries). Bounded by tool_call_retries (default 3) — no infinite loops. Shares retry budget with ModelBehaviorError retry. Tests: 73 adapter tests (17 new), 1148 total, 0 failures. Docs: docs/ollama_no_tools_retry.md Complements PR VectifyAI#210 (ollama_chat/ rewrite + ModelBehaviorError retry).
When no-tools-called retries are exhausted, the model output may contain
raw tool-call JSON (e.g. {name: read_file, ...}) or be empty.
Replace these with a clear user-facing message instead.
- Add _sanitize_ungrounded_output() to detect and replace raw JSON/empty
- Apply in both run_with_retry and arun_with_retry exhaustion paths
- Add 7 tests for sanitize behavior (80 adapter tests total)
- Document minimum model requirements based on live testing
Live test results:
qwen2.5-coder:14b — was returning raw JSON, now returns clear message
gemma4:12b — grounded answer (nudge retry works)
qwen3.5:397b-cloud — grounded answer (no retries needed)
mike-011
force-pushed
the
feat/ollama-no-tools-retry
branch
from
July 29, 2026 21:03
3460dde to
06d875e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Complements PR #210 — adds a no-tools-called retry for local Ollama models (12-14B) that return a final answer without calling any tools.
Problem
After PR #210 fixed the
ollama/→ollama_chat/rewrite andModelBehaviorErrorretry, testing with local models revealed a second failure mode:qwen2.5-coder:14b(local):addworks (227s), butqueryreturns "I cannot find relevant information" — the model didn't callread_filegemma4:12b(local):queryreturns empty output""The KB is correctly built (wiki, concepts, entities all present), but the model doesn't initiate tool calls during query — it answers directly without reading wiki files.
This is a model capability limitation, not a transport bug. The adapter cannot force a model to call tools, but it can nudge it.
Fix
After
Runner.run()completes successfully, the adapter checksresult.new_itemsfor anyToolCallItem. If the agent has tools but none were called, the adapter retries with an escalating nudge message:Attempt 1 (gentle):
Attempt 2 (firm):
Attempt 3 (final):
If all retries are exhausted, the adapter returns the last (ungrounded) answer — no infinite loops.
Interaction with PR #210
This PR includes all changes from PR #210 (ollama_chat/ rewrite, ModelBehaviorError retry, parameterized timeouts, drop_params, api_base propagation) plus the no-tools-called retry. The two retry mechanisms share the same
tool_call_retriesbudget.Testing
Files Changed
openkb/agent/ollama_adapter.py_has_tool_calls(),_build_nudge_message(),_append_nudge(), no-tools retry looptests/test_ollama_adapter.pydocs/ollama_no_tools_retry.mdConfiguration
Same
ollama:config block as PR #210:Limitations
"") may not benefit if the empty output is caused by a timeout or inference failure.ModelBehaviorErrorretries.