Skip to content

feat(agent): no-tools-called retry for local Ollama models (complements #210) - #211

Open
mike-011 wants to merge 2 commits into
VectifyAI:mainfrom
mike-011:feat/ollama-no-tools-retry
Open

feat(agent): no-tools-called retry for local Ollama models (complements #210)#211
mike-011 wants to merge 2 commits into
VectifyAI:mainfrom
mike-011:feat/ollama-no-tools-retry

Conversation

@mike-011

Copy link
Copy Markdown

Summary

Complements PR #210 — adds a no-tools-called retry for local Ollama models (12-14B) that return a final answer without calling any tools.

Problem

After PR #210 fixed the ollama/ollama_chat/ rewrite and ModelBehaviorError retry, testing with local models revealed a second failure mode:

  • qwen2.5-coder:14b (local): add works (227s), but query returns "I cannot find relevant information" — the model didn't call read_file
  • gemma4:12b (local): query returns empty output ""

The KB is correctly built (wiki, concepts, entities all present), but the model doesn't initiate tool calls during query — it answers directly without reading wiki files.

This is a model capability limitation, not a transport bug. The adapter cannot force a model to call tools, but it can nudge it.

Fix

After Runner.run() completes successfully, the adapter checks result.new_items for any ToolCallItem. If the agent has tools but none were called, the adapter retries with an escalating nudge message:

Attempt 1 (gentle):

You answered without reading any wiki files. You have tools available. Call read_file with path 'summaries/index.md' first to see what documents exist, then answer the question.

Attempt 2 (firm):

You MUST use the read_file tool. Do NOT answer without reading files first. Start with read_file(path='summaries/index.md'). Available tools: read_file, get_page_content.

Attempt 3 (final):

IMPORTANT: Your previous answer was not grounded in the wiki. Call read_file now. The available tools are: read_file, get_page_content. Do not answer until you have called a tool.

If all retries are exhausted, the adapter returns the last (ungrounded) answer — no infinite loops.

Interaction with PR #210

This PR includes all changes from PR #210 (ollama_chat/ rewrite, ModelBehaviorError retry, parameterized timeouts, drop_params, api_base propagation) plus the no-tools-called retry. The two retry mechanisms share the same tool_call_retries budget.

Testing

  • 73 adapter unit tests (17 new for no-tools retry) — all passing
  • 1148 total tests — all passing (0 regressions)
python -m pytest tests/test_ollama_adapter.py -v  # 73 passed
python -m pytest tests/ -q                          # 1148 passed

Files Changed

File Description
openkb/agent/ollama_adapter.py Added _has_tool_calls(), _build_nudge_message(), _append_nudge(), no-tools retry loop
tests/test_ollama_adapter.py 17 new tests (73 total)
docs/ollama_no_tools_retry.md New — full documentation
(+ all files from PR #210) ollama_chat/ rewrite, compiler, cli, config, docs

Configuration

Same ollama: config block as PR #210:

ollama:
  timeout: 300              # per-request timeout (s), default 300
  tool_call_retries: 3      # max retries (hallucination + no-tools), default 3

Limitations

  • The nudge cannot force a model to call tools — it can only encourage. Very small models (1B) may ignore it.
  • Models that produce empty output ("") may not benefit if the empty output is caused by a timeout or inference failure.
  • The retry budget is shared with ModelBehaviorError retries.

mike-011 added 2 commits July 29, 2026 22:19
When a local Ollama model (12-14B) returns a final answer without calling
any tools, the adapter now retries with an escalating nudge message:

  Attempt 1: 'You answered without reading wiki files. Call read_file
  with path summaries/index.md first.'
  Attempt 2: 'You MUST use the read_file tool. Do NOT answer without
  reading files first.'
  Attempt 3: 'IMPORTANT: Your previous answer was not grounded. Call
  read_file now. Do not answer until you have called a tool.'

Detection: _has_tool_calls() checks result.new_items for ToolCallItem.
If none found and agent has tools, nudge and retry (up to tool_call_retries).

Bounded by tool_call_retries (default 3) — no infinite loops.
Shares retry budget with ModelBehaviorError retry.

Tests: 73 adapter tests (17 new), 1148 total, 0 failures.
Docs: docs/ollama_no_tools_retry.md

Complements PR VectifyAI#210 (ollama_chat/ rewrite + ModelBehaviorError retry).
When no-tools-called retries are exhausted, the model output may contain
raw tool-call JSON (e.g. {name: read_file, ...}) or be empty.
Replace these with a clear user-facing message instead.

- Add _sanitize_ungrounded_output() to detect and replace raw JSON/empty
- Apply in both run_with_retry and arun_with_retry exhaustion paths
- Add 7 tests for sanitize behavior (80 adapter tests total)
- Document minimum model requirements based on live testing

Live test results:
  qwen2.5-coder:14b — was returning raw JSON, now returns clear message
  gemma4:12b — grounded answer (nudge retry works)
  qwen3.5:397b-cloud — grounded answer (no retries needed)
@mike-011
mike-011 force-pushed the feat/ollama-no-tools-retry branch from 3460dde to 06d875e Compare July 29, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant