feat(agent): Ollama tool-call adapter with parameterized timeouts (fixes #205) - #210
Open
mike-011 wants to merge 5 commits into
Open
feat(agent): Ollama tool-call adapter with parameterized timeouts (fixes #205)#210mike-011 wants to merge 5 commits into
mike-011 wants to merge 5 commits into
Conversation
Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205
Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205
Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205
Add resolve_ollama_settings() to read the 'ollama:' config block:
ollama:
timeout: 300 # per-request timeout (s), default 300
tool_call_retries: 3 # max retries on hallucinated tool names
Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300
Retry precedence: ollama.tool_call_retries > default 3
Changes:
- ollama_adapter.py: resolve_ollama_settings() + all retry/timeout
functions now resolve from config when not explicitly passed
- config.yaml.example: documented ollama: section
- docs/ollama_tool_call_adapter.md: full configuration documentation
- tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings
(56 total, all passing; 1131 total suite, 0 failures)
Issue: VectifyAI#205
mike-011
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #205
When using Ollama models with
openkb queryandopenkb chat, the openai-agents SDK tool-calling loop fails in three ways:ollama/prefix causes LiteLLM to strip tools and inject them into prompt text withformat: json. Model returns tool-call JSON incontentinstead oftool_calls.ModelBehaviorError.Fix
1. Model rewrite:
ollama/toollama_chat/Primary fix — LiteLLM uses native Ollama Chat API with tool support.
2. Retry on hallucination
Catches
ModelBehaviorError, appends corrective message, retries (configurable, default 3).3. Parameterized timeouts and retries
New
ollama:config block inconfig.yaml:4. drop_params + api_base propagation
Testing
See
docs/ollama_tool_call_adapter.mdfor full documentation.