feat(knowledge): opt-in hybrid lexical + vector retrieval for KB search

apps/docs/content/docs/en/integrations/knowledge.mdx

@@ -43,6 +43,7 @@ Search for similar content in a knowledge base using vector similarity
 | `query` | string | No | Search query text \(optional when using tag filters\) |
 | `topK` | number | No | Number of most similar results to return \(1-100\) |
 | `tagFilters` | array | No | Array of tag filters with tagName and tagValue properties |
+| `searchMode` | string | No | Retrieval mode: 'vector' \(default\) uses semantic similarity only, 'hybrid' also runs a full-text leg and fuses both |
 | `rerankerEnabled` | boolean | No | Whether to apply Cohere reranking to vector search results |
 | `rerankerModel` | string | No | Cohere rerank model to use \(one of: rerank-v4.0-pro, rerank-v4.0-fast, rerank-v3.5\) |
 | `rerankerInputCount` | number | No | Number of vector results sent to the Cohere reranker \(1–100\). Defaults to topK × 4 capped at 100. |

apps/docs/content/docs/en/knowledgebase/using-in-workflows.mdx

@@ -37,6 +37,19 @@ In our example, adding `Department equals "Billing"` makes the search consider o
 
 Filters run before the vector comparison, so they make a search both more precise and cheaper. See [Tags and filtering](/knowledgebase/tags) for the full operator list by tag type.
 
+## Retrieval Mode
+
+**Retrieval Mode** is an advanced setting that chooses how matches are found.
+
+| Mode | What it does |
+| --- | --- |
+| Vector only | The default. Ranks purely on meaning, as described above. |
+| Hybrid | Also runs a keyword search over the same chunks and blends the two rankings. |
+
+Semantic search is strong on paraphrase and weak on literal strings: an error code, a ticket key like `PROJ-1234`, a SKU, or a rare product name carries little meaning for the model, so the chunk containing it may not rank near the top. Hybrid adds a keyword pass that matches those tokens exactly, then merges the two lists so a chunk found by either signal can surface.
+
+Turn it on when your documents are full of identifiers, codes, or names people search for verbatim. Leave it off for prose-heavy bases where questions are asked in natural language. Hybrid costs no extra API calls — the keyword pass runs entirely in the database.
+
 ## Rerank Results
 
 **Rerank Results** is an optional second pass. Vector search ranks by raw similarity; reranking re-scores the top matches with a dedicated relevance model (Cohere's rerank models) and reorders them, which sharpens the ordering when the best answer isn't the literal closest vector.
@@ -95,6 +108,7 @@ When the agent's answer is off, the cause is usually in retrieval, not the agent
 - **No results, or wrong documents.** A tag filter may be excluding what you want, or the documents may not be indexed yet. A document is only searchable once its processing status is `completed`; while it is `pending`, `processing`, or `failed`, its chunks won't appear.
 - **Low similarity scores across the board.** The query is too vague, or the information simply isn't in the base. Rewrite the query to match how the documents phrase things.
 - **Right documents, wrong order.** Turn on Rerank Results, or raise Number of Results so the relevant chunk is included.
+- **An exact code, ID, or name isn't found.** Switch Retrieval Mode to Hybrid so a keyword pass runs alongside the semantic one.
 
 See [debugging retrieval](/knowledgebase/debugging-retrieval) for the full diagnostic path, and [chunking strategies](/knowledgebase/chunking-strategies) for how chunk boundaries shape what a search can return.
 

apps/docs/openapi.json

@@ -6040,7 +6040,7 @@
       "post": {
         "operationId": "searchKnowledgeBase",
         "summary": "Search Knowledge Base",
-        "description": "Perform vector similarity search across one or more knowledge bases. Supports semantic search via query text, tag-based filtering, or a combination of both.",
+        "description": "Search across one or more knowledge bases. Supports semantic search via query text, tag-based filtering, or a combination of both. Set `searchMode` to `hybrid` to additionally run a full-text keyword leg and fuse it with the semantic results.",
         "tags": ["Knowledge Bases"],
         "x-codeSamples": [
           {
@@ -6095,14 +6095,21 @@
                     "items": {
                       "$ref": "#/components/schemas/TagFilter"
                     }
+                  },
+                  "searchMode": {
+                    "type": "string",
+                    "enum": ["vector", "hybrid"],
+                    "default": "vector",
+                    "description": "Retrieval strategy. `vector` ranks purely on embedding similarity. `hybrid` also runs a full-text keyword search and fuses the two rankings by reciprocal rank, which retrieves exact tokens — error codes, ticket keys, identifiers, rare product names — that embeddings alone rank poorly. Ignored when only tagFilters are provided."
                   }
                 }
               },
               "example": {
                 "workspaceId": "wsp_abc123",
                 "knowledgeBaseIds": ["d2c8f4a6-1b3e-4c5d-9e7f-8a0b2c4d6e1f"],
                 "query": "How do I reset my password?",
-                "topK": 5
+                "topK": 5,
+                "searchMode": "hybrid"
               }
             }
           }

apps/sim/app/api/knowledge/search/route.test.ts

@@ -21,18 +21,12 @@ import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from 'vites
 
 const {
   mockGetDocumentTagDefinitions,
-  mockHandleTagOnlySearch,
-  mockHandleVectorOnlySearch,
-  mockHandleTagAndVectorSearch,
-  mockGetQueryStrategy,
+  mockExecuteKnowledgeSearch,
   mockGenerateSearchEmbedding,
   mockGetDocumentMetadataByIds,
 } = vi.hoisted(() => ({
   mockGetDocumentTagDefinitions: vi.fn(),
-  mockHandleTagOnlySearch: vi.fn(),
-  mockHandleVectorOnlySearch: vi.fn(),
-  mockHandleTagAndVectorSearch: vi.fn(),
-  mockGetQueryStrategy: vi.fn(),
+  mockExecuteKnowledgeSearch: vi.fn(),
   mockGenerateSearchEmbedding: vi.fn(),
   mockGetDocumentMetadataByIds: vi.fn(),
 }))
@@ -69,10 +63,7 @@ vi.mock('@/lib/knowledge/tags/service', () => ({
 }))
 
 vi.mock('./utils', () => ({
-  handleTagOnlySearch: mockHandleTagOnlySearch,
-  handleVectorOnlySearch: mockHandleVectorOnlySearch,
-  handleTagAndVectorSearch: mockHandleTagAndVectorSearch,
-  getQueryStrategy: mockGetQueryStrategy,
+  executeKnowledgeSearch: mockExecuteKnowledgeSearch,
   generateSearchEmbedding: mockGenerateSearchEmbedding,
   getDocumentMetadataByIds: mockGetDocumentMetadataByIds,
   APIError: class APIError extends Error {
@@ -118,15 +109,7 @@ describe('Knowledge Search API Route', () => {
     resetDbChainMock()
     setEnv({ OPENAI_API_KEY: 'test-api-key' })
 
-    mockHandleTagOnlySearch.mockClear()
-    mockHandleVectorOnlySearch.mockClear()
-    mockHandleTagAndVectorSearch.mockClear()
-    mockGetQueryStrategy.mockClear().mockReturnValue({
-      useParallel: false,
-      distanceThreshold: 1.0,
-      parallelLimit: 15,
-      singleQueryOptimized: true,
-    })
+    mockExecuteKnowledgeSearch.mockClear()
     mockGenerateSearchEmbedding
       .mockClear()
       .mockResolvedValue({ embedding: [0.1, 0.2, 0.3, 0.4, 0.5], isBYOK: false })
@@ -192,7 +175,7 @@ describe('Knowledge Search API Route', () => {
 
       dbChainMockFns.limit.mockResolvedValue([])
 
-      mockHandleVectorOnlySearch.mockResolvedValue(mockSearchResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
 
       mockFetch.mockResolvedValue({
         ok: true,
@@ -212,14 +195,50 @@ describe('Knowledge Search API Route', () => {
       expect(data.data.results[0].similarity).toBe(0.8) // 1 - 0.2
       expect(data.data.query).toBe(validSearchData.query)
       expect(data.data.knowledgeBaseIds).toEqual(['kb-123'])
-      expect(mockHandleVectorOnlySearch).toHaveBeenCalledWith({
+      expect(mockExecuteKnowledgeSearch).toHaveBeenCalledWith({
         knowledgeBaseIds: ['kb-123'],
         topK: 10,
+        searchMode: 'vector',
+        query: validSearchData.query,
         queryVector: JSON.stringify(mockEmbedding),
-        distanceThreshold: expect.any(Number),
+        structuredFilters: undefined,
       })
     })
 
+    it('should forward the hybrid searchMode opt-in to the retrieval layer', async () => {
+      mockGetUserId.mockResolvedValue('user-123')
+
+      mockCheckKnowledgeBaseAccess.mockResolvedValue({
+        hasAccess: true,
+        knowledgeBase: {
+          id: 'kb-123',
+          userId: 'user-123',
+          name: 'Test KB',
+          deletedAt: null,
+        },
+      })
+
+      dbChainMockFns.limit.mockResolvedValue([])
+
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
+
+      mockFetch.mockResolvedValue({
+        ok: true,
+        json: () =>
+          Promise.resolve({
+            data: [{ embedding: mockEmbedding }],
+          }),
+      })
+
+      const req = createMockRequest('POST', { ...validSearchData, searchMode: 'hybrid' })
+      const response = await POST(req)
+
+      expect(response.status).toBe(200)
+      expect(mockExecuteKnowledgeSearch).toHaveBeenCalledWith(
+        expect.objectContaining({ searchMode: 'hybrid' })
+      )
+    })
+
     it('should perform search successfully with multiple knowledge bases', async () => {
       const multiKbData = {
         ...validSearchData,
@@ -239,7 +258,7 @@ describe('Knowledge Search API Route', () => {
 
       dbChainMockFns.limit.mockResolvedValue([])
 
-      mockHandleVectorOnlySearch.mockResolvedValue(mockSearchResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
 
       mockFetch.mockResolvedValue({
         ok: true,
@@ -256,11 +275,13 @@ describe('Knowledge Search API Route', () => {
       expect(response.status).toBe(200)
       expect(data.success).toBe(true)
       expect(data.data.knowledgeBaseIds).toEqual(['kb-123', 'kb-456'])
-      expect(mockHandleVectorOnlySearch).toHaveBeenCalledWith({
+      expect(mockExecuteKnowledgeSearch).toHaveBeenCalledWith({
         knowledgeBaseIds: ['kb-123', 'kb-456'],
         topK: 10,
+        searchMode: 'vector',
+        query: multiKbData.query,
         queryVector: JSON.stringify(mockEmbedding),
-        distanceThreshold: expect.any(Number),
+        structuredFilters: undefined,
       })
     })
 
@@ -284,7 +305,7 @@ describe('Knowledge Search API Route', () => {
 
       dbChainMockFns.limit.mockResolvedValue([])
 
-      mockHandleVectorOnlySearch.mockResolvedValue(mockSearchResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
 
       mockFetch.mockResolvedValue({
         ok: true,
@@ -348,7 +369,7 @@ describe('Knowledge Search API Route', () => {
           embeddingModel: 'text-embedding-3-small',
         },
       })
-      mockHandleVectorOnlySearch.mockResolvedValue(mockSearchResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
       const attribution = encodeURIComponent(
         JSON.stringify({
           actorUserId: 'user-123',
@@ -532,7 +553,7 @@ describe('Knowledge Search API Route', () => {
       mockGetUserId.mockResolvedValue('user-123')
       dbChainMockFns.limit.mockResolvedValueOnce(mockKnowledgeBases)
 
-      mockHandleVectorOnlySearch.mockRejectedValueOnce(new Error('Database error'))
+      mockExecuteKnowledgeSearch.mockRejectedValueOnce(new Error('Database error'))
 
       const req = createMockRequest('POST', validSearchData)
       const response = await POST(req)
@@ -750,7 +771,7 @@ describe('Knowledge Search API Route', () => {
 
       dbChainMockFns.limit.mockResolvedValueOnce(mockTagDefinitions)
 
-      mockHandleTagOnlySearch.mockResolvedValue(mockTaggedResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockTaggedResults)
 
       const req = createMockRequest('POST', tagOnlyData)
       const response = await POST(req)
@@ -763,9 +784,10 @@ describe('Knowledge Search API Route', () => {
       expect(data.data.query).toBe('') // Empty query
       expect(data.data.cost).toBeUndefined() // No cost for tag-only search
       expect(mockGenerateSearchEmbedding).not.toHaveBeenCalled() // No embedding API call
-      expect(mockHandleTagOnlySearch).toHaveBeenCalledWith({
+      expect(mockExecuteKnowledgeSearch).toHaveBeenCalledWith({
         knowledgeBaseIds: ['kb-123'],
         topK: 10,
+        searchMode: 'vector',
         structuredFilters: [
           { tagSlot: 'tag1', fieldType: 'text', operator: 'eq', value: 'api', valueTo: undefined },
         ],
@@ -796,7 +818,7 @@ describe('Knowledge Search API Route', () => {
 
       dbChainMockFns.limit.mockResolvedValueOnce(mockTagDefinitions)
 
-      mockHandleTagAndVectorSearch.mockResolvedValue(mockSearchResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockSearchResults)
 
       mockFetch.mockResolvedValue({
         ok: true,
@@ -816,14 +838,15 @@ describe('Knowledge Search API Route', () => {
       expect(data.data.query).toBe('test search')
       expect(data.data.cost).toBeDefined() // Cost included for vector search
       expect(mockGenerateSearchEmbedding).toHaveBeenCalled() // Embedding API called
-      expect(mockHandleTagAndVectorSearch).toHaveBeenCalledWith({
+      expect(mockExecuteKnowledgeSearch).toHaveBeenCalledWith({
         knowledgeBaseIds: ['kb-123'],
         topK: 10,
+        searchMode: 'vector',
+        query: 'test search',
+        queryVector: JSON.stringify(mockEmbedding),
         structuredFilters: [
           { tagSlot: 'tag1', fieldType: 'text', operator: 'eq', value: 'api', valueTo: undefined },
         ],
-        queryVector: JSON.stringify(mockEmbedding),
-        distanceThreshold: 1, // Single KB uses threshold of 1.0
       })
     })
 
@@ -987,7 +1010,7 @@ describe('Knowledge Search API Route', () => {
 
       mockGetDocumentTagDefinitions.mockResolvedValue(mockTagDefinitions)
 
-      mockHandleTagOnlySearch.mockResolvedValue(mockTaggedResults)
+      mockExecuteKnowledgeSearch.mockResolvedValue(mockTaggedResults)
 
       dbChainMockFns.limit.mockResolvedValueOnce(mockTagDefinitions)
 
@@ -1016,7 +1039,7 @@ describe('Knowledge Search API Route', () => {
         },
       })
 
-      mockHandleVectorOnlySearch.mockResolvedValue([
+      mockExecuteKnowledgeSearch.mockResolvedValue([
         {
           id: 'chunk-1',
           content: 'Content from active document',
@@ -1034,13 +1057,6 @@ describe('Knowledge Search API Route', () => {
         },
       ])
 
-      mockGetQueryStrategy.mockReturnValue({
-        useParallel: false,
-        distanceThreshold: 1.0,
-        parallelLimit: 15,
-        singleQueryOptimized: true,
-      })
-
       mockGenerateSearchEmbedding.mockResolvedValue({ embedding: [0.1, 0.2, 0.3], isBYOK: false })
       mockGetDocumentMetadataByIds.mockResolvedValue({
         'doc-active': {
@@ -1092,7 +1108,7 @@ describe('Knowledge Search API Route', () => {
         { tagSlot: 'tag1', displayName: 'tag1', fieldType: 'text' },
       ])
 
-      mockHandleTagOnlySearch.mockResolvedValue([
+      mockExecuteKnowledgeSearch.mockResolvedValue([
         {
           id: 'chunk-2',
           content: 'Content from active document with tag',
@@ -1110,13 +1126,6 @@ describe('Knowledge Search API Route', () => {
         },
       ])
 
-      mockGetQueryStrategy.mockReturnValue({
-        useParallel: false,
-        distanceThreshold: 1.0,
-        parallelLimit: 15,
-        singleQueryOptimized: true,
-      })
-
       mockGetDocumentMetadataByIds.mockResolvedValue({
         'doc-active-tagged': { filename: 'Active Tagged Document.pdf', sourceUrl: null },
       })
@@ -1164,7 +1173,7 @@ describe('Knowledge Search API Route', () => {
         { tagSlot: 'tag1', displayName: 'tag1', fieldType: 'text' },
       ])
 
-      mockHandleTagAndVectorSearch.mockResolvedValue([
+      mockExecuteKnowledgeSearch.mockResolvedValue([
         {
           id: 'chunk-3',
           content: 'Relevant content from active document',
@@ -1182,13 +1191,6 @@ describe('Knowledge Search API Route', () => {
         },
       ])
 
-      mockGetQueryStrategy.mockReturnValue({
-        useParallel: false,
-        distanceThreshold: 1.0,
-        parallelLimit: 15,
-        singleQueryOptimized: true,
-      })
-
       mockGenerateSearchEmbedding.mockResolvedValue({ embedding: [0.1, 0.2, 0.3], isBYOK: false })
       mockGetDocumentMetadataByIds.mockResolvedValue({
         'doc-active-combined': { filename: 'Active Combined Search.pdf', sourceUrl: null },