Skip to content

Add Typesense plugin - #83

Merged
clarkd merged 6 commits into
mainfrom
work/dc/typesense
Jul 31, 2026
Merged

Add Typesense plugin#83
clarkd merged 6 commits into
mainfrom
work/dc/typesense

Conversation

@clarkd

@clarkd clarkd commented Jul 2, 2026

Copy link
Copy Markdown
Member

🔌 Plugin overview

  • Plugin name: Typesense
  • Purpose / problem solved: Query documents from Typesense collections directly in SquaredUp. Runs structured searches, raw multi_search JSON queries, and result-count summaries against a Typesense Cloud (or self-hosted) instance — without ever needing admin access to the cluster.
  • Primary audience: Teams running Typesense-powered search (platform/product teams and SREs) who want to surface search data and monitor collections in SquaredUp.
  • Authentication method(s): API Key — a search-only key (documents:search action) sent in the X-TYPESENSE-API-KEY header.

🖼️ Plugin screenshots

Plugin configuration

To add: screenshot of the Host URL / Search API Key / Collection setup form.

Default dashboards

To add: screenshot of the Overview dashboard (Total Documents + Search Time KPIs, Documents table).


🧪 Testing

Every stream was built and tested against a live, authenticated tenant (region dev) using squaredup test, against a real collection (companies-production-en-us, ~10,633 documents):

  • Config validationq=* search against the configured collection returns 200 OK; confirms host, key, and collection in one probe.
  • documentSearch — returns one row per matching document with dynamic per-collection columns plus a relevance score; verified that unset optional params (query_by/filter_by/sort_by) are correctly dropped, and that relevance populates on real text queries.
  • multiSearch — raw POST /multi_search from a JSON blob; verified the configured collection is auto-injected as a query-arg default, so a minimal {"searches":[{"q":"*"}]} blob queries the right collection with nothing extra.
  • searchSummary — single count row (found / out_of / search_time_ms); verified per_page=0 returns counts with zero hits transferred.
  • squaredup validate --json passes: 4 data streams, icon, UI config, config validation, and default content all present.

⚠️ Known limitations

  • Search-only scope. No cluster health/metrics/stats, no API-key or alias management, and no collection/document import into the graph — a search key can't list collections (collections:list needs an admin key). This is a query-on-demand plugin with no imported objects (hence no perspectives).
  • Result size. A single search returns at most 250 documents per page (per_page); search is designed to return the most relevant results, not to bulk-export a collection.
  • Response size. Very large result sets can exceed SquaredUp's ~6MB per-request limit — keep per_page and field counts reasonable, or narrow with filter_by.
  • No time range. Typesense search has no time-range parameter, so streams return current results with no timeframe picker (timeframes: false). To restrict by time, add a filter_by on a timestamp field.
  • Future scope. Admin-key features (collection import, cluster monitoring) are intentionally deferred to a later version.

📚 Checklist

  • Plugin, datastream and UI naming follow SquaredUp guidelines
  • Logo added
  • One or more dashboards added
  • README added including configuration guidance
  • No secrets or credentials included
  • I agree to the Code of Conduct

Summary by CodeRabbit

  • New Features
    • Added Typesense integration for secure, search-only access to a configured collection.
    • Added document, multi-search, summary, filtering, sorting, and pagination capabilities.
    • Added an Overview dashboard with document counts, search performance, and searchable results.
    • Added HTTPS host, search-only API key, and collection configuration.
    • Added connection validation with clear success and error messages.
    • Added timestamp filtering through filter_by.
  • Documentation
    • Added setup guidance, prerequisites, security requirements, supported searches, and usage limitations.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: b34c8adf-4249-4952-9f36-daa22d3fcaf4

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce1148 and d9af675.

📒 Files selected for processing (2)
  • plugins/Typesense/v1/dataStreams/documentSearch.json
  • plugins/Typesense/v1/docs/README.md

📝 Walkthrough

Walkthrough

Adds a Typesense v1 plugin with search-only configuration, validation, document, multi-search, and summary data streams, response-processing scripts, documentation, and a default Overview dashboard.

Changes

Typesense v1 integration

Layer / File(s) Summary
Plugin registration and configuration
plugins/Typesense/v1/metadata.json, plugins/Typesense/v1/ui.json, plugins/Typesense/v1/docs/README.md
Defines plugin metadata, HTTPS host, search-only API key, collection inputs, validation details, supported streams, and integration limitations.
Search data streams and response processing
plugins/Typesense/v1/configValidation.json, plugins/Typesense/v1/dataStreams/*
Adds collection authentication validation, document search, multi-search, and search summary streams. The scripts flatten hits and expose search relevance, search indexes, and summary fields.
Default dashboard content
plugins/Typesense/v1/defaultContent/*
Adds the Overview dashboard with collection, document-count, search-time, and document-table tiles. Registers the dashboard in the manifest and sets scopes to an explicit empty array.

Sequence Diagram(s)

sequenceDiagram
  participant Dashboard
  participant DataStream
  participant TypesenseAPI
  participant PostRequestScript
  Dashboard->>DataStream: Request search data
  DataStream->>TypesenseAPI: Send configured HTTP request
  TypesenseAPI-->>PostRequestScript: Return search response
  PostRequestScript-->>Dashboard: Return flattened rows or summary metadata
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: adding a new Typesense plugin.
Description check ✅ Passed The description covers the plugin purpose, authentication, testing, limitations, and checklist, but screenshots and Code of Conduct confirmation remain incomplete.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

Comment thread plugins/Typesense/v1/ui.json Outdated
@clarkd
clarkd marked this pull request as ready for review July 21, 2026 14:58
@clarkd
clarkd requested a review from a team July 21, 2026 14:58
@clarkd clarkd added the new-plugin Used to PR newly added plugins label Jul 21, 2026
@clarkd
clarkd requested a review from andrewmumblebee July 21, 2026 14:58

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/Typesense/v1/dataStreams/documentSearch.json`:
- Around line 11-15: Require a non-empty query_by whenever q is not "*", while
preserving wildcard searches without query_by, in the request parameter mappings
for plugins/Typesense/v1/dataStreams/documentSearch.json lines 11-15 and
plugins/Typesense/v1/dataStreams/searchSummary.json lines 11-14; update both
streams consistently so invalid Typesense requests cannot be produced.

In `@plugins/Typesense/v1/dataStreams/scripts/multiSearch.js`:
- Around line 5-6: Update the _search assignment in the multi-search result
mapping to use 1-based numbering by assigning i + 1, so the first search is
displayed as Search `#1` while preserving the existing result mapping.

In `@plugins/Typesense/v1/docs/README.md`:
- Around line 21-33: Update the two setup examples in the README to use bash
language identifiers on their fenced code blocks and add blank lines immediately
before and after each fence. Preserve the existing curl commands and surrounding
setup text.

In `@plugins/Typesense/v1/ui.json`:
- Around line 13-18: Update the URL pattern in the validation configuration to
accept only https:// Typesense host URLs, while preserving the existing
host-only and no-trailing-slash/path constraints. Keep the validation message
aligned with the HTTPS-only requirement.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 30ec68aa-4563-480f-837b-3a4fa3b57e05

📥 Commits

Reviewing files that changed from the base of the PR and between eb6d864 and 16a4f47.

⛔ Files ignored due to path filters (1)
  • plugins/Typesense/v1/icon.svg is excluded by !**/*.svg
📒 Files selected for processing (14)
  • plugins/Typesense/v1/configValidation.json
  • plugins/Typesense/v1/dataStreams/documentSearch.json
  • plugins/Typesense/v1/dataStreams/multiSearch.json
  • plugins/Typesense/v1/dataStreams/scripts/documentSearch.js
  • plugins/Typesense/v1/dataStreams/scripts/multiSearch.js
  • plugins/Typesense/v1/dataStreams/scripts/searchSummary.js
  • plugins/Typesense/v1/dataStreams/searchSummary.json
  • plugins/Typesense/v1/dataStreams/searchValidation.json
  • plugins/Typesense/v1/defaultContent/manifest.json
  • plugins/Typesense/v1/defaultContent/overviewDashboard.dash.json
  • plugins/Typesense/v1/defaultContent/scopes.json
  • plugins/Typesense/v1/docs/README.md
  • plugins/Typesense/v1/metadata.json
  • plugins/Typesense/v1/ui.json

Comment thread plugins/Typesense/v1/dataStreams/documentSearch.json
Comment thread plugins/Typesense/v1/dataStreams/scripts/multiSearch.js Outdated
Comment thread plugins/Typesense/v1/docs/README.md Outdated
Comment thread plugins/Typesense/v1/ui.json
clarkd added 2 commits July 31, 2026 09:44
Clarify wording and UX across the Typesense plugin and fix an off-by-one in multi-search results. Changes:
- multiSearch.json: updated help to refer to the data source's configured collection.
- multiSearch.js: make _search 1-based (i + 1) when flattening hits.
- searchSummary.json: added help text for query_by field.
- docs/README.md: formatting fixes, clearer Host URL guidance (must be https://) and curl blocks marked as bash; clarify "data source" vs "connection".
- ui.json: tightened validation regex to require https:// (rejects plaintext HTTP).
Reasons: improve clarity/consistency, avoid plaintext API keys over HTTP, and correct result sub-search numbering.
@clarkd clarkd changed the title Typesense v1 Add Typesense plugin Jul 31, 2026
@clarkd

clarkd commented Jul 31, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@clarkd

clarkd commented Jul 31, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/Typesense/v1/dataStreams/documentSearch.json`:
- Around line 44-48: Update the per_page field in documentSearch to validate
integer values and constrain them to the documented range by adding min: 0 and
max: 250; use min: 1 instead if the stream must always return at least one
document.

In `@plugins/Typesense/v1/dataStreams/scripts/multiSearch.js`:
- Around line 5-6: Update the result transformation around data.results to
support union responses: use data.results when it is non-empty, otherwise wrap
top-level data.hits as a single result before flatMap. Preserve each hit’s
document and text_match mapping, and avoid assigning union hits _search: 1
unless the existing contract explicitly defines that meaning.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0446cd62-e9c5-47d8-a212-2e57382b5703

📥 Commits

Reviewing files that changed from the base of the PR and between 22a115d and 646b017.

⛔ Files ignored due to path filters (1)
  • plugins/Typesense/v1/icon.svg is excluded by !**/*.svg
📒 Files selected for processing (14)
  • plugins/Typesense/v1/configValidation.json
  • plugins/Typesense/v1/dataStreams/documentSearch.json
  • plugins/Typesense/v1/dataStreams/multiSearch.json
  • plugins/Typesense/v1/dataStreams/scripts/documentSearch.js
  • plugins/Typesense/v1/dataStreams/scripts/multiSearch.js
  • plugins/Typesense/v1/dataStreams/scripts/searchSummary.js
  • plugins/Typesense/v1/dataStreams/searchSummary.json
  • plugins/Typesense/v1/dataStreams/searchValidation.json
  • plugins/Typesense/v1/defaultContent/manifest.json
  • plugins/Typesense/v1/defaultContent/overviewDashboard.dash.json
  • plugins/Typesense/v1/defaultContent/scopes.json
  • plugins/Typesense/v1/docs/README.md
  • plugins/Typesense/v1/metadata.json
  • plugins/Typesense/v1/ui.json

Comment thread plugins/Typesense/v1/dataStreams/documentSearch.json Outdated
Comment thread plugins/Typesense/v1/dataStreams/scripts/multiSearch.js Outdated
Add per_page validation (min 1, max 250) and clarify its limit. Improve multi_search support: update help and README to document `"union": true` merged results and that merged rows have no Search #. Update multiSearch.js to detect federated (results array) vs merged (single object) responses, flatten hits, set _search to sub-search number for federated rows or null for union rows, and preserve relevance in _relevance.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/Typesense/v1/dataStreams/documentSearch.json`:
- Line 48: Update the help text associated with the per_page validation in the
Typesense document search configuration to describe 250 as this plugin’s
supported maximum, using wording that indicates up to 250 results per page
rather than attributing the limit to Typesense.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7c51a220-230d-4e27-91fc-8479cbd77425

📥 Commits

Reviewing files that changed from the base of the PR and between 646b017 and 7ce1148.

📒 Files selected for processing (4)
  • plugins/Typesense/v1/dataStreams/documentSearch.json
  • plugins/Typesense/v1/dataStreams/multiSearch.json
  • plugins/Typesense/v1/dataStreams/scripts/multiSearch.js
  • plugins/Typesense/v1/docs/README.md

Comment thread plugins/Typesense/v1/dataStreams/documentSearch.json Outdated
Update help text for the documentSearch per_page parameter and expand the README's result-size note. The data stream help now reads: 'This plugin supports up to 250 results per page'. The docs clarify the plugin caps per_page at 250 (matching Typesense's default --max-per-page); self-hosted servers may raise the server limit, but the plugin still enforces 250. Emphasises searches are for relevance rather than bulk-exporting collections.
@github-actions

Copy link
Copy Markdown

🧩 Plugin PR Summary

📦 Modified Plugins

  • plugins/Typesense/v1

📋 Results

Step Status
Validation ✅ Passed
Deployment 🚀 Deployed

🔍 Validation Details

typesense
{
  "valid": true,
  "pluginName": "typesense",
  "pluginType": "hybrid",
  "summary": {
    "Data Streams": 4,
    "Import Definitions": 0,
    "Correlation Rules": 0,
    "UI Configuration": true,
    "Has Icon": true,
    "Has Default Content": true,
    "Config Validation": true,
    "Custom Types": false
  }
}

Comment thread plugins/Typesense/v1/docs/README.md
@clarkd
clarkd merged commit 55b2cfc into main Jul 31, 2026
1 check passed
@clarkd
clarkd deleted the work/dc/typesense branch July 31, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-plugin Used to PR newly added plugins

Development

Successfully merging this pull request may close these issues.

2 participants