Skip to content

feat(tables): surface workspace memory as read-only table - #6145

Open
kerrylu wants to merge 7 commits into
simstudioai:stagingfrom
kerrylu:feat/memory-table
Open

feat(tables): surface workspace memory as read-only table#6145
kerrylu wants to merge 7 commits into
simstudioai:stagingfrom
kerrylu:feat/memory-table

Conversation

@kerrylu

@kerrylu kerrylu commented Aug 1, 2026

Copy link
Copy Markdown

Summary

  • surface workspace-scoped agent memory in Tables, with one read-only row per conversation and the complete transcript
  • support virtual-table filtering, sorting, search, pagination, and synchronous CSV export
  • hide persisted-table mutation, import, lock, and rename controls for virtual tables
  • exclude virtual tables from workflow Table Trigger selection until durable source-level events exist
  • add a concurrent partial index for active workspace memory queries

Type of Change

  • New feature

Testing

  • 7 focused test files passed (76 tests)
  • Sim TypeScript check passed
  • lint and repository audit checks passed
  • API validation and migration safety checks passed

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

@kerrylu is attempting to deploy a commit to the Sim Team on Vercel.

A member of the Team first needs to authorize it.

@cursor

cursor Bot commented Aug 1, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches shared table access, listing, and export paths plus new virtual-table query logic; mutations are blocked server-side, but incorrect virtual routing could affect reads or exports for Memory workspaces.

Overview
Surfaces workspace agent memory in Tables as a synthetic system_memory_* virtual table: one row per conversation with full transcript JSON, listed alongside persisted tables when isVirtual is exposed on list/detail APIs.

Backend: resolveTableById and row/query/find paths dispatch virtual tables to a Memory provider (filter/sort on metadata, transcript filter/sort rejected, byte-budget paging, keyset cursors). Write access to virtual tables returns 423 in shared checkAccess; export adds HEAD preflight and keyset continuation for streamed CSV/JSON; synchronous export stays for virtual tables via shouldUseAsyncTableExport.

UI: Table grid and chrome treat virtual tables as read-only (no add column/import/delete/rename/locks; double-click/keyboard opens expanded viewer; row-query errors toast instead of empty grid). Workflow table trigger selector omits virtual tables in trigger mode.

DB: Concurrent partial index on active memory by workspace + updated_at, id.

Reviewed by Cursor Bugbot for commit 8e45a5b. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Surface workspace-scoped agent memory as a read-only virtual table.

  • Adds bounded transcript hydration with omission metadata for individually oversized transcripts.
  • Supports filtering, sorting, search, cursor pagination, and streaming CSV export for virtual tables.
  • Routes all virtual exports through the synchronous streaming path and preflights download authorization.
  • Hides mutation controls and excludes virtual tables from workflow Table Trigger selection.
  • Adds a concurrent partial database index for active workspace-memory queries.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/lib/virtual-tables/memory-virtual-table.server.ts Implements byte-bounded memory querying, oversized-transcript omission, filtering, sorting, and cursor continuation without leaving the prior pagination failures reachable.
apps/sim/app/api/table/[tableId]/export/route.ts Adds authorization preflight and cursor-driven streaming export that continues across byte-limited virtual pages.
apps/sim/lib/table/capabilities.ts Keeps virtual tables on the supported synchronous streaming exporter instead of persisted-table background jobs.
apps/sim/hooks/queries/tables.ts Starts synchronous exports through direct browser download navigation after a HEAD authorization preflight, avoiding JavaScript response buffering.
apps/sim/app/api/table/utils.ts Resolves virtual tables through the shared resolver and consistently rejects their mutation paths after workspace authorization.
packages/db/migrations/0280_mute_shooting_star.sql Adds the active workspace-memory partial index using the repository’s concurrent migration convention.

Sequence Diagram

sequenceDiagram
  participant UI as Tables UI
  participant API as Table API
  participant Resolver as Virtual Table Resolver
  participant Memory as Memory Provider
  participant DB as PostgreSQL
  UI->>API: Request Memory rows
  API->>Resolver: Resolve synthetic table ID
  Resolver-->>API: Read-only virtual definition
  API->>Memory: Query with filter, sort, and cursor
  Memory->>DB: Fetch bounded candidates and transcripts
  DB-->>Memory: Candidate metadata and transcript data
  Memory-->>API: Bounded rows and continuation cursor
  API-->>UI: Read-only Memory page
  UI->>API: Navigate to streaming CSV export
  loop Until cursor is null
    API->>Memory: Query next bounded page
    Memory-->>API: Rows and continuation cursor
    API-->>UI: Stream CSV chunks
  end
Loading

Reviews (8): Last reviewed commit: "fix(tables): keep oversized memory rows ..." | Re-trigger Greptile

Comment thread apps/sim/lib/virtual-tables/memory-virtual-table.server.ts
Comment thread apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx
Comment thread apps/sim/lib/table/rows/service.ts
@kerrylu
kerrylu force-pushed the feat/memory-table branch from 6f60f8e to d2f0ca3 Compare August 1, 2026 00:58
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile review

Comment thread apps/sim/executor/handlers/agent/memory.ts Outdated
Comment thread apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx
Comment thread apps/sim/lib/table/rows/service.ts
@kerrylu
kerrylu force-pushed the feat/memory-table branch from edfb2a8 to 1bd77e6 Compare August 1, 2026 03:00
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile

@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@cursor review

Comment thread apps/sim/lib/table/capabilities.ts
Comment thread apps/sim/lib/virtual-tables/memory-virtual-table.server.ts
@kerrylu
kerrylu force-pushed the feat/memory-table branch from 1bd77e6 to 576bdc5 Compare August 1, 2026 03:06
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile

@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@cursor review

Comment thread apps/sim/hooks/queries/tables.ts
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@cursor review

@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c31aaf1. Configure here.

…locked tables

The `/api/table` list handler maps each table to an explicit field whitelist
and never emitted `isVirtual`, so every list-surface gate evaluated `undefined`:
`canMutateTable`/`canRenameTable` left Delete, Rename, Import CSV and Move
enabled on the Memory row, and the trigger-mode filter in the table selector
matched nothing. The existing list test mocked a Memory table without the flag
and asserted everything but it, which is why the gap shipped.

Two grid gates were also over-broad on persisted tables. The `json` escape in
the update-lock check is dead for virtual tables (they early-return above it)
and only changed behavior for persisted update-locked ones, contradicting the
comment directly above. Routing `onOpenConfig` through the blocked-add-column
handler made a schema-locked table's column config unopenable and showed
"add column" copy for something that was not an add — opening a column config
is metadata, not schema.

Finally, a Memory transcript that alone exceeds the page byte budget no longer
fails the whole query. This mirrors `fetchRowsBounded` on the persisted path,
which always yields at least one row for a bounded page and reserves the throw
for unbounded queries where a short page would be silent truncation.
`queryMemoryTableRows` is always bounded, so the guard was unreachable by
design; without this, one long conversation made the table unreadable with no
way to page past it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile review

Comment thread apps/sim/app/api/table/[tableId]/export/route.ts
Comment thread apps/sim/lib/virtual-tables/service.server.ts
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile

@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@cursor review

Comment thread apps/sim/lib/virtual-tables/memory-virtual-table.server.ts
@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@greptile

@kerrylu

kerrylu commented Aug 1, 2026

Copy link
Copy Markdown
Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8e45a5b. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant