Skip to content

Add bitemporal, token-efficient recall and reproducible quality evidence - #88

Merged
Coding-Dev-Tools merged 21 commits into
mainfrom
codex/engraphis-quality-evidence
Jul 30, 2026
Merged

Add bitemporal, token-efficient recall and reproducible quality evidence#88
Coding-Dev-Tools merged 21 commits into
mainfrom
codex/engraphis-quality-evidence

Conversation

@Coding-Dev-Tools

@Coding-Dev-Tools Coding-Dev-Tools commented Jul 30, 2026

Copy link
Copy Markdown
Owner

What changed

  • adds true valid_at / known_at bi-temporal reads across memory, graph, support, code, sync, and historical-link evidence while preserving the as_of compatibility alias
  • adds deterministic context packing, hard token budgets, compact recall/MCP responses, usage accounting, privacy-safe token receipts, and packed-only grounded answers
  • adds injected retrieval profiles, calibrated traces, sparse graph retrieval, code-memory bridges, claim-aware lifecycle behavior, and schema v5 temporal history
  • adds engraphis-benchmark/v2, adversarial fixtures, strict artifact validation, fixed-budget curves, bootstrap confidence intervals, LongMemEval-V2 adapter plumbing, canonical export/receipt verification, and release-evidence tooling
  • hardens Pro/Team customer-node behavior, Cloud Sync/checkout UI paths, Railway configuration, dashboard actions, and distribution privacy boundaries
  • publishes reproducible offline quality/token-efficiency evidence in README, now led by a buyer-facing proof band and before/after example while explicitly separating it from official LoCoMo/LongMemEval or leaderboard claims

Why

Engraphis should improve grounded memory quality per context token without rewriting history, leaking evidence across scopes, or making unsupported competitive claims. This change makes those guarantees measurable, reproducible, and fail-closed.

Three final-audit defects were also resolved:

  • changing live memory-link metadata can no longer rewrite historical known_at views
  • synthesized answers that reorder source tokens to reverse meaning fall back to deterministic extractive evidence
  • the sqlite-vec regression now verifies the intended temporal contract: retired vectors remain indexed for historical recall while current retrieval excludes them

Measured offline fixtures

  • CodeMem: Recall@5, Hit@5, and answer-token recall all 1.000 across 26 questions / 44 memories
  • compact MCP recall: 17,172 -> 7,663 serialized response tokens, avoiding 9,509 tokens (55.38% reduction) with unchanged retrieval scores
  • long-document chunking: mean retrieved memory content 808.8 -> 219.0 tokens (72.9% reduction) at Recall@5 1.000
  • long-document chunking: the smallest returned evidence-holding memory 162.2 -> 42.4 tokens (73.9% reduction; about 3.8x smaller) with evidence returned for the same 18 fixture questions
  • packed CodeMem context: a hard 1,500-token budget with observed mean 87.73 and maximum 106 tokens
  • grounded decisions: 10/10 correct (5/5 grounded, 5/5 abstained)

The chunking, compact-response, and budget-accounting measures use different counting boundaries and are not added together. They are deterministic repository fixtures, not official external QA results, model-billing savings, or named-system outperformance claims.

Validation

  • GitHub CI is green: NumPy-only Python 3.9 floor; full Python 3.10, 3.11, and 3.12 matrices; wheel build/install; browser accessibility; Docker build/OCR/dependency audit/health smoke; and CodeQL
  • local full Python suite reached 100%; 2,072 tests collected
  • clean Python 3.11 + sqlite-vec focused run: 6/6 passed
  • Ruff passed
  • focused post-audit regression suite passed
  • local Playwright dashboard/commercial suite: 37/37 passed
  • wheel and sdist built successfully; distribution privacy/content verifier passed
  • four independent review lanes audited history topology, privacy/claims, temporal/grounded correctness, and benchmark tamper resistance

The release-evidence manifest remains a release-time artifact: it requires an actual release tag plus the completed release workflow. Official complete-dataset LoCoMo/LongMemEval/LongMemEval-V2 runs and leaderboard submissions have not been performed.

@Coding-Dev-Tools
Coding-Dev-Tools marked this pull request as ready for review July 30, 2026 07:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8f54b156e6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment thread engraphis/core/store.py
Comment thread engraphis/core/sync.py
Comment thread engraphis/core/engine.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bce39522d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment thread engraphis/core/store.py Outdated
Comment thread engraphis/core/recall.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fedac81339

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/sync.py Outdated
Comment thread engraphis/core/store.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9bc908c4fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment thread engraphis/core/recall.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 501330ddfd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py Outdated
Comment thread engraphis/service.py
Comment thread engraphis/core/context.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be8a9b8cef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread eval/harness.py Outdated
Comment thread engraphis/core/consolidate.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 043c332722

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment thread engraphis/routes/v2_api.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01787cdb67

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/resolve.py Outdated
Comment thread engraphis/core/engine.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac2d99478f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread eval/longmemeval_v2.py Outdated
Comment thread eval/harness.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 632c1d83e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/recall.py Outdated
Comment thread engraphis/core/recall.py Outdated
Comment thread engraphis/core/recall.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3f8858647

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/store.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ff733dd68

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/recall.py Outdated
Comment thread engraphis/core/recall.py Outdated
Comment thread engraphis/core/engine.py
@Coding-Dev-Tools
Coding-Dev-Tools merged commit 19eb85a into main Jul 30, 2026
11 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc776c51bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/recall.py
Comment on lines +347 to +349
identifier,
limit=per_term,
requested_historical=historical,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prioritize exact symbols before capping substring matches

When a repository has more than per_term matching symbols (at most 12) whose names alphabetically precede the exact identifier, Store.search_symbols() applies its ORDER BY name LIMIT before this code can assign the exact-match boost. The exact symbol is therefore absent from symbols, so memories bridged only to that symbol disappear from the code recall arm; request exact name/fqname matches first or rank exact matches before applying this cap.

AGENTS.md reference: AGENTS.md:L94-L100

Useful? React with 👍 / 👎.

Comment thread engraphis/core/recall.py
Comment on lines +707 to +708
sql += " ORDER BY id LIMIT ?"
params.append(max(0, int(limit)))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Rank query-relevant entities before applying the seed cap

When a scope has more than 2,048 entities containing a common query term, the broad OR-substring predicate admits all of them and this ORDER BY id LIMIT keeps only the oldest IDs before the exact entity-pattern check runs. A newer entity whose full name actually occurs in the query can consequently never become a seed, causing both the 1-hop and PPR graph arms to miss its attached memories and relations; prioritize exact/full-name matches before applying the cap.

AGENTS.md reference: AGENTS.md:L94-L100

Useful? React with 👍 / 👎.

@Coding-Dev-Tools
Coding-Dev-Tools deleted the codex/engraphis-quality-evidence branch July 30, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant