From a94781701ea225796af9fa873e070d8f11175a61 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 30 Jul 2026 22:19:58 +0800 Subject: [PATCH 1/3] feat(pm): refresh the package index on a resolution miss, not on a timer (2026.7.30.3, fixes #315) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mcpp build/run/test` refreshed the index whenever its marker was older than an hour — a multi-repo `xlings update` (with 3 retries and 2s/4s backoff) run whether or not anything was missing. On a slow or blocked network that is minutes of waiting, once an hour, for data already on disk. The offline-first policy was not missing: xlings.cppm's xim install gate has had it for a while, and its comment names this exact symptom. prepare.cppm's TTL gate simply fired first, making that policy unreachable. The same decision was being derived in five places, two of which contradicted each other. One source of truth now: mcpp.pm.index_refresh. build / add / the xim gate / the install-failure retry all route through it. A refresh happens when there is no local index, when a descriptor is missing from it, or when a SemVer constraint matches none of the versions it knows — never merely because time passed. Builds whose dependencies resolve locally make no network request. Why the axis changed: "is the index fresh enough" is unanswerable and mtime is a poor proxy (restored CI caches, clock skew, tars that preserve timestamps all make the marker lie in both directions). "Can the resolver work with what is on disk" IS answerable offline — every resolution input is a local file. The marker is now only a debounce timer. The load-bearing rule (SuppressedInconclusive): a miss means nothing unless the index that would have answered is authoritative. xim descriptors declare no namespace, so (xim, x) can never match the identity gate — counting that as a miss would refresh on EVERY build with a toolchain-ish dependency, strictly worse than the TTL being removed. Judgement reused from IndexRoute (#307), locked by a unit test and by e2e 173 step 3. Also in this change: - `mcpp update` stops being a no-op. It only dropped lock entries and told the user to run `mcpp build`, but the build path never reads mcpp.lock, so it changed nothing at all. It now forces an index refresh (explicit intent: no TTL, no debounce) and reports the revision change; skipped when nothing in the project is served by the shared registry. - `--offline` / MCPP_OFFLINE: no index refresh, no downloads, no toolchain auto-install. Checked at the point of download, so an offline build with its dependencies present still succeeds. MCPP_NO_AUTO_INSTALL stays accepted as the older, narrower spelling — one concept had three names. - `[index] auto_refresh` in the global config. Deliberately a boolean, not a three-valued mode: keeping the old TTL path alive to emulate a policy being deleted is how this debt accumulated. Policy lives in the global config, not mcpp.toml — it describes the machine's network, and a project carrying it would stop being portable between a LAN, a laptop and CI. - A marker stamped in the future read as "fresh" forever (age < ttl is true for negatives). Unusable timestamps now mean unknown, and unknown means stale. - The index sync had no concurrency guard while the BMI cache has had one for a while. Non-blocking: whoever holds the lock is already doing the work, and a queue of stalled builds is the symptom this change exists to remove. - mark_known_indexes_refreshed bailed out under a project-scoped env, so machines with custom [indices] never recorded a refresh at all. - `mcpp index status` grows a revision column. .xlings-index-version is the index's content identity (artifact names carry it: mcpp-index-8d67478.tar.gz) and mcpp had zero references to it. Treated as an opaque string — probing showed sub-indexes carry a date version, not a sha. - Resolution failures now carry the index revision and age plus the explicit `mcpp index update` hint, so "no such package" and "your index is from last month" stay distinguishable once refreshes are lazy. Semantic change, documented in docs/05-mcpp-toml.md: `^1.2` resolves against the versions your local index knows. A 1.3.0 published upstream since your last refresh needs `mcpp index update` or `mcpp update` — which is what those commands are for. Design + probe results: .agents/docs/2026-07-30-issue315-index-refresh-policy-design.md --- ...2026-07-30-issue315-implementation-plan.md | 146 ++++++ ...30-issue315-index-refresh-policy-design.md | 423 ++++++++++++++++++ CHANGELOG.md | 36 ++ docs/05-mcpp-toml.md | 32 ++ docs/zh/05-mcpp-toml.md | 29 ++ mcpp.toml | 2 +- src/build/prepare.cppm | 125 ++++-- src/cli.cppm | 11 + src/config.cppm | 19 + src/platform/env.cppm | 33 ++ src/pm/commands.cppm | 55 ++- src/pm/index_management.cppm | 36 +- src/pm/index_refresh.cppm | 354 +++++++++++++++ src/pm/package_fetcher.cppm | 12 + src/toolchain/fingerprint.cppm | 2 +- src/xlings.cppm | 105 ++++- tests/e2e/173_index_refresh_policy.sh | 162 +++++++ tests/unit/test_pm_index_refresh.cpp | 336 ++++++++++++++ tests/unit/test_xlings.cpp | 67 +++ 19 files changed, 1919 insertions(+), 66 deletions(-) create mode 100644 .agents/docs/2026-07-30-issue315-implementation-plan.md create mode 100644 .agents/docs/2026-07-30-issue315-index-refresh-policy-design.md create mode 100644 src/pm/index_refresh.cppm create mode 100755 tests/e2e/173_index_refresh_policy.sh create mode 100644 tests/unit/test_pm_index_refresh.cpp diff --git a/.agents/docs/2026-07-30-issue315-implementation-plan.md b/.agents/docs/2026-07-30-issue315-implementation-plan.md new file mode 100644 index 00000000..039712d1 --- /dev/null +++ b/.agents/docs/2026-07-30-issue315-implementation-plan.md @@ -0,0 +1,146 @@ +# 索引刷新策略收敛 — 实施计划 + +日期：2026-07-30 +设计：`.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md` +关联：#315 +目标版本：**2026.7.30.3**（基线 `d79ab00` / 2026.7.30.2） +形态：**单 PR**，内部按 A–F 六段推进，每段自带验收。 + +--- + +## 0. 基线核对（已完成） + +| 事实 | 位置（@ d79ab00） | +|---|---| +| TTL 站点 | `src/build/prepare.cppm:1390-1433` | +| 每个依赖已自带**有序候选** | `DependencySpec.candidates`（`src/pm/dep_spec.cppm:46`）—— 与 `mcpp add` 用的是同一份 | +| 存在性查找 | `mcpp::pm::lookup_descriptor(route, candidates)`（`src/pm/index_route.cppm:150`） | +| route 工厂 | `prepare.cppm:1548`（`index_route(cfg)`）、`commands.cppm:151-152` | +| 版本可满足性 | `mcpp::pm::resolve_semver(ns, short, req, route, platform)`（`src/pm/resolver.cppm:96`），失败即不可满足 | +| 索引目录/marker/age | `xlings.cppm:381-490`（**匿名命名空间，未导出**） | +| 索引状态（已导出） | `default_index_status` / `official_index_status` → `IndexStatus{dir,present,fresh,ageSeconds}` | +| 索引内容 rev | `/.xlings-index-version`（实测 mcpplibs=`8d67478`，xim-pkgindex=`ebf4020`）—— **mcpp 零引用** | +| 跨平台锁原语 | `mcpp::platform::fs::FileLock::try_acquire(dir)`（`src/platform/fs.cppm:60`，`bmi_cache.cppm:241` 已在用） | +| 索引数据根（已导出） | `mcpp::xlings::paths::index_data(env)` | + +--- + +## A. 探针（先跑，不写代码） + +- **P-1 `.xlings-index-version` 更新语义**：clean-room `MCPP_HOME`，走一次真实 `xlings update`， + 比对刷新前后该文件的值与格式（是否 7 位短 sha、有无尾随空白、artifact/git 两通道是否都写）。 + **降级预案**：若某通道不写 → `index_revision()` 返回 nullopt，advisory 只显示 age，**gate 不受影响**。 +- **P-2 INV-3 误报实测**：对一个同时含 mcpplibs 依赖与 xim 依赖的工程，打印每个依赖的 + `RefreshDecision`，确认 xim 落 `SuppressedInconclusive` 而非 `DescriptorMiss`。 + P-2 依赖 A 段代码，故实际在 B 段末尾以单测 + 一次真实工程 `-v` 运行完成。 + +**验收**：P-1 结论写回设计文档 §10；P-2 由单测 + e2e #6 承接。 + +--- + +## B. 策略模块（新增 `src/pm/index_refresh.cppm`） + +**导出面** + +``` +enum class RefreshReason { None, IndexAbsent, DescriptorMiss, VersionMiss, + SuppressedDebounce, SuppressedOffline, SuppressedInconclusive }; +struct RefreshPolicy { bool offline; std::int64_t debounceSeconds = 120; }; +struct RefreshDecision { bool shouldRefresh; RefreshReason reason; std::string subject; }; + +RefreshDecision decide_for_dependency(const IndexRoute&, const DependencySpec&, + const xlings::Env&, const platform::PlatformKey&, + const RefreshPolicy&); +std::expected apply(const RefreshDecision&, const xlings::Env&, bool quiet); +RefreshPolicy policy_from_env(const config::GlobalConfig&, bool offlineFlag); +std::string_view reason_text(RefreshReason); +``` + +**判据顺序（短路，单测逐行锁）** + +1. `spec.isPath() || spec.isGit()` → `None` +2. `policy.offline` → `SuppressedOffline` +3. 无候选路由到**内置 registry**（`find_for_ns` 为 null 或 `is_builtin`）→ `None` + （项目 `path`/自定义 git 索引：刷全局索引对它无用，保持现状语义） +4. `lookup_descriptor` 未命中且 `!conclusive` → `SuppressedInconclusive`（**INV-3**） +5. 默认索引 `pkgs/` 不存在 → `IndexAbsent` → 刷 +6. 未命中 → `DescriptorMiss` → 刷 +7. 命中 + `is_version_constraint(spec.version)` + `resolve_semver` 失败 → `VersionMiss` → 刷 +8. 上面判刷但 `0 <= age < debounceSeconds` → `SuppressedDebounce` + +**依赖方向**：`index_refresh` → {`index_route`, `resolver`, `dep_spec`, `xlings`, `config`, +`platform.fs`, `ui`, `log`}。`resolver` 已 import `index_route`，无环。 + +**验收**：`tests/unit/test_pm_index_refresh.cpp` 覆盖 8 条判据 + 精确版本不判 VersionMiss。 + +--- + +## C. xlings 侧最小增补（`src/xlings.cppm`） + +1. `IndexStatus` 增字段 `std::optional rev`（读 `.xlings-index-version`，trim 空白； + 缺失/空 → nullopt）。→ `mcpp index status` 与 advisory 免费获得。 +2. 新导出 `std::optional index_revision(const std::filesystem::path& indexDir)`。 +3. **S1**：`is_index_dir_fresh` 中 `age < 0` → 判 **stale**（未来时间戳不得等于永远新鲜）。 +4. **S3**：`update_index` 内部用 `FileLock::try_acquire(paths::index_data(env))` 包裹； + 拿不到锁 → 记 verbose 日志并**返回 0（跳过，不报错、不阻塞）**。 +5. **S4**：`mark_known_indexes_refreshed` 去掉 `projectDir` 早退，改为项目模式下也打标 + （仅供 advisory）。 + +**验收**：`tests/unit/test_xlings.cpp` 增负 age、rev 读取三态（缺失/空/带换行）。 + +--- + +## D. 五站点收敛 + +| 站点 | 动作 | +|---|---| +| `prepare.cppm:1390-1433` | 整段替换：遍历 `m->dependencies` 调 `decide_for_dependency`；命中即 `apply` 一次（进程内 `bool` once），reason 进 `-v` 日志 | +| `commands.cppm:167`（add） | 改为 `decide_*` + `apply`，删除本地 TTL 判断 | +| `xlings.cppm:1400`（xim 门） | 保留判据，`kJustRefreshedSeconds` 改由 `RefreshPolicy::debounceSeconds` 提供默认值，注释指向策略模块 | +| `package_fetcher.cppm:1060` | 保留，输出走统一句式 | +| `index_management.cppm:29`（search） | 保留 TTL，**加注释说明这是唯一允许时间驱动的站点** | + +**验收**：`grep -n "is_index_fresh" src/` 只剩 search 一处 + xlings 内部实现。 + +--- + +## E. D6 + 表达面 + +1. **`mcpp update` 不再空转**（`commands.cppm:403-430`）：先强制刷索引（不看 TTL/debounce， + `offline` 下拒绝并说明），再清 lock 条目，输出 `mcpplibs 8d67478 → a1b2c3d` 或 `already at …`。 +2. **`--offline`**：`build` / `run` / `test` / `update` 注册；语义 = 本次调用不发起任何网络 + （索引 + 包 + 工具链自动安装）。 +3. **`MCPP_OFFLINE=1`**：等价 env；`MCPP_NO_AUTO_INSTALL` 保留为兼容别名（等价于 offline 的 + 工具链子集），help/docs 标注 deprecated。 +4. **`[index] auto_refresh = true|false`**：`config.cppm` 读取，默认 true。 +5. **可解释输出**：刷新一行说明主语与原因；`-v` 打印跳过原因与索引 rev/age； + offline+miss 的结构化错误含 rev + age + `mcpp index update`。 +6. `mcpp index status` 增 rev 列。 + +**优先级**：flag > env > config。 + +--- + +## F. 测试 + 版本 + 交付 + +- 新 e2e `tests/e2e/173_index_refresh_policy.sh`，9 个场景（设计 §8）。 + **反向断言写法**：`out=$(...); echo "$out" | grep -q X && { echo FAIL; exit 1; }` + （`! cmd | grep` 在 errexit 下被豁免，永不失败）。 +- 全量本地回归：`mcpp test`（单测）+ 受影响 e2e（09/14/44/105/169 等涉及索引/离线的用例）。 +- 版本 `2026.7.30.3`：只改 `mcpp.toml` + `src/toolchain/fingerprint.cppm`； + **bootstrap pin（`.xlings.json` / `ci-fresh-install.yml MCPP_PIN`）不动**。 + `bash .github/tools/check_version_pins.sh` 必须过。 +- 单 PR → CI 全绿 → bypass squash 合入。 +- 合入后：release 四平台 → 镜像 xlings-res（gh+gtc）→ xim-pkgindex PR → `xlings install mcpp` + 真装验证 → bootstrap pin bump（独立 commit）。 + +--- + +## 风险与回退 + +| 风险 | 处置 | +|---|---| +| INV-3 接错 → 每次构建都刷 | 单测 + e2e #6 双闸；P-2 真实工程实测 | +| `.xlings-index-version` 语义不符 | gate 不依赖 rev，最坏只损失 advisory 精度 | +| S3 锁在 Windows 行为不一致 | 走 `platform/fs.cppm` 封装；拿不到锁=跳过而非阻塞 | +| 语义变化引发「拉不到新版本」误报 | `mcpp update` 变成真入口 + advisory 提示 + docs 明写 | diff --git a/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md b/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md new file mode 100644 index 00000000..f91ff58f --- /dev/null +++ b/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md @@ -0,0 +1,423 @@ +# 索引刷新策略收敛：从「时间驱动」到「解析驱动」— 设计 + +日期：2026-07-30 +状态：**已实施**（2026.7.30.3）。§10 的两条探针已跑，结果与实现修正记在那一节 +关联：#315（`mcpp build/test/run` 手动刷新包索引而不是自动刷新） +建议目标版本：**2026.7.31.1** + +--- + +## 1. 摘要 + +issue #315 报的症状真实：网络差时 `mcpp build` 会停在 `Updating package index (auto-refresh)`。 + +但这不是「功能缺失」。offline-first / miss-triggered 的刷新策略**已经在仓库里了**，而且写得比 +issue 建议的更完整（`src/xlings.cppm:1400-1431`，含 120s 抖动抑制），其注释甚至点名了这个 +issue 描述的症状（Termux first-run / build hang）。真正的缺陷是： + +| 编号 | 缺陷 | 层级 | +|---|---|---| +| **D1** | 同一个决策（要不要刷索引）在 **5 处各推导一遍**，其中 `prepare.cppm:1300-1311` 推出的策略与 `xlings.cppm:1405` 写死的策略**互相矛盾** | 策略 | +| **D2** | 刷新判据是 **marker mtime（时间轴）**，而「索引能否回答问题」是内容属性。**而且索引自带的内容 revision（`.xlings-index-version`）躺在磁盘上，mcpp 零引用** | 状态/身份 | +| **D3** | `is_index_dir_fresh` 对**未来时间戳**判为「永远新鲜」（`age < 0 < ttl`）；时钟回拨 / tar 保留 mtime / CI cache 恢复都能触发 | 稳定性 | +| **D4** | 无 `--offline` / `MCPP_OFFLINE` 逃生舱；`MCPP_NO_AUTO_INSTALL` 只管**工具链**，不管索引也不管包 | UX | +| **D5** | 刷与不刷的**原因**从不可见；只有一行没有主语的 `Updating package index` | UX | +| **D6** | `mcpp update` 是**空操作**：删 mcpp.lock 条目后叫用户去 `mcpp build`，而 lock 从来没被 build 读过 → 行为影响为零 | 语义 | + +D1/D2 是根。本设计做 D1–D6 的收敛，**不做** 「让 `mcpp.lock` 成为构建期解析输入」（§9，必须单独立项）。 + +--- + +## 2. 现状机制（逐处核过） + +### 2.1 五个刷新站点，四种策略 + +| # | 站点 | 判据 | 评价 | +|---|---|---|---| +| 1 | `mcpp add`
`pm/commands.cppm:160-175` | 先 `lookup_descriptor` 查本地 → **确凿未命中** + 涉及内置 registry + TTL 过期 → 刷一次 → 重查 | 正确样板 | +| 2 | xim 安装前置门
`xlings.cppm:1400` | **纯 miss**：包文件本地缺失才刷；120s 内刚刷过则不重刷 | 最完整 | +| 3 | 安装失败重试
`package_fetcher.cppm:1060-1075` | **版本 miss**：install 失败 → 刷一次 → 重试 | 补 #2 的漏（文件在、版本旧） | +| 4 | **`mcpp build/run/test`**
`prepare.cppm:1300-1311` | **纯 TTL**：根 manifest 有任一依赖路由到内置索引 + marker 超 `searchTtlSeconds`（默认 3600s） | ← 本 issue | +| 5 | `mcpp search`
`index_management.cppm:29` | TTL | 合理，search 语义即查上游 | + +站点 4 已有两道门（`!m->dependencies.empty()` 与 `usesBuiltinIndex(spec)`），所以「无条件刷新」 +的说法不成立；但第三道判据是纯时间，只要有**一个** registry 依赖，门就形同不设。 + +### 2.2 为什么这次网络往返在稳态下是纯浪费 + +**所有解析输入都是本地文件**： + +- `resolve_semver`（`pm/resolver.cppm:96`）经 `IndexRoute::read()` 读 xpkg.lua，从描述符解析可用 + 版本集 —— 纯本地文件解析，本身不联网。 +- 包已装时 `fallback::is_install_complete(verdir)` 直接早退（`package_fetcher.cppm:990`）。 + +即「本地能否回答」是**不需要网络、确定性、可单测**的判定。站点 4 却用一个与之无关的信号 +（marker 有多老）决定是否付网络代价。 + +### 2.3 代价被两个细节放大 + +- `is_index_fresh` 查的是 **default（mcpplibs）索引**的 marker，但 `update_index` 执行的是 + `xlings update` —— **同步所有索引仓库**。一个 marker 过期换一次全量同步。 +- `update_index` 有 **3 次尝试 + 2s/4s 线性退避**（`xlings.cppm:1365-1378`）。网络差时不是等 + 一次超时，是三次。 + +### 2.4 分层体检：真正的缺陷不全在策略层 + +| 层 | 事实 | 结论 | +|---|---|---| +| **L1 传输** | `update_index` = `xlings update`，刷**全部**索引仓库，mcpp 侧无 per-index / per-package 能力 | 粒度不可选 | +| L1 | **`mcpp index update ` 的 `name` 是空承诺**：全局段无条件全量刷（`index_management.cppm:121-124` 硬编码 `"all index repos"`），filterName 只在项目自定义索引循环里过滤，而循环体 `break` + `ensure_project_index_dir` 一次处理全部 | CLI help 承诺了实现做不到的事 | +| L1 | **索引刷新无任何并发锁**。`bmi_cache` 有 flock，`platform/fs.cppm` 已提供跨平台原语（flock / LockFileEx），索引一个都没用 | 并行 CI job 共享 MCPP_HOME、多终端并行构建 → 并发同步 + 并发写 marker | +| **L2 状态** | 唯一新鲜度信号是 `.mcpp-index-updated` 的 **mtime** | 信号选错轴（D2） | +| L2 | **索引自带内容 revision**：`/.xlings-index-version`（实测 `mcpplibs`→`8d67478`，`xim-pkgindex`→`ebf4020`），**mcpp 全仓库零引用** | 内容轴的载体现成，见 §3 INV-2 | +| L2 | `LockedIndex.rev` 只是把用户在 `[indices]` 写的 pin 抄进 lock（`prepare.cppm:3841`），**不是观测值**；内置索引连 rev 概念都没有 | 无内容寻址的索引身份 | +| L2 | **派生缓存烙绝对路径**：`.xlings-index-cache.json` 每条都带 `"path":"<绝对路径>"`。现场证据：本机 `~/.mcpp` 那份的 path 全部指向另一个早先会话的临时 MCPP_HOME | 目录一搬（CI cache 恢复 / MCPP_HOME 迁移 / AUR `/opt` 布局）缓存即坏 | +| L2 | **为上一条写的检测器没接线**：`official_index_cache_matches_package_file` 只被 `is_official_package_index_fresh` 调用，而后者在 production 代码里**零调用**，只有 `tests/unit/test_xlings.cpp` 在用；跑在构建路径上的 `ensure_official_package_index_fresh` 只查 `exists(pkg)` | 检测器写了、测了，没装到产线上 | +| L2 | `mark_known_indexes_refreshed`（`xlings.cppm:444`）在 `env.projectDir` 非空时整段早退 | 项目 env 的刷新永不打标 | +| **L3 策略** | D1 | 见 §3 INV-1 | +| **L4 表达** | `index update/status/pin/unpin` 齐全，`status` 是真 offline（设计正确） | 骨架对 | +| L4 | **`mcpp update` 空操作**（D6，`pm/commands.cppm:403-430`） | 唯一语义为「更新依赖」的命令什么都不做 | + +**一处过时论断的更正**：先前评估称「`xlings update` 是 git-only，Windows no-git 死锁（PR #114） +每小时埋进构建关键路径」。**已过时** —— `IndexSpec` 已有 `source = "auto" | "artifact" | "git"` +与 `artifact` base URL（`config.cppm:40-41`，#269 / xlings ≥ 0.4.68），索引以 **artifact 分发** +为主；本机 `mcpplibs/` 与 `xim-pkgindex/` **都没有 `.git` 目录**，证实了这一点。风险仍在 +（`auto` 会回落 git），但性质从「必经」降为「可能」，Windows 论点相应下调。 + +--- + +## 3. 架构判断（四条不变量） + +### INV-1 刷新策略只有一个真源 + +D1 是典型的「同一决策两处推导」——本仓库已为这种债付过两次代价（feature 请求集解析×激活两处 +不自洽 #242；`build_program` 重推 musl→static 未复用 `prepare`）。加语义时它必然变成构建失败。 +故新增单一策略模块，5 个站点全部改为调用它，**任何站点不得再自行判断 TTL**。 + +### INV-2 gate 用内容，且直接采用已有的 `.xlings-index-version` + +刷新的**必要性**由「解析器能否用本地索引回答」决定；索引的**身份**取 `.xlings-index-version` +（7 位 rev），marker mtime 降级为纯 debounce 计时器。 + +这条同时消解一个既有故障模式：**marker 假新鲜导致上游有新版本却拉不到**（llvm 22.1.8 那次）。 +假新鲜在新模型下不再能压制一次必要的刷新，只能压制一句提示。方向与 #311/#317 的教训一致： +**mtime 不可靠，判等走内容/键。** + +关键是**不需要新建机制** —— rev 文件已存在，读一个 7 字节文件即可。它一次性支撑四件事： +新鲜度显示、`mcpp index status` 的 rev 列、未来 lock 记录**观测到的** index rev、 +「为什么解析结果变了」的诊断。（前置条件见 §10 探针 P-1。） + +### INV-3 不可证伪就不刷 + +`IndexRoute::authoritative_for()`（PR#307）已定义「miss 何时能当作不存在的证明」：lazy git 索引 +未克隆、第三方 namespace、**xim 描述符不写 `namespace` 故 `(xim, x)` 永不匹配身份门** —— 这三类 +miss 不可证伪。 + +**这是本方案最容易踩的坑**：若把「查不到」直接当「要刷」，则任何带 xim 依赖的项目会**每次构建都 +判 miss、每次都刷**，比现状更糟。必须复用 `authoritative_for`：不可证伪 → fall-through，**不刷**。 + +### INV-4 策略是环境属性，不是项目属性 + +刷新策略归**全局 config**（`~/.mcpp/config.toml`），不进 `mcpp.toml`。同一项目在公司内网、家里、 +CI 上该允许不同策略；写进 manifest 会让项目不可移植，还会无谓拓宽 lock/指纹面。 + +--- + +## 4. 目标设计 + +### 4.1 新模块 `src/pm/index_refresh.cppm`（module `mcpp.pm.index_refresh`） + +依赖方向：`index_refresh` → {`index_route`, `resolver`, `xlings`, `config`}。`resolver` 已 import +`index_route`，无环。 + +```cpp +export namespace mcpp::pm { + +enum class RefreshReason { + None, // 本地可解析 —— 零网络 + IndexAbsent, // 索引 pkgs 目录不存在（冷启动） + DescriptorMiss, // 描述符本地缺失，且该 ns 可证伪 + VersionMiss, // 描述符在，但 SemVer 约束在本地版本集内无解 + SuppressedDebounce, // 刚刷过（ env > config，一次解析好往下传 + bool offline = false; + std::int64_t debounceSeconds = 120; // 复用 xim 门的既有语义 +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" —— 进日志与错误文案 +}; + +// 纯函数、零副作用、零网络：可单测，判据表就是它的契约。 +RefreshDecision decide_for_dependency(const IndexRoute& route, + const DependencyCoordinate& coord, + std::string_view versionReq, + const mcpp::xlings::Env& env, + const RefreshPolicy& policy); + +// 执行 + 可解释输出 + 打标；进程内保证一次构建最多刷一次。 +std::expected apply(const RefreshDecision&, const mcpp::xlings::Env&); + +// 索引身份（`.xlings-index-version`），缺失返回 nullopt（本地 path 索引没有它）。 +std::optional index_revision(const std::filesystem::path& indexDir); + +// advisory：仅在解析失败与 `index status` 使用，不参与 gate。 +struct Staleness { std::optional days; std::optional rev; }; +Staleness staleness_hint(const mcpp::xlings::Env&); + +} // namespace mcpp::pm +``` + +判据表（`decide_for_dependency` 的全部行为，按序短路）： + +| 条件 | 结果 | +|---|---| +| `policy.offline` | `SuppressedOffline` → 不刷 | +| `!route.authoritative_for(ns)` | `SuppressedInconclusive` → 不刷（INV-3） | +| `index_pkgs_dir` 不存在 | `IndexAbsent` → 刷 | +| `route.read(coord) == nullopt` | `DescriptorMiss` → 刷 | +| `is_version_constraint(versionReq)` 且 `resolve_semver(...)` 失败 | `VersionMiss` → 刷 | +| 上面命中 miss 但距上次刷新 < `debounceSeconds` | `SuppressedDebounce` → 不刷 | +| 其余 | `None` → 不刷 | + +两条**简洁性**决定，让这张表比初稿更小也更对： + +- **VersionMiss 只对 SemVer 约束成立**。精确版本（`is_version_constraint == false`）不走 + `resolve_semver`，其 miss 由安装层已有的 miss 驱动兜住（`is_install_complete` 早退 → 失败则 + 刷一次重试，站点 3）。这同时避开一个陷阱：**版本表是 per-OS 的**，"本机没有" ≠ "不存在" + （`pm/commands.cppm:46-48` 已记录该判据），在此处硬判会造成每次构建的误触发。 +- **VersionMiss 直接复用 `resolve_semver` 的失败**，不复制约束求解逻辑 —— 否则又是一处 D1。 + +### 4.2 站点改造 + +| 站点 | 改法 | +|---|---| +| `prepare.cppm:1300-1311` | 删掉 `usesBuiltinIndex` + `is_index_fresh` 整段；对根 manifest 每个依赖调 `decide_for_dependency`，任一 `shouldRefresh` 则 `apply` 一次（进程内 once） | +| `pm/commands.cppm:167`（add） | 语义已对，改为调用 `decide_*` 以消除重复推导 | +| `xlings.cppm:1400`（xim 门） | 保留（服务 xim 索引、判据同构），debounce 常量改由 `RefreshPolicy` 提供，不再各写一个 120 | +| `package_fetcher.cppm:1060`（失败重试） | 保留，reason 归一为 `VersionMiss`，走统一输出 | +| `index_management.cppm:29`（search） | 保留 TTL —— 代码里**显式注明**这是唯一允许时间驱动的站点及理由 | + +### 4.3 D6：让 `mcpp update` 真的做事 + +`mcpp update` 是唯一语义为「我要更新依赖」的命令，今天却是空操作。改为： + +1. **强制刷索引**（显式意图 → 不看 TTL、不看 debounce；`offline` 下拒绝并说明）； +2. 清除 lock 条目（保留现有行为）； +3. 输出实际发生了什么：`Updated mcpplibs 8d67478 → a1b2c3d`（无变化时说"already at …"）。 + +这样「我知道上游有新版」有了名正言顺的入口，`build` 侧不必再开洞（§5）。 + +### 4.4 稳定性修复 + +- **S1（D3）** `is_index_dir_fresh`：`age.count() < 0` → 视为 **stale**。未来时间戳来自时钟回拨、 + 容器时间、tar 解包保留 mtime、CI cache 恢复；当前实现让索引「永远新鲜」直到墙钟追上。 +- **S2** 刷新失败**不再直接失败构建**。`apply()` 出错时：本地仍能解析 → 降级 warning 继续； + 只有「本地无答案 + 刷新也失败」才 hard error，并合并 advisory + `mcpp index update` 建议。 + （现状不统一：TTL 刷新失败被静默吞掉，project index clone 失败直接 hard error。） +- **S3** 索引刷新加**跨进程互斥**：复用 `platform/fs.cppm` 的 flock / LockFileEx（`bmi_cache` + 的既有用法），锁文件置于 index data 根。拿不到锁 → 不排队、不报错，直接跳过这次刷新 + （别人正在刷；本次继续用本地解析，失败再按 S2 处理）。这条同时消掉并发写 marker 的竞态。 +- **S4** `mark_known_indexes_refreshed` 在 `projectDir` 非空时早退 → 补打标，仅供 advisory， + 否则 `mcpp index status` 对自定义索引永远 unknown。 +- **S5** 3×retry 的 2s/4s 退避只在真需要刷新时才可能付费；`offline` 下完全不进 `update_index`。 + +### 4.5 跨平台 + +| 平台 | 影响 | +|---|---| +| **Windows** | 主要收益是**少跑一个多仓库同步**：进程创建 + Defender 扫描让每次 sync 显著更贵。另有残余风险 —— `source = "auto"` 可能回落 git，而 no-git + xvm git shim + `XLINGS_HOME` 重定向曾构成死锁（PR #114）；miss 驱动把这条路径从常态挪到例外。**S3 的锁必须用 `platform/fs.cppm` 的封装**，别写 POSIX-only 的 flock | +| **macOS** | 企业网络受限 + 首启 Gatekeeper 场景同理；大包冷路径不受影响（那是 install 轴） | +| **Termux / musl** | musl-static 的 DNS 短板已知（读不到 `$PREFIX/etc/resolv.conf`）。日常构建完全不碰 DNS 是该平台的可用性底线 | +| **mtime 语义** | `file_time_type` 的 epoch 跨平台不同（#317 踩过）。本设计把 mtime 缩到 debounce（秒级容差）+ advisory，并显式处理负 age（S1）；**身份判定改用 rev 文件，与 mtime 无关** | +| **本地 `path` 索引** | 没有 `.xlings-index-version`，也没有"刷新"概念 → `index_revision` 返回 nullopt，`authoritative_for` 判 local 可证伪但 `apply` 对它是 no-op。e2e 正是靠这条离线化 | +| **CI** | 隔离 `MCPP_HOME` + 工程内 `path` 索引即可离线验证全部分支（PR#307 已建立该模式），不需要真网络 | + +### 4.6 UX 与表达面（刻意收窄） + +**不加 `--refresh-index`**，理由见 §5。表达面只有三个旋钮： + +| 面 | 形态 | 语义 | +|---|---|---| +| flag | `--offline` | **本次调用不发起任何网络**：不刷索引、不下载包、不自动装工具链。与 cargo 的 `--offline` 对齐 | +| env | `MCPP_OFFLINE=1` | 同上，作用于整个会话/CI job | +| config | `[index] auto_refresh = true \| false` | 机器级持久开关；`false` = 索引轴永不自动刷（仍允许包下载）。默认 `true` | + +三条简化决定： + +- **不保留 `always`（旧 TTL 行为）模式**。为模拟一个正被删除的策略而留第二条代码路径，正是 D1 + 这笔债的来源。要「每次都最新」的用户写 shell alias 或 `mcpp index update && mcpp build`。 +- **`MCPP_NO_AUTO_INSTALL` 被 `--offline` 吸收**：它今天只管工具链（`prepare.cppm:1027`）， + 是同一概念的第三个名字。保留为**兼容别名**（等价于 offline 的工具链子集），文档标注 deprecated。 + 一个概念替三个。 +- `cache.search_ttl_seconds` **保留不动**（向后兼容），语义收窄为 `search` 站点的门 + advisory + 阈值，不再是构建路径的 gate。 + +**输出可解释**（D5）： + +``` +# miss 触发（复用 xim 门已有的句式） +Refreshing package index — `mcpplibs:fmt@^1.3` not satisfiable locally (one-time) +Updated mcpplibs 8d67478 → a1b2c3d + +# 稳态：默认静默，-v 可见 +[index] skip refresh: 4/4 deps resolvable locally (mcpplibs 8d67478, 3d old) + +# offline + miss：结构化错误 +error: `mcpplibs:fmt@^1.3` not found in the local package index + index mcpplibs is at 8d67478, last refreshed 12 days ago + run `mcpp index update` (or drop --offline) to fetch it +``` + +**陈旧提示只出现在两处**：解析失败时，以及 `mcpp index status` / `mcpp why deps`（后者加 rev 列）。 +**不在每次构建唠叨** —— 否则只是把网络噪音换成文字噪音。 + +**必须写进 `docs/` 的语义变化**：`^1.2` 将稳定解析到「本地索引已知的最新版」，上游新出的 1.3 需 +`mcpp index update` 或 `mcpp update` 才可见。这是 offline-first 的应有代价，也正是那两个命令存在 +的意义 —— 不写进文档就会变成下一个「为什么拉不到新版本」的 issue。 + +--- + +## 5. 为什么不选其他方案 + +| 方案 | 否决理由 | +|---|---| +| **`build/run/test --refresh-index`**（issue 原文与评论均建议） | ①`mcpp index update && mcpp build` 完全等价，加它等于给 3 个命令各加一份表达面，而它对构建产物**零影响**（不进指纹、不改产物、只有网络副作用）——挂在 `build` 上层次是错的；②cargo 的先例正是一对反例：**有** `build --offline`，**没有** `build --refresh-index`，刷新入口是 `cargo update`；③本设计的目的就是让「何时刷」不需要用户判断，留强制开关等于承认判据不可信。真正的需求（"我知道上游有新版"）语义属于 `mcpp update` → 改为 §4.3 | +| **延长默认 TTL（7 天）** | 只把 D2 的信号错误挪远，没修；且**放大** marker 假新鲜导致拉不到新版本的既有故障模式 | +| **只加 `--no-refresh`** | 默认仍然错，成本转嫁给每个用户每次输入；违反 INV-2 | +| **读 `mcpp.lock` 比对 manifest 后跳过刷新**（评论方案核心） | 前提不成立：`prepare.cppm` 对 lock **只写不读**（3829-3883），构建路径上 lock 不是解析输入。见 §9 | +| **直接把「查不到就刷」接上** | 违反 INV-3：xim 依赖永远匹配不上身份门 → 每次构建都刷，比现状更糟 | +| **把策略放进 `mcpp.toml`** | 违反 INV-4：网络环境属性写进项目清单，项目不可移植 | +| **新增 `mcpp refreshpkg` 子命令**（issue 原文） | `mcpp index update` 已在（`cli.cppm:423`）；新子命令只增认知负担 | + +--- + +## 6. 采纳 / 不采纳 issue 反馈一览 + +| issue / 评论诉求 | 处置 | +|---|---| +| 日常构建不该等网络 | **采纳**，且做成 INV-2 而非加 flag | +| 仅在「首次运行 / 依赖更新」刷新 | **采纳其意图，改写其判据**：不是"首次/变更"，而是"本地解析不出来"（更准，且不需要 lock） | +| `--refresh-pkg` / `mcpp refreshpkg` | **不采纳**（§5），改为修 `mcpp update`（D6） | +| `--refresh-index`（评论方案 2） | **不采纳**（§5） | +| 读 lock 比对 manifest（评论方案 1） | **不采纳**（前提不成立），拆为独立 issue（§9） | +| 不新增 `refreshpkg`、沿用 `index update`/`update`/`add`（评论方案 3） | **采纳** | +| `MCPP_OFFLINE=1`（评论方案 4） | **采纳**，并扩为完整 offline 语义、吸收 `MCPP_NO_AUTO_INSTALL` | +| `[index] auto_refresh`（评论方案 4） | **采纳但收窄为布尔**，不做三值（不保留旧 TTL 模式） | +| 延长 TTL（评论方案 D） | **不采纳**（§5） | + +--- + +## 7. 实施顺序与规模 + +| 阶段 | 内容 | 规模 | +|---|---|---| +| **P0（本 issue）** | §10 探针 → §4.1 策略模块（含 `index_revision`）→ §4.2 五站点收敛 → S1/S2/S3 → §4.3 `mcpp update` → §4.6 `--offline` / `MCPP_OFFLINE` / `[index] auto_refresh` + 可解释输出 → e2e | ~350–450 行，单 PR | +| **P1** | advisory 陈旧提示、`mcpp index status`/`why deps` 增加 rev 列、S4、`docs/` 新增「索引刷新策略」章节 | ~150 行 | +| **P2（各自独立 issue）** | ①`.xlings-index-cache.json` 烙绝对路径（主体在 xlings 侧）+ 把已有检测器接进 production；②`mcpp index update ` 的 per-index 粒度（需 xlings 支持）；③lock 成为构建期解析输入（§9） | — | + +P0 内部顺序：**先落 `decide_for_dependency` + 单测**（判据表是契约），再切站点 4，再切其余四站， +最后加表达面。先测再切，否则 INV-3 的回归无法及早暴露。 + +--- + +## 8. 测试矩阵 + +e2e（全部离线可跑：隔离 `MCPP_HOME` + 工程内 `path` 索引） + +| # | 场景 | 断言 | +|---|---|---| +| 1 | 依赖本地可解析 | **无** `Refreshing`/`Updating` 输出，exit 0 | +| 2 | 描述符本地缺失 | 刷一次，且只刷一次 | +| 3 | 本地只有 1.2，依赖 `^1.3` | 触发 `VersionMiss` | +| 4 | 精确版本 `= "1.2.3"` 而本机版本表无此条 | **不**在 decide 层触发（per-OS 版本表陷阱），由安装层兜 | +| 5 | `--offline` / `MCPP_OFFLINE=1` + miss | 结构化错误含 rev + age + `mcpp index update`，且**不发起网络** | +| 6 | **xim / lazy-git / 第三方 ns 依赖** | **不**触发刷新（INV-3 回归闸，最关键的一条） | +| 7 | 一次构建多个 miss | 只刷一次（进程内 once + debounce） | +| 8 | `mcpp update` | 输出 rev 变化（或 already at）；`--offline` 下明确拒绝 | +| 9 | 两个 mcpp 并发构建同一 MCPP_HOME | 只有一个刷新，另一个跳过而**不报错**（S3） | + +单测：`decide_for_dependency` 判据表逐行；`is_index_dir_fresh` 未来时间戳 → stale； +`index_revision` 在文件缺失/为空/含尾随换行时的行为。 + +**测试写法陷阱（本仓库已踩过）**：反向断言不要写 `! $MCPP build | grep -q X` —— 管道左侧在 +`errexit` 下被豁免，该断言**永不失败**。用 +`out=$($MCPP build 2>&1); echo "$out" | grep -q 'Refreshing' && { echo FAIL; exit 1; }`。 + +--- + +## 9. 明确不做：`mcpp.lock` 作为构建期解析输入 + +现状：`prepare.cppm` 只**写** mcpp.lock（3829-3883）；全仓库读 lock 的只有 +`pm/commands.cppm:394,413`（update/remove）与 `index_management.cppm:209`（index pin）。 +`LockedIndex.rev` 还只是用户 pin 的副本，不是观测值。 + +让 lock 成为解析输入是「零网络 + 可重现」的终局，但它牵出：`resolve_semver` 仍绕过 +`index_route` 的遗留（PR#307 的尾巴）；lock schema 演进与 index rev 的可信度； +「lock 与 manifest 漂移」的检测与自动修复语义（等价于 cargo 的 lock 更新规则）。 + +任一项都比本 issue 大。**本设计与它正交**：P0 之后即便永远不做 lock-honoring resolve，日常构建 +也已经零网络。而 INV-2 采纳的 `.xlings-index-version` 恰好是它未来需要的地基 +（把**观测到的** rev 写进 lock）。硬绑在一起只会让 #315 卡在大重构后面。 + +--- + +## 10. 探针结果（已跑，2026-07-30） + +设计里有两个前提来自磁盘观测而非源码，开工前实测（"写判据要先用探针实测误报"）。 + +### P-1 `.xlings-index-version` 的更新语义 — 通过，但推翻了一个格式假设 + +真实 `mcpp index update` 的输出： + +``` +[xlings] [index] updated from artifact xim-index-ebf4020.tar.gz (ebf4020) +[xlings] [index] updated from artifact mcpp-index-8d67478.tar.gz (8d67478) +[xlings] [index] updated from artifact xim-index-awesome-2026.7.30.1.tar.gz (2026.7.30.1) +``` + +- ✅ artifact 通道确实写 `.xlings-index-version`，值等于 artifact 名里的 rev；刷新后上游没变则值不变 + ⇒ **它是内容身份，不是时间戳**。marker mtime 同时被更新（17:16 → 21:42），两者职责因此可以分开。 +- ⚠️ **值不是稳定的 7 位 sha**：主索引是 `8d67478`，而子索引（awesome / scode / d2x）是**日期版本号** + `2026.7.30.1`。实现必须按**不透明字符串**处理 —— 只比较相等、只打印，绝不解析长度或格式。 + 文件本身**无尾随换行**，但实现仍 trim（Windows 侧写入的 `\r` 会让「同一 rev」每次都判成变化）。 +- ❓ git 通道未直接观测（本机全部走 artifact）。按设计的降级预案处理：缺失 → `index_revision` + 返回 nullopt，advisory 只显示 age，**gate 不受影响**（gate 只依赖描述符可读性）。 + +### P-2 INV-3 误报实测 — 通过 + +含 `mcpplibs.cmdline` + `xim.nasm` + `somevendor.thing` 的真实工程，`-v` 下的判定： + +``` +index: mcpplibs:cmdline@0.0.1: resolvable locally +index: xim.nasm@2.16.03: no index can refute this +index: somevendor.thing@1.0.0: no index can refute this +``` + +xim 与第三方 ns 都落在 `SuppressedInconclusive`，**零刷新**。这正是把 miss 直接当刷新理由会 +翻车的那一类：`xim.nasm` 被解析成候选 `(mcpplibs.xim, nasm)`，永远匹配不上身份门。 + +顺带发现并修正：诊断里回显解析后的候选（`mcpplibs.xim:nasm`）会把读者指向一个他从没写过的 +命名空间 —— 主语改为**用户写的 `[dependencies]` 键**。 + +回归闸：单测 `InconclusiveNamespaceNeverRefreshes` + e2e 173 第 3 步。 + +--- + +## 11. 风险登记 + +| 风险 | 缓解 | +|---|---| +| INV-3 判据接错 → 每次构建都刷（比现状更糟） | 探针 P-2 + e2e #6 双闸；先落单测再切站点 | +| `.xlings-index-version` 语义与假设不符 | 探针 P-1；gate 不依赖 rev，最坏只损失 advisory 精度 | +| 用户感知「mcpp 不再自动更新索引」 | advisory 提示 + docs 明写 + `mcpp update` 变成真入口（D6） | +| `VersionMiss` 与 `resolve_semver` 重复实现约束求解 | 强制调用同一入口，不复制逻辑（否则又一处 D1） | +| per-OS 版本表导致 `VersionMiss` 误触发 | 只对 SemVer 约束判 VersionMiss；精确版本交安装层（e2e #4 锁住） | +| 刷新失败的降级掩盖真实网络故障 | 降级只在「本地有答案」时发生，warning 必须可见 + `-v` 打印底层 rc | +| S3 的锁在 Windows 上行为不一致 | 必须走 `platform/fs.cppm` 封装；拿不到锁=跳过而非阻塞，避免死锁面 | diff --git a/CHANGELOG.md b/CHANGELOG.md index df0beb7f..a5d14615 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,42 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.7.30.3] — 2026-07-30 + +### 变更 + +- **索引刷新从「时间驱动」改为「解析驱动」(#315)。** `mcpp build/run/test` 此前只要刷新 marker 超过 1 小时,就无条件跑一次 `xlings update`(同步**全部**索引仓库,失败还带 3 次 2s/4s 退避重试)——不管有没有东西真的缺失。网络差时这就是「每小时一次、为已经在本地的数据等几分钟」。 + + 这不是「功能缺失」:offline-first 的策略**早就在仓库里**了(`xlings.cppm` 的 xim 安装门,注释里甚至点名了 Termux 上的 build hang),但 `prepare.cppm` 那道 TTL 门先开火,把它变成了不可达代码。同一个决策此前在 **5 处各推导一遍**,其中两处互相矛盾。 + + 现在只有一个真源 `mcpp.pm.index_refresh`:`mcpp build` / `mcpp add` / xim 安装门 / 安装失败重试全部走它。刷新只在三种情况发生 —— 本地没有索引、描述符不在本地、SemVer 约束在本地版本集内无解。**依赖全部能在本地解析的构建零网络请求,无论本地索引多旧。** + + **判据换轴的理由**:「索引够不够新」不可判定,而 mtime 是它的坏代理(CI 缓存恢复、时钟回拨、tar 保留时间戳都会让 marker 两个方向说谎)。可判定的问题是「解析器能不能用磁盘上的东西干活」——而**所有**解析输入都在磁盘上(描述符是文件,`resolve_semver` 解析本地 xpkg.lua)。marker 因此降级为纯去抖计时器。 + + **最危险的一条判据**(`SuppressedInconclusive`):「本地查不到」单独不能推出「需要刷新」。xim 描述符不写 `namespace`,`(xim, x)` 永远匹配不上身份门 —— 把这种 miss 当真,任何带 xim 依赖的工程会**每次构建都刷**,比被删掉的 TTL 更糟。判据复用 `IndexRoute::authoritative_for`(#307),单测 + e2e 双闸锁住。 + + 语义变化:`^1.2` 对**本地索引已知的版本**求解。上游新发的 1.3.0 需要 `mcpp index update` 或 `mcpp update` 才可见 —— 这正是那两个命令存在的意义,已写进 `docs/05-mcpp-toml.md`。 + +- **`mcpp update` 不再是空操作。** 它此前只删 mcpp.lock 条目、然后叫用户去跑 `mcpp build` —— 而构建路径**从不读 mcpp.lock**(`prepare` 只写不读),所以删了等于没删,行为影响为零。它现在先强制刷新索引(显式意图 ⇒ 不看 TTL、不看去抖),并报告索引 rev 的变化;工程里没有任何走共享 registry 的依赖时跳过(刷了也没用)。 + +- **新增 `--offline` / `MCPP_OFFLINE=1`。** 一次调用内完全不碰网络:不刷索引、不下载包、不自动装工具链。已安装的东西照常构建 —— 检查点都放在「真要下载」的那一刻,而不是更早。`MCPP_NO_AUTO_INSTALL` 作为它的旧式窄化拼写保留(只管工具链),文档标注 deprecated:同一个概念此前有三个名字。 + +- **新增 `[index] auto_refresh`(全局 `config.toml`)。** 机器级关闭自动刷新,下载仍可用。刻意做成布尔而**不**保留旧 TTL 模式:为模拟一个正在删除的策略而留第二条代码路径,正是本次要还的那笔债。策略归全局配置而非 `mcpp.toml` —— 它描述机器的网络环境,写进工程清单会让工程在内网/家里/CI 之间不可移植。 + +### 修复 + +- **未来时间戳的索引 marker 被判为「永远新鲜」。** `age < ttl` 对负数恒真,所以时钟回拨、容器时间、`tar` 保留 mtime、CI 缓存恢复产生的未来 marker 会让索引一直「新鲜」到墙钟追上为止。不可用的时间戳现在一律读作 unknown,而 unknown 必须是 stale。 + +- **索引刷新此前没有任何并发保护。** BMI 缓存早就用着 `platform/fs.cppm` 的跨平台 flock/LockFileEx,索引一个锁都没有 —— 并行 CI job 共享 `MCPP_HOME`、多个终端同时构建,就会并发重写同一棵树并并发写 marker。改为非阻塞互斥:**拿不到锁就跳过,不排队、不报错**(持锁者正在做的就是我们想要的事,排队只会把本 issue 要消除的症状原样复制一遍)。 + +- **项目级 env 下刷新永不打标。** `mark_known_indexes_refreshed` 在 `projectDir` 非空时整段早退,于是带自定义 `[indices]` 的工程刷新完全局索引却不留痕,`mcpp index status` 对这些机器永远显示 unknown。 + +### 新增 + +- **`mcpp index status` 增加 revision 列。** 索引自带内容身份 `.xlings-index-version`(artifact 分发把 rev 打进文件名:`mcpp-index-8d67478.tar.gz` → `8d67478`),mcpp 此前**零引用**。它是唯一能回答「两台机器/CI 缓存与本地是不是同一份索引」的信号,age 永远回答不了。按**不透明字符串**处理:子索引的值是日期版本号(`2026.7.30.1`)而不是 sha,探针实测,绝不解析。 + +- 依赖解析失败时附带索引身份与年龄(`index: local index 8d67478 (refreshed 12d ago)`)+ `mcpp index update` 建议 —— 变懒之后,「没这个包」和「你的索引是上个月的」必须能被区分开。 + ## [2026.7.30.2] — 2026-07-30 ### 修复 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 3141a82a..a5ff1870 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -416,6 +416,38 @@ accepted, so already-published descriptors keep working. `mcpp xpkg parse` enforces the rule — run it in your index CI. Requires mcpp >= 0.0.106 and xlings >= 0.4.69; full normative text in `docs/spec/package-identity.md`. +#### When mcpp refreshes the package index + +`mcpp build` / `run` / `test` refresh the package index **only when a dependency +cannot be resolved from the local copy** — never merely because time has passed. +Concretely, a refresh happens when there is no local index at all, when a +dependency's descriptor is missing from it, or when a SemVer constraint matches +none of the versions it knows. A build whose dependencies all resolve locally +makes no network request, however old the local index is. + +The consequence worth knowing: a constraint like `^1.2` resolves against the +**versions your local index knows**. If `1.3.0` was published upstream after +your last refresh, you will not see it until you ask: + +```bash +mcpp index update # sync the index +mcpp update # sync, then re-resolve dependencies +mcpp index status # what you have locally: state, age and revision +``` + +Controls, in order of precedence: + +| Control | Effect | +|---|---| +| `--offline` (any command) | Never touch the network — no index refresh, no downloads, no toolchain auto-install. Anything already installed still builds | +| `MCPP_OFFLINE=1` | Same, for a whole shell session or CI job | +| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh the index automatically; downloads still work | + +`MCPP_NO_AUTO_INSTALL=1` remains accepted as the older, narrower spelling of +`--offline` (it gates only toolchain auto-install). + +Run any command with `-v` to see the decision for each dependency and why. + ### 2.6 `[dev-dependencies]` — Test Dependencies ```toml diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 670703ce..dc622085 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -409,6 +409,35 @@ simd = { sources = ["src/simd/**"], flags = [ feature `flags` 是**私有 per-TU 构建旗标**——永不传播给消费者(与 `[build].flags` 同契约),因此不破坏加性模型:glob 限定作用面、顺序确定、无跨包效应。 +#### mcpp 何时刷新包索引 + +`mcpp build` / `run` / `test` **只在依赖无法用本地索引解析时**刷新包索引,绝不会 +因为"时间到了"就刷。具体地说,只有三种情况会触发:本地根本没有索引、依赖的描述符 +不在其中、或 SemVer 约束在本地已知版本里无解。只要所有依赖都能在本地解析出来, +无论本地索引多旧,构建都不会发起任何网络请求。 + +由此带来的一个需要知道的语义:`^1.2` 这类约束是对**本地索引已知的版本**求解的。 +如果上游在你上次刷新之后发布了 `1.3.0`,你需要主动去取: + +```bash +mcpp index update # 同步索引 +mcpp update # 同步索引,并重新解析依赖 +mcpp index status # 看本地现状:状态、年龄、修订号 +``` + +三个开关,优先级从高到低: + +| 开关 | 作用 | +|---|---| +| `--offline`(任意命令) | 完全不碰网络——不刷索引、不下载、不自动装工具链。已安装的东西照常构建 | +| `MCPP_OFFLINE=1` | 同上,作用于整个 shell 会话或 CI job | +| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不自动刷新索引,但下载仍然可用 | + +`MCPP_NO_AUTO_INSTALL=1` 作为 `--offline` 的旧式窄化拼写仍然有效(它只管工具链的 +自动安装)。 + +任意命令加 `-v` 可以看到每个依赖的判定结果与原因。 + ### 2.8.1 `provides` / `requires` —— 能力(后端选择) **capability(能力)** 是一个共享的抽象名字(如 `blas`)。包可以 *provide*(提供) diff --git a/mcpp.toml b/mcpp.toml index b0a3136c..b44e6c9c 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.7.30.2" +version = "2026.7.30.3" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 617ccc0b..5342eb7b 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -42,6 +42,7 @@ import mcpp.fetcher.progress; import mcpp.pm.resolver; import mcpp.pm.index_spec; import mcpp.pm.index_route; +import mcpp.pm.index_refresh; import mcpp.pm.mangle; import mcpp.pm.compat; import mcpp.pm.dep_spec; @@ -1146,27 +1147,39 @@ prepare_build(bool print_fingerprint, mcpp::fetcher::make_path_ctx(&**get_cfg(), *root)))); } else if (tcSpec.has_value() && *tcSpec == "system") { // Explicit user opt-in to system PATH compiler — kept as escape hatch. - } else if (auto* opt = std::getenv("MCPP_NO_AUTO_INSTALL"); opt && *opt && *opt != '0') { + } else if (mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install()) { // CI / offline / test opt-out: hard-error instead of silently // pulling ~800 MB of toolchain. Preserves the original M5.5 // contract for environments that need it. + // + // `--offline` / MCPP_OFFLINE subsumes MCPP_NO_AUTO_INSTALL: the older + // name only ever covered this one gate, which made "don't use the + // network" three separate concepts with three spellings. The old var is + // kept working (it predates offline mode and CI still exports it). namespace pins = mcpp::toolchain::triple::pins; + // Name the knob that actually fired, not a fixed one: telling a user + // who passed `--offline` to unset MCPP_NO_AUTO_INSTALL sends them + // looking for a variable they never set. + std::string_view release = mcpp::platform::env::offline_mode() + ? "or drop --offline / unset MCPP_OFFLINE to let mcpp auto-install." + : "or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install."; if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" " mcpp toolchain install {}\n" " mcpp toolchain default {}\n" - " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install.", - pins::kSuggestLlvm, pins::kFirstRunMacWin)); + " {}", + pins::kSuggestLlvm, pins::kFirstRunMacWin, release)); } else { return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" " mcpp toolchain install {}\n" " mcpp toolchain default {}\n" - " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install.", - pins::kSuggestGccMusl, pins::kFirstRunLinuxOther)); + " {}", + pins::kSuggestGccMusl, pins::kFirstRunLinuxOther, release)); } } else { // First-run UX: no project-level [toolchain], no global default, @@ -1388,48 +1401,43 @@ prepare_build(bool print_fingerprint, // two different exact versions is an error — mcpp prints both // requesting parents and asks the user to align them. - // Auto-refresh the builtin package index only when a version dependency - // is actually routed there. Local/remote project indices are handled by - // the project-scoped setup below; refreshing the global index for those - // packages is both unnecessary and can make offline/local-index builds - // block on unrelated remote repositories. + // Refresh the builtin package index only when a dependency cannot be + // resolved from the local copy (#315). + // + // This used to fire whenever the refresh marker was older than an hour, + // whether or not anything was actually missing — so every build with a + // registry dependency paid a multi-repo network sync once an hour, which is + // minutes on a slow or blocked network for data it already had. The policy + // now lives in mcpp.pm.index_refresh and is shared with `mcpp add` and the + // xim install gate, which had each derived their own (and disagreed). + // + // Nothing here decides anything itself — in particular the "a miss proves + // nothing for this namespace" rule must not be re-derived; see that module. if (!m->dependencies.empty()) { - auto usesBuiltinIndex = [&](const mcpp::manifest::DependencySpec& spec) { - if (spec.isPath() || spec.isGit()) return false; - - auto ns = spec.namespace_.empty() - ? std::string(mcpp::pm::kDefaultNamespace) - : spec.namespace_; - if (ns == mcpp::pm::kDefaultNamespace) { - // R6: `[indices] default = {...}` (normalized to - // kDefaultNamespace by toml.cppm) redirects the default - // namespace away from the builtin registry — consult it - // before assuming builtin, same as any named namespace. - auto it = m->indices.find(std::string(mcpp::pm::kDefaultNamespace)); - return it == m->indices.end() || it->second.is_builtin(); - } - - auto it = m->indices.find(ns); - if (it == m->indices.end()) return true; - return it->second.is_builtin(); - }; - - bool needsBuiltinIndexRefresh = false; - for (auto& [_, spec] : m->dependencies) { - if (usesBuiltinIndex(spec)) { - needsBuiltinIndexRefresh = true; - break; - } - } - if (needsBuiltinIndexRefresh) { - auto cfg2 = get_cfg(); - if (cfg2) { - auto xlEnv = mcpp::config::make_xlings_env(**cfg2); - if (!mcpp::xlings::is_index_fresh(xlEnv, (*cfg2)->searchTtlSeconds)) { - mcpp::ui::status("Updating", "package index (auto-refresh)"); - mcpp::xlings::ensure_index_fresh( - xlEnv, (*cfg2)->searchTtlSeconds, /*quiet=*/true); + if (auto cfg2 = get_cfg()) { + auto xlEnv = mcpp::config::make_xlings_env(**cfg2); + auto policy = mcpp::pm::policy_for(**cfg2); + // Same routing the dependency walk below uses (the `index_route` + // lambda is declared further down; this is the identical value). + mcpp::pm::IndexRoute route{ &m->indices, *root, *cfg2 }; + for (auto& [depName, spec] : m->dependencies) { + auto decision = mcpp::pm::decide_for_dependency( + route, depName, spec, xlEnv, targetPlatform, policy); + if (!decision.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "{}: {}", decision.subject, + mcpp::pm::reason_text(decision.reason))); + continue; } + // A failed refresh is not a failed build: the dependency walk + // below may still resolve everything from what is on disk, and + // if it cannot, it reports the actual missing package with the + // index's age attached. Failing here instead would turn a + // transient network blip into a hard stop for a build that + // needed no network at all. + if (auto r = mcpp::pm::apply(decision, xlEnv); !r) + mcpp::ui::warning(r.error()); + break; // one sync covers every dependency } } } @@ -1753,6 +1761,17 @@ prepare_build(bool print_fingerprint, } } + // Advisory, never a gate (#315): now that a build only refreshes + // the index on a miss, "not found" and "your copy of the index + // is from last month" are easy to confuse. State which index + // answered and how old it is, so the next step is obvious + // instead of guessed at. + if (auto cfgA = get_cfg()) { + hint += std::format("\n index: {}\n hint: `mcpp index update` " + "if it was published recently", + mcpp::pm::staleness_note( + mcpp::config::make_xlings_env(**cfgA))); + } return std::unexpected(std::format( "dependency '{}': no package found under the namespaces " "mcpp searched\n tried: {}{}", @@ -1971,6 +1990,22 @@ prepare_build(bool print_fingerprint, // progress line and errors should echo back. auto displayName = ns.empty() ? shortName : std::format("{}.{}", ns, shortName); + + // Offline (#315). Checked HERE, at the point of download, and not + // any earlier: everything above this line — reading descriptors, + // resolving versions, reusing an already-installed package — is + // local, and an offline build that has its dependencies must + // succeed. Only the download itself is refused, and it names the + // package rather than surfacing a socket error from three layers + // down. (The toolchain payload path has its own gate; this is the + // dependency path, which does not go through resolve_xpkg_path.) + if (mcpp::platform::env::offline_mode()) { + return std::unexpected(std::format( + "offline mode: dependency '{}' v{} is not installed and " + "cannot be downloaded\n" + " run without --offline (or unset MCPP_OFFLINE) to fetch it", + displayName, version)); + } mcpp::ui::info("Downloading", std::format("{} v{}", displayName, version)); // #238: retain whatever error/warn text the child DID emit so we diff --git a/src/cli.cppm b/src/cli.cppm index 8f75b35e..007a37c3 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -28,6 +28,7 @@ import mcpp.cli.cmd_self; import mcpp.cli.cmd_toolchain; import mcpp.pm.commands; import mcpp.toolchain.fingerprint; // MCPP_VERSION +import mcpp.platform.env; // --offline → MCPP_OFFLINE import mcpp.ui; import mcpp.log; @@ -84,6 +85,7 @@ void print_usage() { std::println(" --cache Dependency cache: global (default) | local | off"); std::println(" --no-cache Deprecated alias for --cache=off (clears the build dir)"); std::println(" --no-color Disable colored output"); + std::println(" --offline Never touch the network (also: MCPP_OFFLINE=1)"); std::println(""); std::println("Docs: https://github.com/mcpp-community/mcpp/tree/main/docs"); } @@ -103,6 +105,12 @@ int run(int argc, char** argv) { if (a == "--quiet" || a == "-q") mcpp::ui::set_quiet(true); else if (a == "--no-color") mcpp::ui::disable_color(); else if (a == "--verbose" || a == "-v") mcpp::log::set_verbose(true); + // --offline is published as the env var rather than plumbed through + // BuildOverrides: its consumers are index refresh, package install and + // toolchain auto-install, which sit in three subsystems and would each + // need a parameter threaded down. Same shape as MCPP_VERBOSE above, and + // it makes `MCPP_OFFLINE=1` and `--offline` literally the same switch. + else if (a == "--offline") mcpp::platform::env::set("MCPP_OFFLINE", "1"); } // Env override (observability, esp. CI): MCPP_VERBOSE= // turns on verbose logging for EVERY mcpp invocation — including the ones @@ -203,6 +211,9 @@ int run(int argc, char** argv) { .help("Show detailed progress on stderr").global()) .option(cl::Option("no-color") .help("Disable colored output").global()) + .option(cl::Option("offline") + .help("Never touch the network (index refresh, downloads, toolchain install)") + .global()) // ─── project commands ────────────────────────────────────────── .subcommand(cl::App("new") diff --git a/src/config.cppm b/src/config.cppm index c352f772..129a7419 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -78,8 +78,20 @@ struct GlobalConfig { std::map indices; // From config.toml [cache] + // NOTE (#315): this is no longer a gate on the build path. It bounds + // `mcpp search` (which exists to ask upstream) and dates the status line. + // Whether a build refreshes is decided by mcpp.pm.index_refresh from local + // resolvability, not from a clock. std::int64_t searchTtlSeconds = 3600; + // From config.toml [index] auto_refresh + // Machine-level opt-out from the automatic on-miss index refresh. Distinct + // from offline mode: this only silences the index axis, package downloads + // still happen. Policy lives in the global config, never in mcpp.toml — it + // describes the machine's network, and a project that carried it would stop + // being portable between a corporate LAN, a laptop and CI. + bool indexAutoRefresh = true; + // From config.toml [build] std::int64_t defaultJobs = 0; std::string defaultBackend = "ninja"; @@ -325,6 +337,10 @@ home = "" [index] default = "mcpplibs" +# auto_refresh: refresh the package index automatically when a dependency +# cannot be resolved from the local copy (never merely because time passed). +# Set false to require an explicit `mcpp index update`. +auto_refresh = true [index.repos."mcpplibs"] url = "https://github.com/mcpplibs/mcpp-index.git" @@ -333,6 +349,8 @@ artifact = "https://github.com/xlings-res/mcpp-index" # xlings auto-adds xim / awesome / scode / d2x as defaults. [cache] +# Bounds `mcpp search` and the age shown by `mcpp index status`. It is NOT +# what decides whether a build refreshes the index — see [index] auto_refresh. search_ttl_seconds = 3600 [build] @@ -494,6 +512,7 @@ std::expected load_or_init( if (auto h = doc->get_string("xlings.home"); h && !h->empty()) cfg.xlingsHomeOverride = *h; cfg.defaultIndex = doc->get_string("index.default").value_or("mcpplibs"); + cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); cfg.defaultBackend = doc->get_string("build.default_backend").value_or("ninja"); diff --git a/src/platform/env.cppm b/src/platform/env.cppm index a64210b5..7ac88fb1 100644 --- a/src/platform/env.cppm +++ b/src/platform/env.cppm @@ -23,6 +23,27 @@ std::optional get(std::string_view key); // Set an environment variable in the current process. void set(const std::string& key, const std::string& value); +// ── Network policy: offline mode ────────────────────────────────────────── +// +// One process-wide knob answering "may mcpp reach the network at all?", read +// at each point of use rather than threaded through a dozen signatures — the +// same idiom the older MCPP_NO_AUTO_INSTALL gate already uses. +// +// It lives HERE, in a leaf module, on purpose: the consumers span pm (index +// refresh, package install) and build (toolchain auto-install), and any home +// inside one of those subsystems would make the other import it across a layer +// boundary — mcpp.pm.index_route already imports mcpp.fetcher, so a pm-owned +// helper could not be read from the fetcher without a module cycle. +// +// Set by `--offline` (cli pre-scan writes the env var) or by the user exporting +// MCPP_OFFLINE. "0" and the empty string mean off, matching MCPP_VERBOSE. +bool offline_mode(); + +// The older, narrower spelling: MCPP_NO_AUTO_INSTALL gates ONLY the toolchain +// auto-install, nothing else. Kept working (CI and existing setups export it) +// and kept next to offline_mode() so the two network knobs cannot drift apart. +bool no_auto_install(); + // Temporarily set or unset an env var, restoring the prior value on scope exit. class ScopedEnv { public: @@ -74,6 +95,18 @@ std::string build_env_prefix( namespace mcpp::platform::env { +bool offline_mode() { + // Not cached: `mcpp test` runs nested mcpp invocations in-process in some + // paths, and a test that flips the var must see the change. + auto* v = std::getenv("MCPP_OFFLINE"); + return v && *v && std::string_view(v) != "0"; +} + +bool no_auto_install() { + auto* v = std::getenv("MCPP_NO_AUTO_INSTALL"); + return v && *v && std::string_view(v) != "0"; +} + std::optional get(std::string_view key) { std::string k(key); auto* v = std::getenv(k.c_str()); diff --git a/src/pm/commands.cppm b/src/pm/commands.cppm index 1f709daa..36d74caa 100644 --- a/src/pm/commands.cppm +++ b/src/pm/commands.cppm @@ -21,6 +21,7 @@ import mcpp.platform.axis; // HostPlatform for the published-version chec import mcpp.pm.dep_spec; // DependencyCoordinate import mcpp.pm.dependency_selector; // same candidates the manifest parser derives import mcpp.pm.index_route; // shared index routing (with mcpp.build.prepare) +import mcpp.pm.index_refresh; // shared refresh policy (with mcpp.build.prepare) import mcpp.pm.resolver; // is_version_constraint import mcpp.project; // shared find_manifest_root import mcpp.ui; @@ -161,17 +162,19 @@ inline int cmd_add(const mcpplibs::cmdline::ParsedArgs& parsed) { return !idx || idx->is_builtin(); }); - // Only pay for a refresh when the answer was "no" and the registry - // that would have answered is stale — a package that is already on - // disk costs zero network round-trips. + // Only pay for a refresh when the answer was "no" and the registry is + // the thing that would have answered — a package already on disk costs + // zero network round-trips. This site had the right shape before #315; + // it now shares the decision with `mcpp build` instead of re-deriving + // it, so the two cannot drift on debounce, offline or opt-out. if (!found.hit && found.conclusive && registryInvolved) { auto xlEnv = mcpp::config::make_xlings_env(*cfg); - if (!mcpp::xlings::is_index_fresh(xlEnv, cfg->searchTtlSeconds)) { - mcpp::ui::status("Updating", "package index (auto-refresh)"); - mcpp::xlings::ensure_index_fresh( - xlEnv, cfg->searchTtlSeconds, /*quiet=*/true); + auto d = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(*cfg), xlEnv, nameSpec); + if (auto r = mcpp::pm::apply(d, xlEnv); !r) + mcpp::ui::warning(r.error()); + if (d.shouldRefresh) found = mcpp::pm::lookup_descriptor(route, selector.candidates); - } } if (!found.hit && found.conclusive) { @@ -406,6 +409,42 @@ inline int cmd_update(const mcpplibs::cmdline::ParsedArgs& parsed) { auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) { mcpp::ui::error("no mcpp.toml in current dir or parents"); return 2; } + + // Refresh the index FIRST (#315/D6). + // + // This command used to only drop lock entries and tell the user to run + // `mcpp build` — but the build path never reads mcpp.lock (prepare writes + // it and nothing on that path loads it), so the whole command was a no-op: + // it changed no behaviour whatsoever. It is also the only command whose + // stated purpose is "get me newer dependencies", which since #315 is + // exactly what an index refresh is for. Explicit intent, so no debounce and + // no TTL — but still refused when offline, loudly, rather than silently + // doing nothing again. + // Skipped when nothing in the project is served by the shared registry: + // syncing it does nothing for path deps, git deps or a project `[indices]` + // entry, and paying a multi-repo network round-trip to achieve nothing is + // the exact behaviour #315 is about. + if (auto cfg = mcpp::config::load_or_init( + /*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback())) { + auto m = mcpp::manifest::load(*root / "mcpp.toml"); + bool registryInvolved = false; + if (m) { + auto indices = mcpp::pm::effective_indices(*root); + mcpp::pm::IndexRoute route{ &indices, *root, &*cfg }; + for (auto& [_, spec] : m->dependencies) + if (mcpp::pm::routes_to_builtin(route, spec)) { registryInvolved = true; break; } + } + if (registryInvolved) { + auto xlEnv = mcpp::config::make_xlings_env(*cfg); + if (auto r = mcpp::pm::force_refresh(xlEnv); !r) { + mcpp::ui::error(r.error()); + return 1; + } + } + } else { + mcpp::ui::warning(cfg.error().message); + } + auto lockPath = *root / "mcpp.lock"; if (only) { // Targeted update — drop just that lock entry; next build will refetch. diff --git a/src/pm/index_management.cppm b/src/pm/index_management.cppm index 715cf738..60895fde 100644 --- a/src/pm/index_management.cppm +++ b/src/pm/index_management.cppm @@ -14,6 +14,7 @@ import mcpp.fetcher; import mcpp.fetcher.progress; import mcpp.lockfile; import mcpp.manifest; +import mcpp.platform; import mcpp.project; import mcpp.ui; import mcpp.xlings; @@ -25,6 +26,16 @@ export int search_packages(const std::string& keyword) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } + // THE ONLY TIME-DRIVEN REFRESH LEFT IN MCPP, and deliberately so. + // + // Everywhere else the question is "can the resolver work with what is on + // disk", which is answerable offline and therefore what #315 made the gate. + // Search has no such local answer to check: the user is asking what EXISTS + // upstream, so a stale index gives a confidently wrong answer ("no matches" + // for a package published last week) rather than a slow one. A TTL is the + // right instrument here — do not "unify" this with mcpp.pm.index_refresh. + // Offline is still honoured: update_index() returns without touching the + // network, and search reports whatever the local copy holds. auto xlEnv = mcpp::config::make_xlings_env(*cfg); if (!mcpp::xlings::is_index_fresh(xlEnv, cfg->searchTtlSeconds)) { mcpp::ui::status("Updating", "package index (auto-refresh)"); @@ -113,10 +124,24 @@ export int index_remove(const std::string& name) { } // `mcpp index update [name]` — empty filterName updates everything. +// +// NOTE on `name`: it only filters the PROJECT-level custom indices below. The +// global repos are always synced wholesale because `xlings update` has no +// per-index mode to call. Tracked as a follow-up; the help text says "if given, +// update only this index", which is not what happens for the global set. export int index_update(const std::string& filterName) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } + // Explicit command: offline must refuse audibly rather than no-op. Left to + // this layer because update_index() itself is a silent skip by design (it + // is called from paths where proceeding locally is the right answer). + if (mcpp::platform::env::offline_mode()) { + mcpp::ui::error("offline mode is on — cannot update the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + return 1; + } + // Update global index repos. mcpp::ui::status("Updating", "all index repos"); auto xlEnv = mcpp::config::make_xlings_env(*cfg); @@ -173,15 +198,20 @@ export int index_status() { std::string state = !st.present ? "missing" : st.fresh ? "fresh" : "stale"; - std::println(" {:<10} {:<8} {:<12} {}", - label, state, fmt_age(st.ageSeconds), st.dir.string()); + // The revision is the index's CONTENT identity (see index_revision): + // it is what actually tells you whether two machines — or a CI cache + // and a laptop — hold the same index, which the age never could. + std::println(" {:<10} {:<8} {:<12} {:<12} {}", + label, state, fmt_age(st.ageSeconds), + st.rev.value_or("-"), st.dir.string()); }; auto official = mcpp::xlings::official_index_status(xlEnv, cfg->searchTtlSeconds); auto deflt = mcpp::xlings::default_index_status(xlEnv, cfg->searchTtlSeconds); std::println(""); - std::println(" {:<10} {:<8} {:<12} {}", "index", "state", "refreshed", "path"); + std::println(" {:<10} {:<8} {:<12} {:<12} {}", + "index", "state", "refreshed", "revision", "path"); show("xim", official); show("mcpplibs", deflt); std::println(""); diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm new file mode 100644 index 00000000..eddd14b0 --- /dev/null +++ b/src/pm/index_refresh.cppm @@ -0,0 +1,354 @@ +// mcpp.pm.index_refresh — the single source of truth for "should mcpp touch +// the network to refresh a package index right now?" +// +// WHY THIS MODULE EXISTS +// +// The answer used to be derived independently in five places, and two of them +// disagreed. `mcpp.xlings`' xim gate (ensure_official_package_index_fresh) had +// been offline-first for a while — it refreshes only when a requested package +// is genuinely absent locally, and its comment says in so many words that a +// TTL must NOT trigger a network sync, because that is what hangs a build on a +// slow or blocked network. `mcpp.build.prepare` meanwhile refreshed on exactly +// that: a marker older than an hour, whether or not anything was missing. So +// the offline-first policy was real but unreachable — the TTL fired first, +// every hour, on every build with a registry dependency (#315). +// +// THE AXIS WAS WRONG, NOT JUST THE THRESHOLD +// +// "Is this index fresh enough?" is unanswerable, and mtime is a bad proxy for +// it: a restored CI cache, a clock skew, or a tar that preserved timestamps all +// produce a marker that lies in either direction. The answerable question is +// "can the resolver do its job with what is on disk?" — and every resolution +// input IS on disk (descriptors are files; `resolve_semver` parses a local +// xpkg.lua). So the decision below is a pure, offline, deterministic function +// of local state, and the marker is demoted to a debounce timer. +// +// THE TRAP THIS MODULE IS MOSTLY WRITTEN AROUND +// +// "Not found locally" must NEVER be read as "needs a refresh" on its own. A +// miss only means something when the index that would have answered is both +// readable and authoritative — `IndexRoute::authoritative_for` (#307) is what +// decides that. Notably, xim descriptors declare no `namespace`, so `(xim, x)` +// can never match the identity gate: treat that miss as real and every build +// with a toolchain-ish dependency refreshes EVERY TIME, which is strictly worse +// than the TTL this change removes. See `SuppressedInconclusive`. + +export module mcpp.pm.index_refresh; + +import std; +import mcpp.config; +import mcpp.log; +import mcpp.platform; +import mcpp.platform.axis; +import mcpp.pm.dep_spec; +import mcpp.pm.index_route; +import mcpp.pm.resolver; +import mcpp.ui; +import mcpp.xlings; + +export namespace mcpp::pm { + +// Why a refresh was, or was not, triggered. Every value is user-visible under +// `-v`, and the Suppressed* ones are the interesting half when diagnosing +// "why did/didn't mcpp go to the network". +enum class RefreshReason { + None, // resolvable locally — the steady state, zero network + IndexAbsent, // no local index at all (cold start) + DescriptorMiss, // package unknown locally, and the index can say so + VersionMiss, // package known, constraint unsatisfiable locally + SuppressedOffline, // --offline / MCPP_OFFLINE + SuppressedDisabled, // [index] auto_refresh = false + SuppressedDebounce, // refreshed moments ago; upstream simply lacks it + SuppressedInconclusive, // a miss here proves nothing (see header) +}; + +struct RefreshPolicy { + bool offline = false; + bool autoRefresh = true; + // Shared with the xim install gate — one constant, one rationale, in the + // leaf module both layers can see (mcpp::xlings). + std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors +}; + +// Human tail for the status line: "package index — ". +std::string_view reason_text(RefreshReason r); + +// flag > env > config. `offlineFlag` is the parsed `--offline`. +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); + +// Pure: no network, no filesystem writes, no side effects. The judgement table +// is the contract — `tests/unit/test_pm_index_refresh.cpp` locks it row by row. +RefreshDecision decide_for_dependency(const IndexRoute& route, + std::string_view depKey, + const DependencySpec& spec, + const mcpp::xlings::Env& env, + const mcpp::platform::PlatformKey& platform, + const RefreshPolicy& policy); + +// Is this dependency served by the shared registry (as opposed to a project +// `[indices]` entry, a path, or a git URL)? Refreshing the global index does +// nothing for the others, so callers use this to skip the sync entirely. +bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec); + +// For a caller that has ALREADY established a conclusive miss through +// `lookup_descriptor` (that is `mcpp add`): only the policy half of the +// judgement is left to make. Keeps the opt-outs in one place rather than +// re-tested at each call site. +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject); + +// Run the sync a decision asked for. At most ONE per process: a build whose +// deps miss for the same reason should pay one sync, not one per dep. +// Returns an error only when the sync itself failed; callers decide whether +// that is fatal (it is not, if the build can still resolve locally). +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env); + +// Explicit user intent (`mcpp update`): ignores debounce and the once-per- +// process guard. Still refuses when offline, and says so. +std::expected force_refresh(const mcpp::xlings::Env& env); + +// One line of advisory context for a resolution failure: what the index is and +// how old it is. Never a gate — only ever appended to an error the user is +// already seeing. +std::string staleness_note(const mcpp::xlings::Env& env); + +} // namespace mcpp::pm + +namespace mcpp::pm { + +namespace { + +// One sync per process, however many dependencies ask for one. +bool g_refreshed_this_process = false; + +// Echo back what the USER wrote (the `[dependencies]` key), not the resolved +// coordinate. `xim.nasm` is parsed into the candidate `(mcpplibs.xim, nasm)`, +// and reporting that spelling in a diagnostic sends the reader looking for a +// namespace they never typed. +std::string subject_of(std::string_view depKey, const DependencySpec& spec) { + std::string name{depKey}; + if (name.empty()) { + name = spec.shortName; + if (!spec.namespace_.empty()) + name = std::format("{}:{}", spec.namespace_, name); + } + return spec.version.empty() ? name : std::format("{}@{}", name, spec.version); +} + +// Does ANY candidate for this dependency resolve through the shared registry? +// A project `[indices] path = …` is whatever the user has on disk and a custom +// git index is synced by its own project-scoped path — refreshing the global +// index would do nothing for either, so those keep today's behaviour exactly. +bool coords_route_to_builtin(const IndexRoute& route, + const std::vector& coords) { + for (auto& c : coords) { + auto* idx = route.find_for_ns(c.namespace_); + if (!idx || idx->is_builtin()) return true; + } + return false; +} + +std::vector coords_of(std::string_view depKey, + const DependencySpec& spec) { + if (!spec.candidates.empty()) return spec.candidates; + // Specs built by hand (tests, older call sites) carry no candidate list. + return { DependencyCoordinate{ + .namespace_ = spec.namespace_, + .shortName = spec.shortName.empty() ? std::string(depKey) : spec.shortName, + } }; +} + +std::string age_phrase(std::int64_t s) { + if (s < 0) return "never refreshed"; + if (s < 90) return std::format("refreshed {}s ago", s); + if (s < 5400) return std::format("refreshed {}m ago", s / 60); + if (s < 172800) return std::format("refreshed {}h ago", s / 3600); + return std::format("refreshed {}d ago", s / 86400); +} + +} // namespace + +std::string_view reason_text(RefreshReason r) { + switch (r) { + case RefreshReason::None: return "resolvable locally"; + case RefreshReason::IndexAbsent: return "no local package index"; + case RefreshReason::DescriptorMiss: return "not found locally"; + case RefreshReason::VersionMiss: return "not satisfiable locally"; + case RefreshReason::SuppressedOffline: return "offline mode"; + case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; + case RefreshReason::SuppressedDebounce: return "index was just refreshed"; + case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + } + return ""; +} + +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { + RefreshPolicy p; + p.offline = mcpp::platform::env::offline_mode(); + p.autoRefresh = cfg.indexAutoRefresh; + return p; +} + +RefreshDecision decide_for_dependency(const IndexRoute& route, + std::string_view depKey, + const DependencySpec& spec, + const mcpp::xlings::Env& env, + const mcpp::platform::PlatformKey& platform, + const RefreshPolicy& policy) +{ + RefreshDecision d; + d.subject = subject_of(depKey, spec); + + // 1. Sources that never consult an index. + if (spec.isPath() || spec.isGit()) return d; + + // 2. Only the shared registry is refreshable from here. + auto coords = coords_of(depKey, spec); + if (!coords_route_to_builtin(route, coords)) return d; + + // 3. INV-3: a miss only counts when the index could have refuted it. + auto found = lookup_descriptor(route, coords); + if (!found.hit && !found.conclusive) { + d.reason = RefreshReason::SuppressedInconclusive; + return d; + } + + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + + // 4-6. The three ways local state fails to answer. + // + // Deliberately evaluated BEFORE the opt-outs below: this is all local file + // I/O, and knowing whether the answer was actually available is what makes + // the diagnostic worth reading. Short-circuiting on `--offline` first would + // report "offline mode" for a dependency that resolved perfectly well — + // which is precisely the case the user is trying to confirm. + if (!status.present) { + d.reason = RefreshReason::IndexAbsent; + d.shouldRefresh = true; + } else if (!found.hit) { + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + } else if (is_version_constraint(spec.version)) { + // Only CONSTRAINTS are checked here. An exact pin is deliberately not: + // version tables are per-OS, so "this host publishes no 1.2.3" is not + // "1.2.3 does not exist" (the same judgement `mcpp add` reaches — it + // flags an unpublished exact version rather than refusing it). Treating + // it as a miss would refresh on every build for a dependency that is + // simply not built for this platform. The install layer already covers + // the real version-miss case: it refreshes once when an install fails. + auto resolved = resolve_semver(found.hit->coord.namespace_, + found.hit->coord.shortName, + spec.version, route, platform); + if (!resolved) { + d.reason = RefreshReason::VersionMiss; + d.shouldRefresh = true; + } + } + + if (!d.shouldRefresh) return d; // nothing to suppress + + // 7. Opt-outs, in order of authority: the user's flag, the machine's + // config, then the "we just did this" guard. + if (policy.offline) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedDisabled; return d; } + + // 8. Debounce — but never for a cold start, where there is nothing to + // debounce against and the build cannot proceed without the data. + if (d.reason != RefreshReason::IndexAbsent + && status.ageSeconds >= 0 && status.ageSeconds < policy.debounceSeconds) { + d.shouldRefresh = false; + d.reason = RefreshReason::SuppressedDebounce; + } + + return d; +} + +bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec) { + if (spec.isPath() || spec.isGit()) return false; + return coords_route_to_builtin(route, coords_of(spec.shortName, spec)); +} + +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject) { + RefreshDecision d; + d.subject = std::string(subject); + if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } + // Same debounce as the build path: two `mcpp add` typos in a row should not + // buy two multi-repo syncs, for the same reason a build with two missing + // packages does not. + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + if (status.present && status.ageSeconds >= 0 + && status.ageSeconds < policy.debounceSeconds) { + d.reason = RefreshReason::SuppressedDebounce; + return d; + } + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + return d; +} + +namespace { + +std::expected run_sync(const mcpp::xlings::Env& env, + std::string_view banner) { + auto before = mcpp::xlings::default_index_status(env, 0).rev; + mcpp::ui::status("Refreshing", banner); + int rc = mcpp::xlings::update_index(env, /*quiet=*/true); + if (rc != 0) + return std::unexpected(std::format("package index refresh failed (rc {})", rc)); + auto after = mcpp::xlings::default_index_status(env, 0).rev; + if (before && after && *before != *after) + mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); + else if (after) + mcpp::log::verbose("index", std::format("package index still at {}", *after)); + return {}; +} + +} // namespace + +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env) +{ + if (!d.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: {}", d.subject, reason_text(d.reason))); + return {}; + } + if (g_refreshed_this_process) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: already refreshed in this run", d.subject)); + return {}; + } + g_refreshed_this_process = true; + return run_sync(env, std::format("package index — `{}` {} (one-time)", + d.subject, reason_text(d.reason))); +} + +std::expected force_refresh(const mcpp::xlings::Env& env) { + if (mcpp::platform::env::offline_mode()) + return std::unexpected( + "offline mode is on — cannot refresh the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + g_refreshed_this_process = true; + return run_sync(env, "package index (requested)"); +} + +std::string staleness_note(const mcpp::xlings::Env& env) { + auto st = mcpp::xlings::default_index_status(env, 0); + if (!st.present) return "no local package index"; + return st.rev + ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) + : std::format("local index {}", age_phrase(st.ageSeconds)); +} + +} // namespace mcpp::pm diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index b6e26403..8d5aecf5 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -15,6 +15,7 @@ export module mcpp.pm.package_fetcher; import std; import mcpp.config; import mcpp.log; +import mcpp.platform; // env::offline_mode import mcpp.manifest; // xpkg_lua_identity_matches — descriptor identity gate import mcpp.pm.compat; import mcpp.pm.dep_spec; @@ -1004,6 +1005,17 @@ Fetcher::resolve_xpkg_path(std::string_view target, // 3. Install via xlings (primary path). if (autoInstall) { + // Offline is checked HERE, after the "already installed" fast paths + // above: a build whose packages are all present must stay fully + // functional offline — that is the whole point. Only an actual + // download attempt is refused, and it names the package so the user + // knows what to fetch rather than seeing a generic network error. + if (mcpp::platform::env::offline_mode()) { + return std::unexpected(CallError{ std::format( + "offline mode: `{}@{}` is not installed and cannot be downloaded\n" + " run without --offline (or unset MCPP_OFFLINE) to fetch it", + parsed.packageName, parsed.version) }); + } if (parsed.indexName == "xim") { mcpp::xlings::Env xlEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; // quiet=false: this only ever prints when a dependency is missing diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index d21a4140..df831d60 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "2026.7.30.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.7.30.3"; struct FingerprintInputs { Toolchain toolchain; diff --git a/src/xlings.cppm b/src/xlings.cppm index c94a5384..d5d50c66 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -307,6 +307,15 @@ std::optional find_sandbox_nasm(const Env& env); // ─── Index freshness ──────────────────────────────────────────────── +// How long a just-completed index sync suppresses the next one. +// +// Re-running a multi-repo sync because a SECOND package is also missing cannot +// help: the first sync already fetched everything upstream had, so a package +// still absent afterwards is absent upstream. Lives here, in the leaf module, +// because both users need it and the policy layer (mcpp.pm.index_refresh) +// imports this module — the reverse would be a cycle. +inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; + // Check whether the default mcpplibs index data exists and is fresh // (within ttlSeconds). // Returns true if index is present and fresh, false otherwise. @@ -346,14 +355,26 @@ void ensure_official_package_index_fresh(const Env& env, // Snapshot of a local index directory — computed without touching the // network, for `mcpp index status`. struct IndexStatus { - std::filesystem::path dir; // on-disk index directory - bool present; // pkgs/ tree exists locally - bool fresh; // refreshed within ttlSeconds - std::int64_t ageSeconds; // since last refresh marker, -1 if unknown + std::filesystem::path dir; // on-disk index directory + bool present; // pkgs/ tree exists locally + bool fresh; // refreshed within ttlSeconds + std::int64_t ageSeconds; // since last refresh marker, -1 if unknown + std::optional rev; // index CONTENT identity, nullopt if absent }; IndexStatus default_index_status(const Env& env, std::int64_t ttlSeconds); IndexStatus official_index_status(const Env& env, std::int64_t ttlSeconds); +// The index's own content identity, as written by xlings into +// `/.xlings-index-version` when it materializes the tree. +// +// OPAQUE BY CONTRACT. Observed values are a 7-char short sha for the artifact- +// distributed indexes (`mcpp-index-8d67478.tar.gz` → `8d67478`) but a DATE +// VERSION for the sub-indexes (`xim-index-awesome-2026.7.30.1.tar.gz` → +// `2026.7.30.1`). Never parse it, never assume a length — only compare it for +// equality and print it. Returns nullopt when the file is missing (a local +// `path` index has none) or blank; callers must degrade, never hard-fail. +std::optional index_revision(const std::filesystem::path& indexDir); + // ─── run_capture utility ──────────────────────────────────────────── std::expected run_capture(const std::string& cmd); @@ -394,6 +415,10 @@ std::filesystem::path index_refresh_marker(const std::filesystem::path& indexDir return indexDir / ".mcpp-index-updated"; } +std::filesystem::path index_version_file(const std::filesystem::path& indexDir) { + return indexDir / ".xlings-index-version"; +} + std::filesystem::path official_package_file(const Env& env, std::string_view packageName) { if (packageName.empty()) return {}; std::string name(packageName); @@ -442,7 +467,12 @@ void mark_index_refreshed(const std::filesystem::path& indexDir) { } void mark_known_indexes_refreshed(const Env& env) { - if (!env.projectDir.empty()) return; + // Project-scoped envs used to return early here, which left the global + // indexes unmarked whenever a project with custom `[indices]` triggered the + // sync — so `mcpp index status` reported "unknown" forever on exactly the + // machines that refresh most. The marker is advisory (it debounces and it + // dates the status line; it is NOT what decides whether a refresh is + // needed), so marking it from either env is both safe and more accurate. mark_index_refreshed(default_index_dir(env)); mark_index_refreshed(official_index_dir(env)); } @@ -459,6 +489,12 @@ bool is_index_dir_fresh(const std::filesystem::path& indexDir, std::int64_t ttlS auto now = std::filesystem::file_time_type::clock::now(); auto age = std::chrono::duration_cast(now - newest); + // A marker stamped in the FUTURE yields a negative age, which the plain + // `age < ttl` test read as "fresh" — and stayed fresh until the wall clock + // caught up, potentially for years. Future timestamps are routine: clock + // skew in containers/VMs, a tar that preserved mtimes, a restored CI cache. + // An unusable timestamp means "unknown", and unknown must mean stale. + if (age.count() < 0) return false; return age.count() < ttlSeconds; } @@ -470,7 +506,27 @@ std::int64_t index_age_seconds(const std::filesystem::path& indexDir) { auto newest = std::filesystem::last_write_time(marker, ec); if (ec) return -1; auto now = std::filesystem::file_time_type::clock::now(); - return std::chrono::duration_cast(now - newest).count(); + auto age = std::chrono::duration_cast(now - newest).count(); + return age < 0 ? -1 : age; // future stamp → "unknown", same rule as above +} + +// Defined inside the anonymous namespace so `index_status_for` (also here) can +// use it; the exported `index_revision` below forwards to it. +std::optional read_index_revision(const std::filesystem::path& indexDir) { + std::ifstream is(index_version_file(indexDir), std::ios::binary); + if (!is) return std::nullopt; + std::string body((std::istreambuf_iterator(is)), {}); + // The observed files carry no trailing newline, but trim anyway: this value + // is compared for equality and printed, and a stray \r from a Windows-side + // writer would otherwise turn "same rev" into "changed rev" every run. + auto isSpace = [](unsigned char c) { return std::isspace(c) != 0; }; + while (!body.empty() && isSpace(static_cast(body.back()))) + body.pop_back(); + std::size_t b = 0; + while (b < body.size() && isSpace(static_cast(body[b]))) ++b; + body.erase(0, b); + if (body.empty()) return std::nullopt; + return body; } IndexStatus index_status_for(const std::filesystem::path& indexDir, @@ -482,6 +538,7 @@ IndexStatus index_status_for(const std::filesystem::path& indexDir, .present = present, .fresh = is_index_dir_fresh(indexDir, ttlSeconds), .ageSeconds = index_age_seconds(indexDir), + .rev = read_index_revision(indexDir), }; } @@ -558,6 +615,12 @@ struct LineScan { } // anonymous namespace +// ─── Index identity ───────────────────────────────────────────────── + +std::optional index_revision(const std::filesystem::path& indexDir) { + return read_index_revision(indexDir); +} + // ─── run_capture ──────────────────────────────────────────────────── std::expected run_capture(const std::string& cmd) { @@ -1356,6 +1419,33 @@ bool is_official_package_index_fresh(const Env& env, } int update_index(const Env& env, bool quiet) { + // Offline is absolute: no caller gets to reach the network by going around + // the decision layer. Reported as success so a build that can still resolve + // everything locally proceeds — the caller that genuinely needed the data + // fails on its own missing answer, with a message that says so. + if (mcpp::platform::env::offline_mode()) { + mcpp::log::verbose("index", "offline mode: skipping index update"); + return 0; + } + + // Cross-process mutex. The sync rewrites a tree several concurrent mcpp + // processes read (parallel CI jobs on one MCPP_HOME, two terminals, a + // workspace fan-out), and it also writes the refresh markers. The BMI cache + // has had this guard for a while; the index never did. + // + // NON-BLOCKING BY DESIGN: whoever holds the lock is already doing the exact + // work we wanted, so waiting buys nothing and a queue of stalled builds is + // precisely the symptom this whole change exists to remove. Skipping is + // also why this must not be reported as failure. + std::error_code lockEc; + std::filesystem::create_directories(paths::index_data(env), lockEc); + auto lock = mcpp::platform::fs::FileLock::try_acquire(paths::index_data(env)); + if (!lock) { + mcpp::log::verbose("index", + "another process is refreshing the index — skipping this one"); + return 0; + } + std::string cmd = build_command_prefix(env) + " update 2>&1"; // The index sync is a network git operation; a single transient blip (DNS, // TLS reset, a mirror hiccup) otherwise fails a cold `mcpp self env` / @@ -1420,8 +1510,7 @@ void ensure_official_package_index_fresh(const Env& env, // `xlings update` per package: if the index was refreshed moments ago and // the package is STILL missing, upstream simply lacks it; re-pulling won't // help. (A package added upstream before this run lands in that one pull.) - constexpr std::int64_t kJustRefreshedSeconds = 120; - if (is_official_index_fresh(env, kJustRefreshedSeconds)) return; + if (is_official_index_fresh(env, kIndexRefreshDebounceSeconds)) return; if (!quiet) print_status("Refreshing", diff --git a/tests/e2e/173_index_refresh_policy.sh b/tests/e2e/173_index_refresh_policy.sh new file mode 100755 index 00000000..344a5071 --- /dev/null +++ b/tests/e2e/173_index_refresh_policy.sh @@ -0,0 +1,162 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# #315: `mcpp build` must not refresh the package index just because time +# passed. It refreshes when — and only when — the local index cannot answer. +# +# Every case here is HERMETIC: the builtin registry is fabricated on disk and +# every path that could reach the network is either offline-gated or debounced, +# so this test never downloads anything and never depends on upstream state. +# +# The load-bearing case is INV-3 (step 3 below): a namespace the registry cannot +# refute — `xim`, whose descriptors declare no namespace, or any third-party ns +# — must NOT count as a miss. If it did, every build with such a dependency +# would sync EVERY TIME, which is strictly worse than the hourly TTL this +# change removes. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +INDEX="$MCPP_HOME/registry/data/mcpplibs" +mkdir -p "$INDEX/pkgs/w" + +cat > "$INDEX/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", + namespace = "mcpplibs", + name = "widget", + description = "Present in the local index", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { ["1.2.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + macosx = { ["1.2.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + windows = { ["1.2.0"] = { url = "https://example.invalid/w.zip", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + }, +} +EOF +echo "abc1234" > "$INDEX/.xlings-index-version" + +# Back-date the refresh marker well past any debounce window: under the old +# TTL policy this alone was enough to trigger a network sync on every build. +: > "$INDEX/.mcpp-index-updated" +touch -d '30 days ago' "$INDEX/.mcpp-index-updated" 2>/dev/null \ + || touch -A -300000 "$INDEX/.mcpp-index-updated" 2>/dev/null || true + +mkdir -p "$TMP/proj/src" +cd "$TMP/proj" +echo 'int main() { return 0; }' > src/main.cpp + +manifest() { # manifest + cat > mcpp.toml < + if grep -qE "$2" <<< "$1"; then + printf '%s\n' "$1" + echo "FAIL: $3" + exit 1 + fi +} +expect() { # expect + if ! grep -qE "$2" <<< "$1"; then + printf '%s\n' "$1" + echo "FAIL: $3" + exit 1 + fi +} + +# ── 1. Steady state: a resolvable dependency costs no network ───────────── +# The descriptor is present and the constraint is satisfiable, so the decision +# is "resolvable locally" no matter how old the marker is. The build is run +# offline so that a regression here surfaces as a missing/extra decision line +# rather than as a download. +manifest 'widget = "1.2.0"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'widget@1.2.0: resolvable locally' \ + 'a locally-resolvable dependency should be reported as such' +refute "$log" 'Refreshing +package index|Updating +package index' \ + 'a 30-day-old marker must NOT trigger a refresh when the index can answer' + +# A SemVer constraint that the local versions satisfy is equally steady. +manifest 'widget = "^1.2"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'widget@\^1.2: resolvable locally' \ + 'a satisfiable constraint should not be a version miss' +refute "$log" 'Refreshing +package index' \ + 'a satisfiable constraint must not trigger a refresh' + +# ── 2. A real miss is reported as one, and offline suppresses the sync ──── +manifest 'absentpkg = "1.0.0"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'absentpkg@1.0.0: offline mode' \ + 'offline mode must suppress the refresh and say so' +refute "$log" 'Refreshing +package index' \ + 'offline mode must not reach the network' +# The failure the user actually sees carries the index identity and age, so +# "not found" and "your index is from last month" are distinguishable. +expect "$log" 'index: local index abc1234' \ + 'the resolution failure should name the index revision' +expect "$log" 'mcpp index update' \ + 'the resolution failure should point at the explicit refresh command' + +# ── 3. INV-3: an unrefutable miss is not a miss ─────────────────────────── +# Neither of these may trigger a refresh. `xim` descriptors declare no +# namespace so `(xim, nasm)` can never match the identity gate, and a +# third-party namespace is outside the identities mcpp defines at all. +for dep in 'xim.nasm = "2.16.03"' 'somevendor.thing = "1.0.0"'; do + manifest "$dep" + log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline + expect "$log" 'no index can refute this' \ + "a miss under an unrefutable namespace must be inconclusive: $dep" + refute "$log" 'Refreshing +package index' \ + "an unrefutable miss must never trigger a refresh: $dep" +done + +# ── 4. Debounce: a second miss inside the window does not re-sync ───────── +: > "$INDEX/.mcpp-index-updated" # "just refreshed" +manifest 'absentpkg = "1.0.0"' +log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline +expect "$log" 'index was just refreshed' \ + 'a miss moments after a sync must be debounced, not re-synced' +refute "$log" 'Refreshing +package index' \ + 'debounce must prevent the second sync' + +# ── 5. [index] auto_refresh = false is honoured ─────────────────────────── +touch -d '30 days ago' "$INDEX/.mcpp-index-updated" 2>/dev/null \ + || touch -A -300000 "$INDEX/.mcpp-index-updated" 2>/dev/null || true +printf '\n[index]\nauto_refresh = false\n' >> "$MCPP_HOME/config.toml" +log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline +expect "$log" 'auto_refresh = false' \ + 'the config opt-out must suppress the refresh and name itself' +refute "$log" 'Refreshing +package index' \ + 'auto_refresh = false must prevent the sync' + +# ── 6. `mcpp index update` refuses offline instead of silently no-opping ── +log=$(MCPP_OFFLINE=1 "$MCPP" index update 2>&1 || true) +expect "$log" 'offline mode is on' \ + 'an explicit index update must refuse audibly when offline' + +# ── 7. `mcpp index status` reports the index revision ───────────────────── +log=$(MCPP_OFFLINE=1 "$MCPP" index status 2>&1 || true) +expect "$log" 'revision' 'index status should have a revision column' +expect "$log" 'abc1234' 'index status should show the local index revision' + +echo "PASS: index refresh is driven by local resolvability, not by the clock" diff --git a/tests/unit/test_pm_index_refresh.cpp b/tests/unit/test_pm_index_refresh.cpp new file mode 100644 index 00000000..a4a912a9 --- /dev/null +++ b/tests/unit/test_pm_index_refresh.cpp @@ -0,0 +1,336 @@ +// The judgement table of mcpp.pm.index_refresh, row by row. +// +// This file is the contract for #315: a build refreshes the package index when +// — and only when — the local copy cannot answer. The rows worth staring at: +// +// InconclusiveNamespaceNeverRefreshes — the one that decides whether this +// whole change is an improvement or a regression. xim descriptors declare +// no `namespace`, so `(xim, nasm)` can never satisfy the identity gate; if +// that miss counted, every build with a xim dependency would sync EVERY +// TIME, which is worse than the hourly TTL being removed. +// ExactVersionAbsentIsNotAVersionMiss — version tables are per-OS, so "this +// host publishes no 1.2.3" is not "1.2.3 does not exist". +// ColdStartIgnoresDebounce — an absent index has nothing to debounce against. + +#include + +import std; +import mcpp.config; +import mcpp.pm.dep_spec; +import mcpp.pm.index_refresh; +import mcpp.pm.index_route; +import mcpp.pm.index_spec; +import mcpp.platform.axis; +import mcpp.xlings; + +namespace { + +using mcpp::pm::RefreshReason; + +// A throwaway MCPP_HOME whose layout matches what the builtin registry reader +// walks: /data//pkgs//.lua +struct FakeRegistry { + std::filesystem::path home; + + explicit FakeRegistry(std::string_view tag) { + home = std::filesystem::temp_directory_path() + / std::format("mcpp-index-refresh-{}-{}", tag, + std::filesystem::hash_value( + std::filesystem::path(std::string(tag)))); + std::filesystem::remove_all(home); + std::filesystem::create_directories(home / "data" / "mcpplibs" / "pkgs"); + } + ~FakeRegistry() { std::error_code ec; std::filesystem::remove_all(home, ec); } + + std::filesystem::path index_dir() const { return home / "data" / "mcpplibs"; } + + void publish(std::string_view name, std::string_view versions) { + auto dir = index_dir() / "pkgs" / std::string(1, name.front()); + std::filesystem::create_directories(dir); + std::ofstream(dir / std::format("{}.lua", name)) << std::format(R"( +package = {{ + spec = "1", + namespace = "mcpplibs", + name = "{}", + type = "package", + xpm = {{ + linux = {{ {} }}, + macosx = {{ {} }}, + windows = {{ {} }}, + }}, +}} +)", name, versions, versions, versions); + } + + // Age the refresh marker by writing it and back-dating it. + void mark_refreshed(std::chrono::seconds ago = std::chrono::seconds{0}) { + auto marker = index_dir() / ".mcpp-index-updated"; + std::ofstream(marker) << "ok\n"; + std::error_code ec; + std::filesystem::last_write_time( + marker, std::filesystem::file_time_type::clock::now() - ago, ec); + } + + void remove_index() { std::filesystem::remove_all(index_dir()); } + + mcpp::xlings::Env env() const { return mcpp::xlings::Env{ .home = home }; } + + mcpp::config::GlobalConfig config() const { + mcpp::config::GlobalConfig cfg; + cfg.registryDir = home; + return cfg; + } +}; + +mcpp::pm::DependencySpec version_dep(std::string_view ns, + std::string_view shortName, + std::string_view version) { + mcpp::pm::DependencySpec spec; + spec.namespace_ = std::string(ns); + spec.shortName = std::string(shortName); + spec.version = std::string(version); + spec.candidates.push_back(mcpp::pm::DependencyCoordinate{ + .namespace_ = std::string(ns), .shortName = std::string(shortName) }); + return spec; +} + +RefreshReason decide(const mcpp::pm::IndexRoute& route, + const mcpp::pm::DependencySpec& spec, + const mcpp::xlings::Env& env, + mcpp::pm::RefreshPolicy policy = {}) { + return mcpp::pm::decide_for_dependency( + route, spec.shortName, spec, env, + mcpp::platform::TargetPlatform::for_os("linux"), policy).reason; +} + +bool refreshes(const mcpp::pm::IndexRoute& route, + const mcpp::pm::DependencySpec& spec, + const mcpp::xlings::Env& env, + mcpp::pm::RefreshPolicy policy = {}) { + return mcpp::pm::decide_for_dependency( + route, spec.shortName, spec, env, + mcpp::platform::TargetPlatform::for_os("linux"), policy).shouldRefresh; +} + +constexpr std::string_view kOneVersion = R"(["1.2.0"] = { url = "u", sha256 = "s" },)"; + +} // namespace + +// ─── Rows that never reach the filesystem ──────────────────────────────── + +TEST(PmIndexRefresh, PathAndGitDependenciesNeverRefresh) { + FakeRegistry reg("srcdeps"); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + mcpp::pm::DependencySpec pathDep; + pathDep.path = "../sibling"; + EXPECT_EQ(decide(route, pathDep, reg.env()), RefreshReason::None); + + mcpp::pm::DependencySpec gitDep; + gitDep.git = "https://example.invalid/x.git"; + EXPECT_EQ(decide(route, gitDep, reg.env()), RefreshReason::None); +} + +TEST(PmIndexRefresh, OfflineSuppressesEvenAGenuineMiss) { + FakeRegistry reg("offline"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + mcpp::pm::RefreshPolicy offline; offline.offline = true; + + EXPECT_EQ(decide(route, spec, reg.env(), offline), RefreshReason::SuppressedOffline); + EXPECT_FALSE(refreshes(route, spec, reg.env(), offline)); +} + +TEST(PmIndexRefresh, OfflineDoesNotMisreportAResolvableDependency) { + // The opt-outs are applied AFTER the local analysis, so a dependency that + // resolved fine is reported as such even offline. Short-circuiting on the + // flag first would answer "offline mode" for every dependency — hiding + // exactly what an offline user is checking. + FakeRegistry reg("offlinesteady"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + mcpp::pm::RefreshPolicy offline; offline.offline = true; + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "1.2.0"), reg.env(), offline), + RefreshReason::None); +} + +TEST(PmIndexRefresh, AutoRefreshOptOutSuppressesAGenuineMiss) { + FakeRegistry reg("optout"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + mcpp::pm::RefreshPolicy off; off.autoRefresh = false; + + EXPECT_EQ(decide(route, spec, reg.env(), off), RefreshReason::SuppressedDisabled); +} + +// ─── INV-3: a miss must be refutable to mean anything ──────────────────── + +TEST(PmIndexRefresh, InconclusiveNamespaceNeverRefreshes) { + FakeRegistry reg("inconclusive"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // `xim` and any third-party namespace: the builtin registry is what would + // answer, but it is not authoritative for identities mcpp does not define, + // so "absent" here is not evidence of anything. + for (auto ns : { "xim", "somevendor" }) { + auto spec = version_dep(ns, "thing", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::SuppressedInconclusive) + << "namespace: " << ns; + EXPECT_FALSE(refreshes(route, spec, reg.env())) << "namespace: " << ns; + } +} + +TEST(PmIndexRefresh, LazyGitIndexNeverRefreshesTheGlobalOne) { + FakeRegistry reg("lazygit"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + indices["acme"] = mcpp::pm::IndexSpec{ .name = "acme", .url = "https://x/y" }; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // Served by a custom index; syncing the shared registry would not help. + auto spec = version_dep("acme", "widget", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::None); +} + +// ─── The three ways local state fails to answer ────────────────────────── + +TEST(PmIndexRefresh, PresentDescriptorIsTheSteadyStateAndCostsNothing) { + FakeRegistry reg("steady"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); // old, and irrelevant + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "1.2.0"), reg.env()), + RefreshReason::None); + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "^1.2"), reg.env()), + RefreshReason::None); +} + +TEST(PmIndexRefresh, MissingDescriptorTriggersARefresh) { + FakeRegistry reg("descmiss"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::DescriptorMiss); + EXPECT_TRUE(refreshes(route, spec, reg.env())); +} + +TEST(PmIndexRefresh, UnsatisfiableConstraintTriggersARefresh) { + FakeRegistry reg("vermiss"); + reg.publish("widget", kOneVersion); // only 1.2.0 exists + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "widget", "^9.9"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::VersionMiss); +} + +TEST(PmIndexRefresh, ExactVersionAbsentIsNotAVersionMiss) { + FakeRegistry reg("exactver"); + reg.publish("widget", kOneVersion); // only 1.2.0 exists + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // Version tables are per-OS: an exact pin the local table does not list may + // simply not be built for this host. Refreshing on it would fire on EVERY + // build for such a dependency. The install layer covers the real case. + auto spec = version_dep("mcpplibs", "widget", "9.9.9"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::None); +} + +// ─── Debounce ──────────────────────────────────────────────────────────── + +TEST(PmIndexRefresh, DebounceSuppressesTheSecondMissInAWindow) { + FakeRegistry reg("debounce"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::seconds{5}); // just synced + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::SuppressedDebounce); + EXPECT_FALSE(refreshes(route, spec, reg.env())); +} + +TEST(PmIndexRefresh, ColdStartIgnoresDebounce) { + FakeRegistry reg("coldstart"); + reg.mark_refreshed(std::chrono::seconds{5}); + reg.remove_index(); // marker gone with it; no local index at all + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "widget", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::IndexAbsent); + EXPECT_TRUE(refreshes(route, spec, reg.env())); +} + +// ─── decide_for_miss (the `mcpp add` half) ─────────────────────────────── + +TEST(PmIndexRefresh, DecideForMissHonoursTheSameOptOutsAndDebounce) { + FakeRegistry reg("formiss"); + reg.mark_refreshed(std::chrono::hours{99}); + auto env = reg.env(); + + mcpp::pm::RefreshPolicy allowed; + EXPECT_TRUE(mcpp::pm::decide_for_miss(allowed, env, "pkg").shouldRefresh); + + mcpp::pm::RefreshPolicy offline; offline.offline = true; + EXPECT_EQ(mcpp::pm::decide_for_miss(offline, env, "pkg").reason, + RefreshReason::SuppressedOffline); + + mcpp::pm::RefreshPolicy disabled; disabled.autoRefresh = false; + EXPECT_EQ(mcpp::pm::decide_for_miss(disabled, env, "pkg").reason, + RefreshReason::SuppressedDisabled); + + // Two `mcpp add` typos in a row must not buy two multi-repo syncs. + reg.mark_refreshed(std::chrono::seconds{5}); + EXPECT_EQ(mcpp::pm::decide_for_miss(allowed, env, "pkg").reason, + RefreshReason::SuppressedDebounce); +} + +TEST(PmIndexRefresh, SubjectEchoesWhatTheUserWrote) { + FakeRegistry reg("subject"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // The manifest key `xim.nasm` parses into the candidate (mcpplibs.xim, + // nasm); a diagnostic naming that spelling sends the reader hunting for a + // namespace they never typed. + auto spec = version_dep("mcpplibs.xim", "nasm", "2.16.03"); + auto d = mcpp::pm::decide_for_dependency( + route, "xim.nasm", spec, reg.env(), + mcpp::platform::TargetPlatform::for_os("linux"), {}); + EXPECT_EQ(d.subject, "xim.nasm@2.16.03"); +} diff --git a/tests/unit/test_xlings.cpp b/tests/unit/test_xlings.cpp index 01a54a1b..d7afd3d8 100644 --- a/tests/unit/test_xlings.cpp +++ b/tests/unit/test_xlings.cpp @@ -454,3 +454,70 @@ TEST(XlingsSeed, ArtifactAndSourceEmittedWhenSet) { std::string::npos) << text; std::filesystem::remove_all(dir); } + +// ─── Index identity + clock-skew hardening (#315) ──────────────────────── + +TEST(XlingsIndexFreshness, FutureMarkerIsStaleNotEternallyFresh) { + auto home = make_tempdir("mcpp-xlings-index-future"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + auto marker = dir / ".mcpp-index-updated"; + std::ofstream(marker) << "ok\n"; + // Clock skew in a container, a tar that preserved mtimes, a restored CI + // cache: a marker in the future used to yield a negative age, which read as + // "fresh" and stayed that way until the wall clock caught up. + std::error_code ec; + std::filesystem::last_write_time( + marker, std::filesystem::file_time_type::clock::now() + std::chrono::hours{72}, ec); + + mcpp::xlings::Env env{.home = home}; + auto st = mcpp::xlings::default_index_status(env, 3600); + + EXPECT_FALSE(st.fresh); + EXPECT_EQ(st.ageSeconds, -1); // unusable timestamp reports as unknown + + std::filesystem::remove_all(home); +} + +TEST(XlingsIndexRevision, ReadsTrimmedValueAndDegradesToNullopt) { + auto home = make_tempdir("mcpp-xlings-index-rev"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + + // Absent: a local `path` index has no such file — must not be an error. + EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + + // Present. The observed files carry no trailing newline, but a writer that + // added one (or a \r from Windows) must not change the value, or every run + // would look like the index had changed. + std::ofstream(dir / ".xlings-index-version") << " 8d67478\r\n"; + auto rev = mcpp::xlings::index_revision(dir); + ASSERT_TRUE(rev.has_value()); + EXPECT_EQ(*rev, "8d67478"); + + // Blank is not an identity. + std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "\n"; + EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + + // The value is OPAQUE: sub-indexes carry a date version, not a sha. + std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "2026.7.30.1"; + EXPECT_EQ(mcpp::xlings::index_revision(dir).value(), "2026.7.30.1"); + + std::filesystem::remove_all(home); +} + +TEST(XlingsIndexRevision, StatusCarriesTheRevision) { + auto home = make_tempdir("mcpp-xlings-index-rev-status"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + std::ofstream(dir / ".mcpp-index-updated") << "ok\n"; + std::ofstream(dir / ".xlings-index-version") << "abc1234"; + + mcpp::xlings::Env env{.home = home}; + auto st = mcpp::xlings::default_index_status(env, 3600); + + ASSERT_TRUE(st.rev.has_value()); + EXPECT_EQ(*st.rev, "abc1234"); + + std::filesystem::remove_all(home); +} From 37ac0ead7f22638592d26594db229ab0e31f21ac Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 30 Jul 2026 22:23:50 +0800 Subject: [PATCH 2/3] feat(doctor): show the package index revision + age in `mcpp why deps` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since the refresh is lazy, "why did I get this version" often answers to "because it is the newest one your local index knows" — which is unguessable without naming the index and its age. --- src/doctor.cppm | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/doctor.cppm b/src/doctor.cppm index 0105a062..6488422c 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -19,6 +19,7 @@ import mcpp.fetcher.progress; import mcpp.home; import mcpp.platform; import mcpp.platform.process; +import mcpp.pm.index_refresh; // staleness_note for `mcpp why deps` import mcpp.toolchain.detect; import mcpp.toolchain.msvc; import mcpp.toolchain.registry; @@ -417,6 +418,14 @@ export int why_report(const std::string& topic) { } } if (all || topic == "deps") { + // Which index answered, and how stale it is. Since #315 a build only + // refreshes on a resolution miss, so "why did I get this version" + // frequently has "because that is the newest one your local index + // knows" as its answer — which is unguessable without this line. + if (auto cfgW = mcpp::config::load_or_init(/*quiet=*/true)) { + std::println("package index: {}", + mcpp::pm::staleness_note(mcpp::config::make_xlings_env(*cfgW))); + } std::println("dependencies (mcpp.lock):"); std::ifstream in(ctx->projectRoot / "mcpp.lock"); if (!in) { From 4918286657fccbb143397ecb8501c1f3c304fb70 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 30 Jul 2026 22:24:32 +0800 Subject: [PATCH 3/3] docs(plan): record where the implementation deviated from the plan, and why --- .../2026-07-30-issue315-implementation-plan.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.agents/docs/2026-07-30-issue315-implementation-plan.md b/.agents/docs/2026-07-30-issue315-implementation-plan.md index 039712d1..a4ccfb99 100644 --- a/.agents/docs/2026-07-30-issue315-implementation-plan.md +++ b/.agents/docs/2026-07-30-issue315-implementation-plan.md @@ -5,6 +5,7 @@ 关联：#315 目标版本：**2026.7.30.3**（基线 `d79ab00` / 2026.7.30.2） 形态：**单 PR**，内部按 A–F 六段推进，每段自带验收。 +状态：**已完成**（PR #320）。实施中偏离计划的地方记在 §G。 --- @@ -136,6 +137,19 @@ std::string_view reason_text(RefreshReason); --- +## G. 实施中相对本计划的偏离（都是实现时才看清的约束） + +| 计划 | 实际 | 原因 | +|---|---|---| +| `--offline` 走 `BuildOverrides` 透传 | CLI 全局 flag → **写 `MCPP_OFFLINE` 环境变量**，各处按需读 | 消费者横跨 pm（索引/包）与 build（工具链），透传要改三个子系统的签名；env 又恰好让 `--offline` 与 `MCPP_OFFLINE=1` 成为字面上的同一个开关 | +| `offline_mode()` 放 `mcpp.pm.index_refresh` | 放 **`mcpp.platform.env`**（叶子模块） | `mcpp.pm.index_route` 已 import `mcpp.fetcher`，pm 里的 helper 会让 fetcher 反向依赖 pm ⇒ **模块环** | +| 判据顺序：offline 最先短路 | **本地分析在前，opt-out 在后** | 先短路会把「解析得好好的依赖」也报成 `offline mode`，而那恰恰是 offline 用户要确认的事。多出的成本只是本地文件读 | +| `decide_for_miss(policy, subject)` | 加 `env` 参数，**同样吃去抖** | 连着两次 `mcpp add` 打错字不该换来两次多仓库同步，理由与构建路径完全一致 | +| `mcpp update` 无条件强制刷新 | 工程里**没有走共享 registry 的依赖时跳过** | 纯 path/git 依赖的工程刷全局索引毫无意义；顺带让 e2e 23/24 不必平白联网 | +| `index_revision` 直接定义在实现区 | 拆成匿名命名空间里的 `read_index_revision` + 外层转发 | 实现区那一大段在**匿名命名空间**内，直接定义会得到内部链接 ⇒ 导出声明未定义（link 期才暴露） | +| P1 单列一期 | **全部并入 P0** | advisory、`index status` rev 列、S4 打标、docs 都只有几十行，分期反而让「变懒」上线时缺少配套的可观测性 | +| 计划未列 | 追加 `mcpp why deps` 的 `package index: ()` 行 | 刷新变懒后，「为什么解析到这个版本」的答案常常是「因为它是你本地索引已知的最新版」 | + ## 风险与回退 | 风险 | 处置 |