diff --git a/.agents/docs/2026-07-30-issue315-implementation-plan.md b/.agents/docs/2026-07-30-issue315-implementation-plan.md new file mode 100644 index 00000000..a4ccfb99 --- /dev/null +++ b/.agents/docs/2026-07-30-issue315-implementation-plan.md @@ -0,0 +1,160 @@ +# 索引刷新策略收敛 — 实施计划 + +日期：2026-07-30 +设计：`.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md` +关联：#315 +目标版本：**2026.7.30.3**（基线 `d79ab00` / 2026.7.30.2） +形态：**单 PR**，内部按 A–F 六段推进，每段自带验收。 +状态：**已完成**（PR #320）。实施中偏离计划的地方记在 §G。 + +--- + +## 0. 基线核对（已完成） + +| 事实 | 位置（@ d79ab00） | +|---|---| +| TTL 站点 | `src/build/prepare.cppm:1390-1433` | +| 每个依赖已自带**有序候选** | `DependencySpec.candidates`（`src/pm/dep_spec.cppm:46`）—— 与 `mcpp add` 用的是同一份 | +| 存在性查找 | `mcpp::pm::lookup_descriptor(route, candidates)`（`src/pm/index_route.cppm:150`） | +| route 工厂 | `prepare.cppm:1548`（`index_route(cfg)`）、`commands.cppm:151-152` | +| 版本可满足性 | `mcpp::pm::resolve_semver(ns, short, req, route, platform)`（`src/pm/resolver.cppm:96`），失败即不可满足 | +| 索引目录/marker/age | `xlings.cppm:381-490`（**匿名命名空间，未导出**） | +| 索引状态（已导出） | `default_index_status` / `official_index_status` → `IndexStatus{dir,present,fresh,ageSeconds}` | +| 索引内容 rev | `/.xlings-index-version`（实测 mcpplibs=`8d67478`，xim-pkgindex=`ebf4020`）—— **mcpp 零引用** | +| 跨平台锁原语 | `mcpp::platform::fs::FileLock::try_acquire(dir)`（`src/platform/fs.cppm:60`，`bmi_cache.cppm:241` 已在用） | +| 索引数据根（已导出） | `mcpp::xlings::paths::index_data(env)` | + +--- + +## A. 探针（先跑，不写代码） + +- **P-1 `.xlings-index-version` 更新语义**：clean-room `MCPP_HOME`，走一次真实 `xlings update`， + 比对刷新前后该文件的值与格式（是否 7 位短 sha、有无尾随空白、artifact/git 两通道是否都写）。 + **降级预案**：若某通道不写 → `index_revision()` 返回 nullopt，advisory 只显示 age，**gate 不受影响**。 +- **P-2 INV-3 误报实测**：对一个同时含 mcpplibs 依赖与 xim 依赖的工程，打印每个依赖的 + `RefreshDecision`，确认 xim 落 `SuppressedInconclusive` 而非 `DescriptorMiss`。 + P-2 依赖 A 段代码，故实际在 B 段末尾以单测 + 一次真实工程 `-v` 运行完成。 + +**验收**：P-1 结论写回设计文档 §10；P-2 由单测 + e2e #6 承接。 + +--- + +## B. 策略模块（新增 `src/pm/index_refresh.cppm`） + +**导出面** + +``` +enum class RefreshReason { None, IndexAbsent, DescriptorMiss, VersionMiss, + SuppressedDebounce, SuppressedOffline, SuppressedInconclusive }; +struct RefreshPolicy { bool offline; std::int64_t debounceSeconds = 120; }; +struct RefreshDecision { bool shouldRefresh; RefreshReason reason; std::string subject; }; + +RefreshDecision decide_for_dependency(const IndexRoute&, const DependencySpec&, + const xlings::Env&, const platform::PlatformKey&, + const RefreshPolicy&); +std::expected apply(const RefreshDecision&, const xlings::Env&, bool quiet); +RefreshPolicy policy_from_env(const config::GlobalConfig&, bool offlineFlag); +std::string_view reason_text(RefreshReason); +``` + +**判据顺序（短路，单测逐行锁）** + +1. `spec.isPath() || spec.isGit()` → `None` +2. `policy.offline` → `SuppressedOffline` +3. 无候选路由到**内置 registry**（`find_for_ns` 为 null 或 `is_builtin`）→ `None` + （项目 `path`/自定义 git 索引：刷全局索引对它无用，保持现状语义） +4. `lookup_descriptor` 未命中且 `!conclusive` → `SuppressedInconclusive`（**INV-3**） +5. 默认索引 `pkgs/` 不存在 → `IndexAbsent` → 刷 +6. 未命中 → `DescriptorMiss` → 刷 +7. 命中 + `is_version_constraint(spec.version)` + `resolve_semver` 失败 → `VersionMiss` → 刷 +8. 上面判刷但 `0 <= age < debounceSeconds` → `SuppressedDebounce` + +**依赖方向**：`index_refresh` → {`index_route`, `resolver`, `dep_spec`, `xlings`, `config`, +`platform.fs`, `ui`, `log`}。`resolver` 已 import `index_route`，无环。 + +**验收**：`tests/unit/test_pm_index_refresh.cpp` 覆盖 8 条判据 + 精确版本不判 VersionMiss。 + +--- + +## C. xlings 侧最小增补（`src/xlings.cppm`） + +1. `IndexStatus` 增字段 `std::optional rev`（读 `.xlings-index-version`，trim 空白； + 缺失/空 → nullopt）。→ `mcpp index status` 与 advisory 免费获得。 +2. 新导出 `std::optional index_revision(const std::filesystem::path& indexDir)`。 +3. **S1**：`is_index_dir_fresh` 中 `age < 0` → 判 **stale**（未来时间戳不得等于永远新鲜）。 +4. **S3**：`update_index` 内部用 `FileLock::try_acquire(paths::index_data(env))` 包裹； + 拿不到锁 → 记 verbose 日志并**返回 0（跳过，不报错、不阻塞）**。 +5. **S4**：`mark_known_indexes_refreshed` 去掉 `projectDir` 早退，改为项目模式下也打标 + （仅供 advisory）。 + +**验收**：`tests/unit/test_xlings.cpp` 增负 age、rev 读取三态（缺失/空/带换行）。 + +--- + +## D. 五站点收敛 + +| 站点 | 动作 | +|---|---| +| `prepare.cppm:1390-1433` | 整段替换：遍历 `m->dependencies` 调 `decide_for_dependency`；命中即 `apply` 一次（进程内 `bool` once），reason 进 `-v` 日志 | +| `commands.cppm:167`（add） | 改为 `decide_*` + `apply`，删除本地 TTL 判断 | +| `xlings.cppm:1400`（xim 门） | 保留判据，`kJustRefreshedSeconds` 改由 `RefreshPolicy::debounceSeconds` 提供默认值，注释指向策略模块 | +| `package_fetcher.cppm:1060` | 保留，输出走统一句式 | +| `index_management.cppm:29`（search） | 保留 TTL，**加注释说明这是唯一允许时间驱动的站点** | + +**验收**：`grep -n "is_index_fresh" src/` 只剩 search 一处 + xlings 内部实现。 + +--- + +## E. D6 + 表达面 + +1. **`mcpp update` 不再空转**（`commands.cppm:403-430`）：先强制刷索引（不看 TTL/debounce， + `offline` 下拒绝并说明），再清 lock 条目，输出 `mcpplibs 8d67478 → a1b2c3d` 或 `already at …`。 +2. **`--offline`**：`build` / `run` / `test` / `update` 注册；语义 = 本次调用不发起任何网络 + （索引 + 包 + 工具链自动安装）。 +3. **`MCPP_OFFLINE=1`**：等价 env；`MCPP_NO_AUTO_INSTALL` 保留为兼容别名（等价于 offline 的 + 工具链子集），help/docs 标注 deprecated。 +4. **`[index] auto_refresh = true|false`**：`config.cppm` 读取，默认 true。 +5. **可解释输出**：刷新一行说明主语与原因；`-v` 打印跳过原因与索引 rev/age； + offline+miss 的结构化错误含 rev + age + `mcpp index update`。 +6. `mcpp index status` 增 rev 列。 + +**优先级**：flag > env > config。 + +--- + +## F. 测试 + 版本 + 交付 + +- 新 e2e `tests/e2e/173_index_refresh_policy.sh`，9 个场景（设计 §8）。 + **反向断言写法**：`out=$(...); echo "$out" | grep -q X && { echo FAIL; exit 1; }` + （`! cmd | grep` 在 errexit 下被豁免，永不失败）。 +- 全量本地回归：`mcpp test`（单测）+ 受影响 e2e（09/14/44/105/169 等涉及索引/离线的用例）。 +- 版本 `2026.7.30.3`：只改 `mcpp.toml` + `src/toolchain/fingerprint.cppm`； + **bootstrap pin（`.xlings.json` / `ci-fresh-install.yml MCPP_PIN`）不动**。 + `bash .github/tools/check_version_pins.sh` 必须过。 +- 单 PR → CI 全绿 → bypass squash 合入。 +- 合入后：release 四平台 → 镜像 xlings-res（gh+gtc）→ xim-pkgindex PR → `xlings install mcpp` + 真装验证 → bootstrap pin bump（独立 commit）。 + +--- + +## G. 实施中相对本计划的偏离（都是实现时才看清的约束） + +| 计划 | 实际 | 原因 | +|---|---|---| +| `--offline` 走 `BuildOverrides` 透传 | CLI 全局 flag → **写 `MCPP_OFFLINE` 环境变量**，各处按需读 | 消费者横跨 pm（索引/包）与 build（工具链），透传要改三个子系统的签名；env 又恰好让 `--offline` 与 `MCPP_OFFLINE=1` 成为字面上的同一个开关 | +| `offline_mode()` 放 `mcpp.pm.index_refresh` | 放 **`mcpp.platform.env`**（叶子模块） | `mcpp.pm.index_route` 已 import `mcpp.fetcher`，pm 里的 helper 会让 fetcher 反向依赖 pm ⇒ **模块环** | +| 判据顺序：offline 最先短路 | **本地分析在前，opt-out 在后** | 先短路会把「解析得好好的依赖」也报成 `offline mode`，而那恰恰是 offline 用户要确认的事。多出的成本只是本地文件读 | +| `decide_for_miss(policy, subject)` | 加 `env` 参数，**同样吃去抖** | 连着两次 `mcpp add` 打错字不该换来两次多仓库同步，理由与构建路径完全一致 | +| `mcpp update` 无条件强制刷新 | 工程里**没有走共享 registry 的依赖时跳过** | 纯 path/git 依赖的工程刷全局索引毫无意义；顺带让 e2e 23/24 不必平白联网 | +| `index_revision` 直接定义在实现区 | 拆成匿名命名空间里的 `read_index_revision` + 外层转发 | 实现区那一大段在**匿名命名空间**内，直接定义会得到内部链接 ⇒ 导出声明未定义（link 期才暴露） | +| P1 单列一期 | **全部并入 P0** | advisory、`index status` rev 列、S4 打标、docs 都只有几十行，分期反而让「变懒」上线时缺少配套的可观测性 | +| 计划未列 | 追加 `mcpp why deps` 的 `package index: ()` 行 | 刷新变懒后，「为什么解析到这个版本」的答案常常是「因为它是你本地索引已知的最新版」 | + +## 风险与回退 + +| 风险 | 处置 | +|---|---| +| INV-3 接错 → 每次构建都刷 | 单测 + e2e #6 双闸；P-2 真实工程实测 | +| `.xlings-index-version` 语义不符 | gate 不依赖 rev，最坏只损失 advisory 精度 | +| S3 锁在 Windows 行为不一致 | 走 `platform/fs.cppm` 封装；拿不到锁=跳过而非阻塞 | +| 语义变化引发「拉不到新版本」误报 | `mcpp update` 变成真入口 + advisory 提示 + docs 明写 | diff --git a/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md b/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md new file mode 100644 index 00000000..f91ff58f --- /dev/null +++ b/.agents/docs/2026-07-30-issue315-index-refresh-policy-design.md @@ -0,0 +1,423 @@ +# 索引刷新策略收敛：从「时间驱动」到「解析驱动」— 设计 + +日期：2026-07-30 +状态：**已实施**（2026.7.30.3）。§10 的两条探针已跑，结果与实现修正记在那一节 +关联：#315（`mcpp build/test/run` 手动刷新包索引而不是自动刷新） +建议目标版本：**2026.7.31.1** + +--- + +## 1. 摘要 + +issue #315 报的症状真实：网络差时 `mcpp build` 会停在 `Updating package index (auto-refresh)`。 + +但这不是「功能缺失」。offline-first / miss-triggered 的刷新策略**已经在仓库里了**，而且写得比 +issue 建议的更完整（`src/xlings.cppm:1400-1431`，含 120s 抖动抑制），其注释甚至点名了这个 +issue 描述的症状（Termux first-run / build hang）。真正的缺陷是： + +| 编号 | 缺陷 | 层级 | +|---|---|---| +| **D1** | 同一个决策（要不要刷索引）在 **5 处各推导一遍**，其中 `prepare.cppm:1300-1311` 推出的策略与 `xlings.cppm:1405` 写死的策略**互相矛盾** | 策略 | +| **D2** | 刷新判据是 **marker mtime（时间轴）**，而「索引能否回答问题」是内容属性。**而且索引自带的内容 revision（`.xlings-index-version`）躺在磁盘上，mcpp 零引用** | 状态/身份 | +| **D3** | `is_index_dir_fresh` 对**未来时间戳**判为「永远新鲜」（`age < 0 < ttl`）；时钟回拨 / tar 保留 mtime / CI cache 恢复都能触发 | 稳定性 | +| **D4** | 无 `--offline` / `MCPP_OFFLINE` 逃生舱；`MCPP_NO_AUTO_INSTALL` 只管**工具链**，不管索引也不管包 | UX | +| **D5** | 刷与不刷的**原因**从不可见；只有一行没有主语的 `Updating package index` | UX | +| **D6** | `mcpp update` 是**空操作**：删 mcpp.lock 条目后叫用户去 `mcpp build`，而 lock 从来没被 build 读过 → 行为影响为零 | 语义 | + +D1/D2 是根。本设计做 D1–D6 的收敛，**不做** 「让 `mcpp.lock` 成为构建期解析输入」（§9，必须单独立项）。 + +--- + +## 2. 现状机制（逐处核过） + +### 2.1 五个刷新站点，四种策略 + +| # | 站点 | 判据 | 评价 | +|---|---|---|---| +| 1 | `mcpp add`
`pm/commands.cppm:160-175` | 先 `lookup_descriptor` 查本地 → **确凿未命中** + 涉及内置 registry + TTL 过期 → 刷一次 → 重查 | 正确样板 | +| 2 | xim 安装前置门
`xlings.cppm:1400` | **纯 miss**：包文件本地缺失才刷；120s 内刚刷过则不重刷 | 最完整 | +| 3 | 安装失败重试
`package_fetcher.cppm:1060-1075` | **版本 miss**：install 失败 → 刷一次 → 重试 | 补 #2 的漏（文件在、版本旧） | +| 4 | **`mcpp build/run/test`**
`prepare.cppm:1300-1311` | **纯 TTL**：根 manifest 有任一依赖路由到内置索引 + marker 超 `searchTtlSeconds`（默认 3600s） | ← 本 issue | +| 5 | `mcpp search`
`index_management.cppm:29` | TTL | 合理，search 语义即查上游 | + +站点 4 已有两道门（`!m->dependencies.empty()` 与 `usesBuiltinIndex(spec)`），所以「无条件刷新」 +的说法不成立；但第三道判据是纯时间，只要有**一个** registry 依赖，门就形同不设。 + +### 2.2 为什么这次网络往返在稳态下是纯浪费 + +**所有解析输入都是本地文件**： + +- `resolve_semver`（`pm/resolver.cppm:96`）经 `IndexRoute::read()` 读 xpkg.lua，从描述符解析可用 + 版本集 —— 纯本地文件解析，本身不联网。 +- 包已装时 `fallback::is_install_complete(verdir)` 直接早退（`package_fetcher.cppm:990`）。 + +即「本地能否回答」是**不需要网络、确定性、可单测**的判定。站点 4 却用一个与之无关的信号 +（marker 有多老）决定是否付网络代价。 + +### 2.3 代价被两个细节放大 + +- `is_index_fresh` 查的是 **default（mcpplibs）索引**的 marker，但 `update_index` 执行的是 + `xlings update` —— **同步所有索引仓库**。一个 marker 过期换一次全量同步。 +- `update_index` 有 **3 次尝试 + 2s/4s 线性退避**（`xlings.cppm:1365-1378`）。网络差时不是等 + 一次超时，是三次。 + +### 2.4 分层体检：真正的缺陷不全在策略层 + +| 层 | 事实 | 结论 | +|---|---|---| +| **L1 传输** | `update_index` = `xlings update`，刷**全部**索引仓库，mcpp 侧无 per-index / per-package 能力 | 粒度不可选 | +| L1 | **`mcpp index update ` 的 `name` 是空承诺**：全局段无条件全量刷（`index_management.cppm:121-124` 硬编码 `"all index repos"`），filterName 只在项目自定义索引循环里过滤，而循环体 `break` + `ensure_project_index_dir` 一次处理全部 | CLI help 承诺了实现做不到的事 | +| L1 | **索引刷新无任何并发锁**。`bmi_cache` 有 flock，`platform/fs.cppm` 已提供跨平台原语（flock / LockFileEx），索引一个都没用 | 并行 CI job 共享 MCPP_HOME、多终端并行构建 → 并发同步 + 并发写 marker | +| **L2 状态** | 唯一新鲜度信号是 `.mcpp-index-updated` 的 **mtime** | 信号选错轴（D2） | +| L2 | **索引自带内容 revision**：`/.xlings-index-version`（实测 `mcpplibs`→`8d67478`，`xim-pkgindex`→`ebf4020`），**mcpp 全仓库零引用** | 内容轴的载体现成，见 §3 INV-2 | +| L2 | `LockedIndex.rev` 只是把用户在 `[indices]` 写的 pin 抄进 lock（`prepare.cppm:3841`），**不是观测值**；内置索引连 rev 概念都没有 | 无内容寻址的索引身份 | +| L2 | **派生缓存烙绝对路径**：`.xlings-index-cache.json` 每条都带 `"path":"<绝对路径>"`。现场证据：本机 `~/.mcpp` 那份的 path 全部指向另一个早先会话的临时 MCPP_HOME | 目录一搬（CI cache 恢复 / MCPP_HOME 迁移 / AUR `/opt` 布局）缓存即坏 | +| L2 | **为上一条写的检测器没接线**：`official_index_cache_matches_package_file` 只被 `is_official_package_index_fresh` 调用，而后者在 production 代码里**零调用**，只有 `tests/unit/test_xlings.cpp` 在用；跑在构建路径上的 `ensure_official_package_index_fresh` 只查 `exists(pkg)` | 检测器写了、测了，没装到产线上 | +| L2 | `mark_known_indexes_refreshed`（`xlings.cppm:444`）在 `env.projectDir` 非空时整段早退 | 项目 env 的刷新永不打标 | +| **L3 策略** | D1 | 见 §3 INV-1 | +| **L4 表达** | `index update/status/pin/unpin` 齐全，`status` 是真 offline（设计正确） | 骨架对 | +| L4 | **`mcpp update` 空操作**（D6，`pm/commands.cppm:403-430`） | 唯一语义为「更新依赖」的命令什么都不做 | + +**一处过时论断的更正**：先前评估称「`xlings update` 是 git-only，Windows no-git 死锁（PR #114） +每小时埋进构建关键路径」。**已过时** —— `IndexSpec` 已有 `source = "auto" | "artifact" | "git"` +与 `artifact` base URL（`config.cppm:40-41`，#269 / xlings ≥ 0.4.68），索引以 **artifact 分发** +为主；本机 `mcpplibs/` 与 `xim-pkgindex/` **都没有 `.git` 目录**，证实了这一点。风险仍在 +（`auto` 会回落 git），但性质从「必经」降为「可能」，Windows 论点相应下调。 + +--- + +## 3. 架构判断（四条不变量） + +### INV-1 刷新策略只有一个真源 + +D1 是典型的「同一决策两处推导」——本仓库已为这种债付过两次代价（feature 请求集解析×激活两处 +不自洽 #242；`build_program` 重推 musl→static 未复用 `prepare`）。加语义时它必然变成构建失败。 +故新增单一策略模块，5 个站点全部改为调用它，**任何站点不得再自行判断 TTL**。 + +### INV-2 gate 用内容，且直接采用已有的 `.xlings-index-version` + +刷新的**必要性**由「解析器能否用本地索引回答」决定；索引的**身份**取 `.xlings-index-version` +（7 位 rev），marker mtime 降级为纯 debounce 计时器。 + +这条同时消解一个既有故障模式：**marker 假新鲜导致上游有新版本却拉不到**（llvm 22.1.8 那次）。 +假新鲜在新模型下不再能压制一次必要的刷新，只能压制一句提示。方向与 #311/#317 的教训一致： +**mtime 不可靠，判等走内容/键。** + +关键是**不需要新建机制** —— rev 文件已存在，读一个 7 字节文件即可。它一次性支撑四件事： +新鲜度显示、`mcpp index status` 的 rev 列、未来 lock 记录**观测到的** index rev、 +「为什么解析结果变了」的诊断。（前置条件见 §10 探针 P-1。） + +### INV-3 不可证伪就不刷 + +`IndexRoute::authoritative_for()`（PR#307）已定义「miss 何时能当作不存在的证明」：lazy git 索引 +未克隆、第三方 namespace、**xim 描述符不写 `namespace` 故 `(xim, x)` 永不匹配身份门** —— 这三类 +miss 不可证伪。 + +**这是本方案最容易踩的坑**：若把「查不到」直接当「要刷」，则任何带 xim 依赖的项目会**每次构建都 +判 miss、每次都刷**，比现状更糟。必须复用 `authoritative_for`：不可证伪 → fall-through，**不刷**。 + +### INV-4 策略是环境属性，不是项目属性 + +刷新策略归**全局 config**（`~/.mcpp/config.toml`），不进 `mcpp.toml`。同一项目在公司内网、家里、 +CI 上该允许不同策略；写进 manifest 会让项目不可移植，还会无谓拓宽 lock/指纹面。 + +--- + +## 4. 目标设计 + +### 4.1 新模块 `src/pm/index_refresh.cppm`（module `mcpp.pm.index_refresh`） + +依赖方向：`index_refresh` → {`index_route`, `resolver`, `xlings`, `config`}。`resolver` 已 import +`index_route`，无环。 + +```cpp +export namespace mcpp::pm { + +enum class RefreshReason { + None, // 本地可解析 —— 零网络 + IndexAbsent, // 索引 pkgs 目录不存在（冷启动） + DescriptorMiss, // 描述符本地缺失，且该 ns 可证伪 + VersionMiss, // 描述符在，但 SemVer 约束在本地版本集内无解 + SuppressedDebounce, // 刚刷过（ env > config，一次解析好往下传 + bool offline = false; + std::int64_t debounceSeconds = 120; // 复用 xim 门的既有语义 +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" —— 进日志与错误文案 +}; + +// 纯函数、零副作用、零网络：可单测，判据表就是它的契约。 +RefreshDecision decide_for_dependency(const IndexRoute& route, + const DependencyCoordinate& coord, + std::string_view versionReq, + const mcpp::xlings::Env& env, + const RefreshPolicy& policy); + +// 执行 + 可解释输出 + 打标；进程内保证一次构建最多刷一次。 +std::expected apply(const RefreshDecision&, const mcpp::xlings::Env&); + +// 索引身份（`.xlings-index-version`），缺失返回 nullopt（本地 path 索引没有它）。 +std::optional index_revision(const std::filesystem::path& indexDir); + +// advisory：仅在解析失败与 `index status` 使用，不参与 gate。 +struct Staleness { std::optional days; std::optional rev; }; +Staleness staleness_hint(const mcpp::xlings::Env&); + +} // namespace mcpp::pm +``` + +判据表（`decide_for_dependency` 的全部行为，按序短路）： + +| 条件 | 结果 | +|---|---| +| `policy.offline` | `SuppressedOffline` → 不刷 | +| `!route.authoritative_for(ns)` | `SuppressedInconclusive` → 不刷（INV-3） | +| `index_pkgs_dir` 不存在 | `IndexAbsent` → 刷 | +| `route.read(coord) == nullopt` | `DescriptorMiss` → 刷 | +| `is_version_constraint(versionReq)` 且 `resolve_semver(...)` 失败 | `VersionMiss` → 刷 | +| 上面命中 miss 但距上次刷新 < `debounceSeconds` | `SuppressedDebounce` → 不刷 | +| 其余 | `None` → 不刷 | + +两条**简洁性**决定，让这张表比初稿更小也更对： + +- **VersionMiss 只对 SemVer 约束成立**。精确版本（`is_version_constraint == false`）不走 + `resolve_semver`，其 miss 由安装层已有的 miss 驱动兜住（`is_install_complete` 早退 → 失败则 + 刷一次重试，站点 3）。这同时避开一个陷阱：**版本表是 per-OS 的**，"本机没有" ≠ "不存在" + （`pm/commands.cppm:46-48` 已记录该判据），在此处硬判会造成每次构建的误触发。 +- **VersionMiss 直接复用 `resolve_semver` 的失败**，不复制约束求解逻辑 —— 否则又是一处 D1。 + +### 4.2 站点改造 + +| 站点 | 改法 | +|---|---| +| `prepare.cppm:1300-1311` | 删掉 `usesBuiltinIndex` + `is_index_fresh` 整段；对根 manifest 每个依赖调 `decide_for_dependency`，任一 `shouldRefresh` 则 `apply` 一次（进程内 once） | +| `pm/commands.cppm:167`（add） | 语义已对，改为调用 `decide_*` 以消除重复推导 | +| `xlings.cppm:1400`（xim 门） | 保留（服务 xim 索引、判据同构），debounce 常量改由 `RefreshPolicy` 提供，不再各写一个 120 | +| `package_fetcher.cppm:1060`（失败重试） | 保留，reason 归一为 `VersionMiss`，走统一输出 | +| `index_management.cppm:29`（search） | 保留 TTL —— 代码里**显式注明**这是唯一允许时间驱动的站点及理由 | + +### 4.3 D6：让 `mcpp update` 真的做事 + +`mcpp update` 是唯一语义为「我要更新依赖」的命令，今天却是空操作。改为： + +1. **强制刷索引**（显式意图 → 不看 TTL、不看 debounce；`offline` 下拒绝并说明）； +2. 清除 lock 条目（保留现有行为）； +3. 输出实际发生了什么：`Updated mcpplibs 8d67478 → a1b2c3d`（无变化时说"already at …"）。 + +这样「我知道上游有新版」有了名正言顺的入口，`build` 侧不必再开洞（§5）。 + +### 4.4 稳定性修复 + +- **S1（D3）** `is_index_dir_fresh`：`age.count() < 0` → 视为 **stale**。未来时间戳来自时钟回拨、 + 容器时间、tar 解包保留 mtime、CI cache 恢复；当前实现让索引「永远新鲜」直到墙钟追上。 +- **S2** 刷新失败**不再直接失败构建**。`apply()` 出错时：本地仍能解析 → 降级 warning 继续； + 只有「本地无答案 + 刷新也失败」才 hard error，并合并 advisory + `mcpp index update` 建议。 + （现状不统一：TTL 刷新失败被静默吞掉，project index clone 失败直接 hard error。） +- **S3** 索引刷新加**跨进程互斥**：复用 `platform/fs.cppm` 的 flock / LockFileEx（`bmi_cache` + 的既有用法），锁文件置于 index data 根。拿不到锁 → 不排队、不报错，直接跳过这次刷新 + （别人正在刷；本次继续用本地解析，失败再按 S2 处理）。这条同时消掉并发写 marker 的竞态。 +- **S4** `mark_known_indexes_refreshed` 在 `projectDir` 非空时早退 → 补打标，仅供 advisory， + 否则 `mcpp index status` 对自定义索引永远 unknown。 +- **S5** 3×retry 的 2s/4s 退避只在真需要刷新时才可能付费；`offline` 下完全不进 `update_index`。 + +### 4.5 跨平台 + +| 平台 | 影响 | +|---|---| +| **Windows** | 主要收益是**少跑一个多仓库同步**：进程创建 + Defender 扫描让每次 sync 显著更贵。另有残余风险 —— `source = "auto"` 可能回落 git，而 no-git + xvm git shim + `XLINGS_HOME` 重定向曾构成死锁（PR #114）；miss 驱动把这条路径从常态挪到例外。**S3 的锁必须用 `platform/fs.cppm` 的封装**，别写 POSIX-only 的 flock | +| **macOS** | 企业网络受限 + 首启 Gatekeeper 场景同理；大包冷路径不受影响（那是 install 轴） | +| **Termux / musl** | musl-static 的 DNS 短板已知（读不到 `$PREFIX/etc/resolv.conf`）。日常构建完全不碰 DNS 是该平台的可用性底线 | +| **mtime 语义** | `file_time_type` 的 epoch 跨平台不同（#317 踩过）。本设计把 mtime 缩到 debounce（秒级容差）+ advisory，并显式处理负 age（S1）；**身份判定改用 rev 文件，与 mtime 无关** | +| **本地 `path` 索引** | 没有 `.xlings-index-version`，也没有"刷新"概念 → `index_revision` 返回 nullopt，`authoritative_for` 判 local 可证伪但 `apply` 对它是 no-op。e2e 正是靠这条离线化 | +| **CI** | 隔离 `MCPP_HOME` + 工程内 `path` 索引即可离线验证全部分支（PR#307 已建立该模式），不需要真网络 | + +### 4.6 UX 与表达面（刻意收窄） + +**不加 `--refresh-index`**，理由见 §5。表达面只有三个旋钮： + +| 面 | 形态 | 语义 | +|---|---|---| +| flag | `--offline` | **本次调用不发起任何网络**：不刷索引、不下载包、不自动装工具链。与 cargo 的 `--offline` 对齐 | +| env | `MCPP_OFFLINE=1` | 同上，作用于整个会话/CI job | +| config | `[index] auto_refresh = true \| false` | 机器级持久开关；`false` = 索引轴永不自动刷（仍允许包下载）。默认 `true` | + +三条简化决定： + +- **不保留 `always`（旧 TTL 行为）模式**。为模拟一个正被删除的策略而留第二条代码路径，正是 D1 + 这笔债的来源。要「每次都最新」的用户写 shell alias 或 `mcpp index update && mcpp build`。 +- **`MCPP_NO_AUTO_INSTALL` 被 `--offline` 吸收**：它今天只管工具链（`prepare.cppm:1027`）， + 是同一概念的第三个名字。保留为**兼容别名**（等价于 offline 的工具链子集），文档标注 deprecated。 + 一个概念替三个。 +- `cache.search_ttl_seconds` **保留不动**（向后兼容），语义收窄为 `search` 站点的门 + advisory + 阈值，不再是构建路径的 gate。 + +**输出可解释**（D5）： + +``` +# miss 触发（复用 xim 门已有的句式） +Refreshing package index — `mcpplibs:fmt@^1.3` not satisfiable locally (one-time) +Updated mcpplibs 8d67478 → a1b2c3d + +# 稳态：默认静默，-v 可见 +[index] skip refresh: 4/4 deps resolvable locally (mcpplibs 8d67478, 3d old) + +# offline + miss：结构化错误 +error: `mcpplibs:fmt@^1.3` not found in the local package index + index mcpplibs is at 8d67478, last refreshed 12 days ago + run `mcpp index update` (or drop --offline) to fetch it +``` + +**陈旧提示只出现在两处**：解析失败时，以及 `mcpp index status` / `mcpp why deps`（后者加 rev 列）。 +**不在每次构建唠叨** —— 否则只是把网络噪音换成文字噪音。 + +**必须写进 `docs/` 的语义变化**：`^1.2` 将稳定解析到「本地索引已知的最新版」，上游新出的 1.3 需 +`mcpp index update` 或 `mcpp update` 才可见。这是 offline-first 的应有代价，也正是那两个命令存在 +的意义 —— 不写进文档就会变成下一个「为什么拉不到新版本」的 issue。 + +--- + +## 5. 为什么不选其他方案 + +| 方案 | 否决理由 | +|---|---| +| **`build/run/test --refresh-index`**（issue 原文与评论均建议） | ①`mcpp index update && mcpp build` 完全等价，加它等于给 3 个命令各加一份表达面，而它对构建产物**零影响**（不进指纹、不改产物、只有网络副作用）——挂在 `build` 上层次是错的；②cargo 的先例正是一对反例：**有** `build --offline`，**没有** `build --refresh-index`，刷新入口是 `cargo update`；③本设计的目的就是让「何时刷」不需要用户判断，留强制开关等于承认判据不可信。真正的需求（"我知道上游有新版"）语义属于 `mcpp update` → 改为 §4.3 | +| **延长默认 TTL（7 天）** | 只把 D2 的信号错误挪远，没修；且**放大** marker 假新鲜导致拉不到新版本的既有故障模式 | +| **只加 `--no-refresh`** | 默认仍然错，成本转嫁给每个用户每次输入；违反 INV-2 | +| **读 `mcpp.lock` 比对 manifest 后跳过刷新**（评论方案核心） | 前提不成立：`prepare.cppm` 对 lock **只写不读**（3829-3883），构建路径上 lock 不是解析输入。见 §9 | +| **直接把「查不到就刷」接上** | 违反 INV-3：xim 依赖永远匹配不上身份门 → 每次构建都刷，比现状更糟 | +| **把策略放进 `mcpp.toml`** | 违反 INV-4：网络环境属性写进项目清单，项目不可移植 | +| **新增 `mcpp refreshpkg` 子命令**（issue 原文） | `mcpp index update` 已在（`cli.cppm:423`）；新子命令只增认知负担 | + +--- + +## 6. 采纳 / 不采纳 issue 反馈一览 + +| issue / 评论诉求 | 处置 | +|---|---| +| 日常构建不该等网络 | **采纳**，且做成 INV-2 而非加 flag | +| 仅在「首次运行 / 依赖更新」刷新 | **采纳其意图，改写其判据**：不是"首次/变更"，而是"本地解析不出来"（更准，且不需要 lock） | +| `--refresh-pkg` / `mcpp refreshpkg` | **不采纳**（§5），改为修 `mcpp update`（D6） | +| `--refresh-index`（评论方案 2） | **不采纳**（§5） | +| 读 lock 比对 manifest（评论方案 1） | **不采纳**（前提不成立），拆为独立 issue（§9） | +| 不新增 `refreshpkg`、沿用 `index update`/`update`/`add`（评论方案 3） | **采纳** | +| `MCPP_OFFLINE=1`（评论方案 4） | **采纳**，并扩为完整 offline 语义、吸收 `MCPP_NO_AUTO_INSTALL` | +| `[index] auto_refresh`（评论方案 4） | **采纳但收窄为布尔**，不做三值（不保留旧 TTL 模式） | +| 延长 TTL（评论方案 D） | **不采纳**（§5） | + +--- + +## 7. 实施顺序与规模 + +| 阶段 | 内容 | 规模 | +|---|---|---| +| **P0（本 issue）** | §10 探针 → §4.1 策略模块（含 `index_revision`）→ §4.2 五站点收敛 → S1/S2/S3 → §4.3 `mcpp update` → §4.6 `--offline` / `MCPP_OFFLINE` / `[index] auto_refresh` + 可解释输出 → e2e | ~350–450 行，单 PR | +| **P1** | advisory 陈旧提示、`mcpp index status`/`why deps` 增加 rev 列、S4、`docs/` 新增「索引刷新策略」章节 | ~150 行 | +| **P2（各自独立 issue）** | ①`.xlings-index-cache.json` 烙绝对路径（主体在 xlings 侧）+ 把已有检测器接进 production；②`mcpp index update ` 的 per-index 粒度（需 xlings 支持）；③lock 成为构建期解析输入（§9） | — | + +P0 内部顺序：**先落 `decide_for_dependency` + 单测**（判据表是契约），再切站点 4，再切其余四站， +最后加表达面。先测再切，否则 INV-3 的回归无法及早暴露。 + +--- + +## 8. 测试矩阵 + +e2e（全部离线可跑：隔离 `MCPP_HOME` + 工程内 `path` 索引） + +| # | 场景 | 断言 | +|---|---|---| +| 1 | 依赖本地可解析 | **无** `Refreshing`/`Updating` 输出，exit 0 | +| 2 | 描述符本地缺失 | 刷一次，且只刷一次 | +| 3 | 本地只有 1.2，依赖 `^1.3` | 触发 `VersionMiss` | +| 4 | 精确版本 `= "1.2.3"` 而本机版本表无此条 | **不**在 decide 层触发（per-OS 版本表陷阱），由安装层兜 | +| 5 | `--offline` / `MCPP_OFFLINE=1` + miss | 结构化错误含 rev + age + `mcpp index update`，且**不发起网络** | +| 6 | **xim / lazy-git / 第三方 ns 依赖** | **不**触发刷新（INV-3 回归闸，最关键的一条） | +| 7 | 一次构建多个 miss | 只刷一次（进程内 once + debounce） | +| 8 | `mcpp update` | 输出 rev 变化（或 already at）；`--offline` 下明确拒绝 | +| 9 | 两个 mcpp 并发构建同一 MCPP_HOME | 只有一个刷新，另一个跳过而**不报错**（S3） | + +单测：`decide_for_dependency` 判据表逐行；`is_index_dir_fresh` 未来时间戳 → stale； +`index_revision` 在文件缺失/为空/含尾随换行时的行为。 + +**测试写法陷阱（本仓库已踩过）**：反向断言不要写 `! $MCPP build | grep -q X` —— 管道左侧在 +`errexit` 下被豁免，该断言**永不失败**。用 +`out=$($MCPP build 2>&1); echo "$out" | grep -q 'Refreshing' && { echo FAIL; exit 1; }`。 + +--- + +## 9. 明确不做：`mcpp.lock` 作为构建期解析输入 + +现状：`prepare.cppm` 只**写** mcpp.lock（3829-3883）；全仓库读 lock 的只有 +`pm/commands.cppm:394,413`（update/remove）与 `index_management.cppm:209`（index pin）。 +`LockedIndex.rev` 还只是用户 pin 的副本，不是观测值。 + +让 lock 成为解析输入是「零网络 + 可重现」的终局，但它牵出：`resolve_semver` 仍绕过 +`index_route` 的遗留（PR#307 的尾巴）；lock schema 演进与 index rev 的可信度； +「lock 与 manifest 漂移」的检测与自动修复语义（等价于 cargo 的 lock 更新规则）。 + +任一项都比本 issue 大。**本设计与它正交**：P0 之后即便永远不做 lock-honoring resolve，日常构建 +也已经零网络。而 INV-2 采纳的 `.xlings-index-version` 恰好是它未来需要的地基 +（把**观测到的** rev 写进 lock）。硬绑在一起只会让 #315 卡在大重构后面。 + +--- + +## 10. 探针结果（已跑，2026-07-30） + +设计里有两个前提来自磁盘观测而非源码，开工前实测（"写判据要先用探针实测误报"）。 + +### P-1 `.xlings-index-version` 的更新语义 — 通过，但推翻了一个格式假设 + +真实 `mcpp index update` 的输出： + +``` +[xlings] [index] updated from artifact xim-index-ebf4020.tar.gz (ebf4020) +[xlings] [index] updated from artifact mcpp-index-8d67478.tar.gz (8d67478) +[xlings] [index] updated from artifact xim-index-awesome-2026.7.30.1.tar.gz (2026.7.30.1) +``` + +- ✅ artifact 通道确实写 `.xlings-index-version`，值等于 artifact 名里的 rev；刷新后上游没变则值不变 + ⇒ **它是内容身份，不是时间戳**。marker mtime 同时被更新（17:16 → 21:42），两者职责因此可以分开。 +- ⚠️ **值不是稳定的 7 位 sha**：主索引是 `8d67478`，而子索引（awesome / scode / d2x）是**日期版本号** + `2026.7.30.1`。实现必须按**不透明字符串**处理 —— 只比较相等、只打印，绝不解析长度或格式。 + 文件本身**无尾随换行**，但实现仍 trim（Windows 侧写入的 `\r` 会让「同一 rev」每次都判成变化）。 +- ❓ git 通道未直接观测（本机全部走 artifact）。按设计的降级预案处理：缺失 → `index_revision` + 返回 nullopt，advisory 只显示 age，**gate 不受影响**（gate 只依赖描述符可读性）。 + +### P-2 INV-3 误报实测 — 通过 + +含 `mcpplibs.cmdline` + `xim.nasm` + `somevendor.thing` 的真实工程，`-v` 下的判定： + +``` +index: mcpplibs:cmdline@0.0.1: resolvable locally +index: xim.nasm@2.16.03: no index can refute this +index: somevendor.thing@1.0.0: no index can refute this +``` + +xim 与第三方 ns 都落在 `SuppressedInconclusive`，**零刷新**。这正是把 miss 直接当刷新理由会 +翻车的那一类：`xim.nasm` 被解析成候选 `(mcpplibs.xim, nasm)`，永远匹配不上身份门。 + +顺带发现并修正：诊断里回显解析后的候选（`mcpplibs.xim:nasm`）会把读者指向一个他从没写过的 +命名空间 —— 主语改为**用户写的 `[dependencies]` 键**。 + +回归闸：单测 `InconclusiveNamespaceNeverRefreshes` + e2e 173 第 3 步。 + +--- + +## 11. 风险登记 + +| 风险 | 缓解 | +|---|---| +| INV-3 判据接错 → 每次构建都刷（比现状更糟） | 探针 P-2 + e2e #6 双闸；先落单测再切站点 | +| `.xlings-index-version` 语义与假设不符 | 探针 P-1；gate 不依赖 rev，最坏只损失 advisory 精度 | +| 用户感知「mcpp 不再自动更新索引」 | advisory 提示 + docs 明写 + `mcpp update` 变成真入口（D6） | +| `VersionMiss` 与 `resolve_semver` 重复实现约束求解 | 强制调用同一入口，不复制逻辑（否则又一处 D1） | +| per-OS 版本表导致 `VersionMiss` 误触发 | 只对 SemVer 约束判 VersionMiss；精确版本交安装层（e2e #4 锁住） | +| 刷新失败的降级掩盖真实网络故障 | 降级只在「本地有答案」时发生，warning 必须可见 + `-v` 打印底层 rc | +| S3 的锁在 Windows 上行为不一致 | 必须走 `platform/fs.cppm` 封装；拿不到锁=跳过而非阻塞，避免死锁面 | diff --git a/CHANGELOG.md b/CHANGELOG.md index df0beb7f..a5d14615 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,42 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.7.30.3] — 2026-07-30 + +### 变更 + +- **索引刷新从「时间驱动」改为「解析驱动」(#315)。** `mcpp build/run/test` 此前只要刷新 marker 超过 1 小时,就无条件跑一次 `xlings update`(同步**全部**索引仓库,失败还带 3 次 2s/4s 退避重试)——不管有没有东西真的缺失。网络差时这就是「每小时一次、为已经在本地的数据等几分钟」。 + + 这不是「功能缺失」:offline-first 的策略**早就在仓库里**了(`xlings.cppm` 的 xim 安装门,注释里甚至点名了 Termux 上的 build hang),但 `prepare.cppm` 那道 TTL 门先开火,把它变成了不可达代码。同一个决策此前在 **5 处各推导一遍**,其中两处互相矛盾。 + + 现在只有一个真源 `mcpp.pm.index_refresh`:`mcpp build` / `mcpp add` / xim 安装门 / 安装失败重试全部走它。刷新只在三种情况发生 —— 本地没有索引、描述符不在本地、SemVer 约束在本地版本集内无解。**依赖全部能在本地解析的构建零网络请求,无论本地索引多旧。** + + **判据换轴的理由**:「索引够不够新」不可判定,而 mtime 是它的坏代理(CI 缓存恢复、时钟回拨、tar 保留时间戳都会让 marker 两个方向说谎)。可判定的问题是「解析器能不能用磁盘上的东西干活」——而**所有**解析输入都在磁盘上(描述符是文件,`resolve_semver` 解析本地 xpkg.lua)。marker 因此降级为纯去抖计时器。 + + **最危险的一条判据**(`SuppressedInconclusive`):「本地查不到」单独不能推出「需要刷新」。xim 描述符不写 `namespace`,`(xim, x)` 永远匹配不上身份门 —— 把这种 miss 当真,任何带 xim 依赖的工程会**每次构建都刷**,比被删掉的 TTL 更糟。判据复用 `IndexRoute::authoritative_for`(#307),单测 + e2e 双闸锁住。 + + 语义变化:`^1.2` 对**本地索引已知的版本**求解。上游新发的 1.3.0 需要 `mcpp index update` 或 `mcpp update` 才可见 —— 这正是那两个命令存在的意义,已写进 `docs/05-mcpp-toml.md`。 + +- **`mcpp update` 不再是空操作。** 它此前只删 mcpp.lock 条目、然后叫用户去跑 `mcpp build` —— 而构建路径**从不读 mcpp.lock**(`prepare` 只写不读),所以删了等于没删,行为影响为零。它现在先强制刷新索引(显式意图 ⇒ 不看 TTL、不看去抖),并报告索引 rev 的变化;工程里没有任何走共享 registry 的依赖时跳过(刷了也没用)。 + +- **新增 `--offline` / `MCPP_OFFLINE=1`。** 一次调用内完全不碰网络:不刷索引、不下载包、不自动装工具链。已安装的东西照常构建 —— 检查点都放在「真要下载」的那一刻,而不是更早。`MCPP_NO_AUTO_INSTALL` 作为它的旧式窄化拼写保留(只管工具链),文档标注 deprecated:同一个概念此前有三个名字。 + +- **新增 `[index] auto_refresh`(全局 `config.toml`)。** 机器级关闭自动刷新,下载仍可用。刻意做成布尔而**不**保留旧 TTL 模式:为模拟一个正在删除的策略而留第二条代码路径,正是本次要还的那笔债。策略归全局配置而非 `mcpp.toml` —— 它描述机器的网络环境,写进工程清单会让工程在内网/家里/CI 之间不可移植。 + +### 修复 + +- **未来时间戳的索引 marker 被判为「永远新鲜」。** `age < ttl` 对负数恒真,所以时钟回拨、容器时间、`tar` 保留 mtime、CI 缓存恢复产生的未来 marker 会让索引一直「新鲜」到墙钟追上为止。不可用的时间戳现在一律读作 unknown,而 unknown 必须是 stale。 + +- **索引刷新此前没有任何并发保护。** BMI 缓存早就用着 `platform/fs.cppm` 的跨平台 flock/LockFileEx,索引一个锁都没有 —— 并行 CI job 共享 `MCPP_HOME`、多个终端同时构建,就会并发重写同一棵树并并发写 marker。改为非阻塞互斥:**拿不到锁就跳过,不排队、不报错**(持锁者正在做的就是我们想要的事,排队只会把本 issue 要消除的症状原样复制一遍)。 + +- **项目级 env 下刷新永不打标。** `mark_known_indexes_refreshed` 在 `projectDir` 非空时整段早退,于是带自定义 `[indices]` 的工程刷新完全局索引却不留痕,`mcpp index status` 对这些机器永远显示 unknown。 + +### 新增 + +- **`mcpp index status` 增加 revision 列。** 索引自带内容身份 `.xlings-index-version`(artifact 分发把 rev 打进文件名:`mcpp-index-8d67478.tar.gz` → `8d67478`),mcpp 此前**零引用**。它是唯一能回答「两台机器/CI 缓存与本地是不是同一份索引」的信号,age 永远回答不了。按**不透明字符串**处理:子索引的值是日期版本号(`2026.7.30.1`)而不是 sha,探针实测,绝不解析。 + +- 依赖解析失败时附带索引身份与年龄(`index: local index 8d67478 (refreshed 12d ago)`)+ `mcpp index update` 建议 —— 变懒之后,「没这个包」和「你的索引是上个月的」必须能被区分开。 + ## [2026.7.30.2] — 2026-07-30 ### 修复 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 3141a82a..a5ff1870 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -416,6 +416,38 @@ accepted, so already-published descriptors keep working. `mcpp xpkg parse` enforces the rule — run it in your index CI. Requires mcpp >= 0.0.106 and xlings >= 0.4.69; full normative text in `docs/spec/package-identity.md`. +#### When mcpp refreshes the package index + +`mcpp build` / `run` / `test` refresh the package index **only when a dependency +cannot be resolved from the local copy** — never merely because time has passed. +Concretely, a refresh happens when there is no local index at all, when a +dependency's descriptor is missing from it, or when a SemVer constraint matches +none of the versions it knows. A build whose dependencies all resolve locally +makes no network request, however old the local index is. + +The consequence worth knowing: a constraint like `^1.2` resolves against the +**versions your local index knows**. If `1.3.0` was published upstream after +your last refresh, you will not see it until you ask: + +```bash +mcpp index update # sync the index +mcpp update # sync, then re-resolve dependencies +mcpp index status # what you have locally: state, age and revision +``` + +Controls, in order of precedence: + +| Control | Effect | +|---|---| +| `--offline` (any command) | Never touch the network — no index refresh, no downloads, no toolchain auto-install. Anything already installed still builds | +| `MCPP_OFFLINE=1` | Same, for a whole shell session or CI job | +| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh the index automatically; downloads still work | + +`MCPP_NO_AUTO_INSTALL=1` remains accepted as the older, narrower spelling of +`--offline` (it gates only toolchain auto-install). + +Run any command with `-v` to see the decision for each dependency and why. + ### 2.6 `[dev-dependencies]` — Test Dependencies ```toml diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 670703ce..dc622085 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -409,6 +409,35 @@ simd = { sources = ["src/simd/**"], flags = [ feature `flags` 是**私有 per-TU 构建旗标**——永不传播给消费者(与 `[build].flags` 同契约),因此不破坏加性模型:glob 限定作用面、顺序确定、无跨包效应。 +#### mcpp 何时刷新包索引 + +`mcpp build` / `run` / `test` **只在依赖无法用本地索引解析时**刷新包索引,绝不会 +因为"时间到了"就刷。具体地说,只有三种情况会触发:本地根本没有索引、依赖的描述符 +不在其中、或 SemVer 约束在本地已知版本里无解。只要所有依赖都能在本地解析出来, +无论本地索引多旧,构建都不会发起任何网络请求。 + +由此带来的一个需要知道的语义:`^1.2` 这类约束是对**本地索引已知的版本**求解的。 +如果上游在你上次刷新之后发布了 `1.3.0`,你需要主动去取: + +```bash +mcpp index update # 同步索引 +mcpp update # 同步索引,并重新解析依赖 +mcpp index status # 看本地现状:状态、年龄、修订号 +``` + +三个开关,优先级从高到低: + +| 开关 | 作用 | +|---|---| +| `--offline`(任意命令) | 完全不碰网络——不刷索引、不下载、不自动装工具链。已安装的东西照常构建 | +| `MCPP_OFFLINE=1` | 同上,作用于整个 shell 会话或 CI job | +| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不自动刷新索引,但下载仍然可用 | + +`MCPP_NO_AUTO_INSTALL=1` 作为 `--offline` 的旧式窄化拼写仍然有效(它只管工具链的 +自动安装)。 + +任意命令加 `-v` 可以看到每个依赖的判定结果与原因。 + ### 2.8.1 `provides` / `requires` —— 能力(后端选择) **capability(能力)** 是一个共享的抽象名字(如 `blas`)。包可以 *provide*(提供) diff --git a/mcpp.toml b/mcpp.toml index b0a3136c..b44e6c9c 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.7.30.2" +version = "2026.7.30.3" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 617ccc0b..5342eb7b 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -42,6 +42,7 @@ import mcpp.fetcher.progress; import mcpp.pm.resolver; import mcpp.pm.index_spec; import mcpp.pm.index_route; +import mcpp.pm.index_refresh; import mcpp.pm.mangle; import mcpp.pm.compat; import mcpp.pm.dep_spec; @@ -1146,27 +1147,39 @@ prepare_build(bool print_fingerprint, mcpp::fetcher::make_path_ctx(&**get_cfg(), *root)))); } else if (tcSpec.has_value() && *tcSpec == "system") { // Explicit user opt-in to system PATH compiler — kept as escape hatch. - } else if (auto* opt = std::getenv("MCPP_NO_AUTO_INSTALL"); opt && *opt && *opt != '0') { + } else if (mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install()) { // CI / offline / test opt-out: hard-error instead of silently // pulling ~800 MB of toolchain. Preserves the original M5.5 // contract for environments that need it. + // + // `--offline` / MCPP_OFFLINE subsumes MCPP_NO_AUTO_INSTALL: the older + // name only ever covered this one gate, which made "don't use the + // network" three separate concepts with three spellings. The old var is + // kept working (it predates offline mode and CI still exports it). namespace pins = mcpp::toolchain::triple::pins; + // Name the knob that actually fired, not a fixed one: telling a user + // who passed `--offline` to unset MCPP_NO_AUTO_INSTALL sends them + // looking for a variable they never set. + std::string_view release = mcpp::platform::env::offline_mode() + ? "or drop --offline / unset MCPP_OFFLINE to let mcpp auto-install." + : "or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install."; if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" " mcpp toolchain install {}\n" " mcpp toolchain default {}\n" - " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install.", - pins::kSuggestLlvm, pins::kFirstRunMacWin)); + " {}", + pins::kSuggestLlvm, pins::kFirstRunMacWin, release)); } else { return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" " mcpp toolchain install {}\n" " mcpp toolchain default {}\n" - " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install.", - pins::kSuggestGccMusl, pins::kFirstRunLinuxOther)); + " {}", + pins::kSuggestGccMusl, pins::kFirstRunLinuxOther, release)); } } else { // First-run UX: no project-level [toolchain], no global default, @@ -1388,48 +1401,43 @@ prepare_build(bool print_fingerprint, // two different exact versions is an error — mcpp prints both // requesting parents and asks the user to align them. - // Auto-refresh the builtin package index only when a version dependency - // is actually routed there. Local/remote project indices are handled by - // the project-scoped setup below; refreshing the global index for those - // packages is both unnecessary and can make offline/local-index builds - // block on unrelated remote repositories. + // Refresh the builtin package index only when a dependency cannot be + // resolved from the local copy (#315). + // + // This used to fire whenever the refresh marker was older than an hour, + // whether or not anything was actually missing — so every build with a + // registry dependency paid a multi-repo network sync once an hour, which is + // minutes on a slow or blocked network for data it already had. The policy + // now lives in mcpp.pm.index_refresh and is shared with `mcpp add` and the + // xim install gate, which had each derived their own (and disagreed). + // + // Nothing here decides anything itself — in particular the "a miss proves + // nothing for this namespace" rule must not be re-derived; see that module. if (!m->dependencies.empty()) { - auto usesBuiltinIndex = [&](const mcpp::manifest::DependencySpec& spec) { - if (spec.isPath() || spec.isGit()) return false; - - auto ns = spec.namespace_.empty() - ? std::string(mcpp::pm::kDefaultNamespace) - : spec.namespace_; - if (ns == mcpp::pm::kDefaultNamespace) { - // R6: `[indices] default = {...}` (normalized to - // kDefaultNamespace by toml.cppm) redirects the default - // namespace away from the builtin registry — consult it - // before assuming builtin, same as any named namespace. - auto it = m->indices.find(std::string(mcpp::pm::kDefaultNamespace)); - return it == m->indices.end() || it->second.is_builtin(); - } - - auto it = m->indices.find(ns); - if (it == m->indices.end()) return true; - return it->second.is_builtin(); - }; - - bool needsBuiltinIndexRefresh = false; - for (auto& [_, spec] : m->dependencies) { - if (usesBuiltinIndex(spec)) { - needsBuiltinIndexRefresh = true; - break; - } - } - if (needsBuiltinIndexRefresh) { - auto cfg2 = get_cfg(); - if (cfg2) { - auto xlEnv = mcpp::config::make_xlings_env(**cfg2); - if (!mcpp::xlings::is_index_fresh(xlEnv, (*cfg2)->searchTtlSeconds)) { - mcpp::ui::status("Updating", "package index (auto-refresh)"); - mcpp::xlings::ensure_index_fresh( - xlEnv, (*cfg2)->searchTtlSeconds, /*quiet=*/true); + if (auto cfg2 = get_cfg()) { + auto xlEnv = mcpp::config::make_xlings_env(**cfg2); + auto policy = mcpp::pm::policy_for(**cfg2); + // Same routing the dependency walk below uses (the `index_route` + // lambda is declared further down; this is the identical value). + mcpp::pm::IndexRoute route{ &m->indices, *root, *cfg2 }; + for (auto& [depName, spec] : m->dependencies) { + auto decision = mcpp::pm::decide_for_dependency( + route, depName, spec, xlEnv, targetPlatform, policy); + if (!decision.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "{}: {}", decision.subject, + mcpp::pm::reason_text(decision.reason))); + continue; } + // A failed refresh is not a failed build: the dependency walk + // below may still resolve everything from what is on disk, and + // if it cannot, it reports the actual missing package with the + // index's age attached. Failing here instead would turn a + // transient network blip into a hard stop for a build that + // needed no network at all. + if (auto r = mcpp::pm::apply(decision, xlEnv); !r) + mcpp::ui::warning(r.error()); + break; // one sync covers every dependency } } } @@ -1753,6 +1761,17 @@ prepare_build(bool print_fingerprint, } } + // Advisory, never a gate (#315): now that a build only refreshes + // the index on a miss, "not found" and "your copy of the index + // is from last month" are easy to confuse. State which index + // answered and how old it is, so the next step is obvious + // instead of guessed at. + if (auto cfgA = get_cfg()) { + hint += std::format("\n index: {}\n hint: `mcpp index update` " + "if it was published recently", + mcpp::pm::staleness_note( + mcpp::config::make_xlings_env(**cfgA))); + } return std::unexpected(std::format( "dependency '{}': no package found under the namespaces " "mcpp searched\n tried: {}{}", @@ -1971,6 +1990,22 @@ prepare_build(bool print_fingerprint, // progress line and errors should echo back. auto displayName = ns.empty() ? shortName : std::format("{}.{}", ns, shortName); + + // Offline (#315). Checked HERE, at the point of download, and not + // any earlier: everything above this line — reading descriptors, + // resolving versions, reusing an already-installed package — is + // local, and an offline build that has its dependencies must + // succeed. Only the download itself is refused, and it names the + // package rather than surfacing a socket error from three layers + // down. (The toolchain payload path has its own gate; this is the + // dependency path, which does not go through resolve_xpkg_path.) + if (mcpp::platform::env::offline_mode()) { + return std::unexpected(std::format( + "offline mode: dependency '{}' v{} is not installed and " + "cannot be downloaded\n" + " run without --offline (or unset MCPP_OFFLINE) to fetch it", + displayName, version)); + } mcpp::ui::info("Downloading", std::format("{} v{}", displayName, version)); // #238: retain whatever error/warn text the child DID emit so we diff --git a/src/cli.cppm b/src/cli.cppm index 8f75b35e..007a37c3 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -28,6 +28,7 @@ import mcpp.cli.cmd_self; import mcpp.cli.cmd_toolchain; import mcpp.pm.commands; import mcpp.toolchain.fingerprint; // MCPP_VERSION +import mcpp.platform.env; // --offline → MCPP_OFFLINE import mcpp.ui; import mcpp.log; @@ -84,6 +85,7 @@ void print_usage() { std::println(" --cache Dependency cache: global (default) | local | off"); std::println(" --no-cache Deprecated alias for --cache=off (clears the build dir)"); std::println(" --no-color Disable colored output"); + std::println(" --offline Never touch the network (also: MCPP_OFFLINE=1)"); std::println(""); std::println("Docs: https://github.com/mcpp-community/mcpp/tree/main/docs"); } @@ -103,6 +105,12 @@ int run(int argc, char** argv) { if (a == "--quiet" || a == "-q") mcpp::ui::set_quiet(true); else if (a == "--no-color") mcpp::ui::disable_color(); else if (a == "--verbose" || a == "-v") mcpp::log::set_verbose(true); + // --offline is published as the env var rather than plumbed through + // BuildOverrides: its consumers are index refresh, package install and + // toolchain auto-install, which sit in three subsystems and would each + // need a parameter threaded down. Same shape as MCPP_VERBOSE above, and + // it makes `MCPP_OFFLINE=1` and `--offline` literally the same switch. + else if (a == "--offline") mcpp::platform::env::set("MCPP_OFFLINE", "1"); } // Env override (observability, esp. CI): MCPP_VERBOSE= // turns on verbose logging for EVERY mcpp invocation — including the ones @@ -203,6 +211,9 @@ int run(int argc, char** argv) { .help("Show detailed progress on stderr").global()) .option(cl::Option("no-color") .help("Disable colored output").global()) + .option(cl::Option("offline") + .help("Never touch the network (index refresh, downloads, toolchain install)") + .global()) // ─── project commands ────────────────────────────────────────── .subcommand(cl::App("new") diff --git a/src/config.cppm b/src/config.cppm index c352f772..129a7419 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -78,8 +78,20 @@ struct GlobalConfig { std::map indices; // From config.toml [cache] + // NOTE (#315): this is no longer a gate on the build path. It bounds + // `mcpp search` (which exists to ask upstream) and dates the status line. + // Whether a build refreshes is decided by mcpp.pm.index_refresh from local + // resolvability, not from a clock. std::int64_t searchTtlSeconds = 3600; + // From config.toml [index] auto_refresh + // Machine-level opt-out from the automatic on-miss index refresh. Distinct + // from offline mode: this only silences the index axis, package downloads + // still happen. Policy lives in the global config, never in mcpp.toml — it + // describes the machine's network, and a project that carried it would stop + // being portable between a corporate LAN, a laptop and CI. + bool indexAutoRefresh = true; + // From config.toml [build] std::int64_t defaultJobs = 0; std::string defaultBackend = "ninja"; @@ -325,6 +337,10 @@ home = "" [index] default = "mcpplibs" +# auto_refresh: refresh the package index automatically when a dependency +# cannot be resolved from the local copy (never merely because time passed). +# Set false to require an explicit `mcpp index update`. +auto_refresh = true [index.repos."mcpplibs"] url = "https://github.com/mcpplibs/mcpp-index.git" @@ -333,6 +349,8 @@ artifact = "https://github.com/xlings-res/mcpp-index" # xlings auto-adds xim / awesome / scode / d2x as defaults. [cache] +# Bounds `mcpp search` and the age shown by `mcpp index status`. It is NOT +# what decides whether a build refreshes the index — see [index] auto_refresh. search_ttl_seconds = 3600 [build] @@ -494,6 +512,7 @@ std::expected load_or_init( if (auto h = doc->get_string("xlings.home"); h && !h->empty()) cfg.xlingsHomeOverride = *h; cfg.defaultIndex = doc->get_string("index.default").value_or("mcpplibs"); + cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); cfg.defaultBackend = doc->get_string("build.default_backend").value_or("ninja"); diff --git a/src/doctor.cppm b/src/doctor.cppm index 0105a062..6488422c 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -19,6 +19,7 @@ import mcpp.fetcher.progress; import mcpp.home; import mcpp.platform; import mcpp.platform.process; +import mcpp.pm.index_refresh; // staleness_note for `mcpp why deps` import mcpp.toolchain.detect; import mcpp.toolchain.msvc; import mcpp.toolchain.registry; @@ -417,6 +418,14 @@ export int why_report(const std::string& topic) { } } if (all || topic == "deps") { + // Which index answered, and how stale it is. Since #315 a build only + // refreshes on a resolution miss, so "why did I get this version" + // frequently has "because that is the newest one your local index + // knows" as its answer — which is unguessable without this line. + if (auto cfgW = mcpp::config::load_or_init(/*quiet=*/true)) { + std::println("package index: {}", + mcpp::pm::staleness_note(mcpp::config::make_xlings_env(*cfgW))); + } std::println("dependencies (mcpp.lock):"); std::ifstream in(ctx->projectRoot / "mcpp.lock"); if (!in) { diff --git a/src/platform/env.cppm b/src/platform/env.cppm index a64210b5..7ac88fb1 100644 --- a/src/platform/env.cppm +++ b/src/platform/env.cppm @@ -23,6 +23,27 @@ std::optional get(std::string_view key); // Set an environment variable in the current process. void set(const std::string& key, const std::string& value); +// ── Network policy: offline mode ────────────────────────────────────────── +// +// One process-wide knob answering "may mcpp reach the network at all?", read +// at each point of use rather than threaded through a dozen signatures — the +// same idiom the older MCPP_NO_AUTO_INSTALL gate already uses. +// +// It lives HERE, in a leaf module, on purpose: the consumers span pm (index +// refresh, package install) and build (toolchain auto-install), and any home +// inside one of those subsystems would make the other import it across a layer +// boundary — mcpp.pm.index_route already imports mcpp.fetcher, so a pm-owned +// helper could not be read from the fetcher without a module cycle. +// +// Set by `--offline` (cli pre-scan writes the env var) or by the user exporting +// MCPP_OFFLINE. "0" and the empty string mean off, matching MCPP_VERBOSE. +bool offline_mode(); + +// The older, narrower spelling: MCPP_NO_AUTO_INSTALL gates ONLY the toolchain +// auto-install, nothing else. Kept working (CI and existing setups export it) +// and kept next to offline_mode() so the two network knobs cannot drift apart. +bool no_auto_install(); + // Temporarily set or unset an env var, restoring the prior value on scope exit. class ScopedEnv { public: @@ -74,6 +95,18 @@ std::string build_env_prefix( namespace mcpp::platform::env { +bool offline_mode() { + // Not cached: `mcpp test` runs nested mcpp invocations in-process in some + // paths, and a test that flips the var must see the change. + auto* v = std::getenv("MCPP_OFFLINE"); + return v && *v && std::string_view(v) != "0"; +} + +bool no_auto_install() { + auto* v = std::getenv("MCPP_NO_AUTO_INSTALL"); + return v && *v && std::string_view(v) != "0"; +} + std::optional get(std::string_view key) { std::string k(key); auto* v = std::getenv(k.c_str()); diff --git a/src/pm/commands.cppm b/src/pm/commands.cppm index 1f709daa..36d74caa 100644 --- a/src/pm/commands.cppm +++ b/src/pm/commands.cppm @@ -21,6 +21,7 @@ import mcpp.platform.axis; // HostPlatform for the published-version chec import mcpp.pm.dep_spec; // DependencyCoordinate import mcpp.pm.dependency_selector; // same candidates the manifest parser derives import mcpp.pm.index_route; // shared index routing (with mcpp.build.prepare) +import mcpp.pm.index_refresh; // shared refresh policy (with mcpp.build.prepare) import mcpp.pm.resolver; // is_version_constraint import mcpp.project; // shared find_manifest_root import mcpp.ui; @@ -161,17 +162,19 @@ inline int cmd_add(const mcpplibs::cmdline::ParsedArgs& parsed) { return !idx || idx->is_builtin(); }); - // Only pay for a refresh when the answer was "no" and the registry - // that would have answered is stale — a package that is already on - // disk costs zero network round-trips. + // Only pay for a refresh when the answer was "no" and the registry is + // the thing that would have answered — a package already on disk costs + // zero network round-trips. This site had the right shape before #315; + // it now shares the decision with `mcpp build` instead of re-deriving + // it, so the two cannot drift on debounce, offline or opt-out. if (!found.hit && found.conclusive && registryInvolved) { auto xlEnv = mcpp::config::make_xlings_env(*cfg); - if (!mcpp::xlings::is_index_fresh(xlEnv, cfg->searchTtlSeconds)) { - mcpp::ui::status("Updating", "package index (auto-refresh)"); - mcpp::xlings::ensure_index_fresh( - xlEnv, cfg->searchTtlSeconds, /*quiet=*/true); + auto d = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(*cfg), xlEnv, nameSpec); + if (auto r = mcpp::pm::apply(d, xlEnv); !r) + mcpp::ui::warning(r.error()); + if (d.shouldRefresh) found = mcpp::pm::lookup_descriptor(route, selector.candidates); - } } if (!found.hit && found.conclusive) { @@ -406,6 +409,42 @@ inline int cmd_update(const mcpplibs::cmdline::ParsedArgs& parsed) { auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) { mcpp::ui::error("no mcpp.toml in current dir or parents"); return 2; } + + // Refresh the index FIRST (#315/D6). + // + // This command used to only drop lock entries and tell the user to run + // `mcpp build` — but the build path never reads mcpp.lock (prepare writes + // it and nothing on that path loads it), so the whole command was a no-op: + // it changed no behaviour whatsoever. It is also the only command whose + // stated purpose is "get me newer dependencies", which since #315 is + // exactly what an index refresh is for. Explicit intent, so no debounce and + // no TTL — but still refused when offline, loudly, rather than silently + // doing nothing again. + // Skipped when nothing in the project is served by the shared registry: + // syncing it does nothing for path deps, git deps or a project `[indices]` + // entry, and paying a multi-repo network round-trip to achieve nothing is + // the exact behaviour #315 is about. + if (auto cfg = mcpp::config::load_or_init( + /*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback())) { + auto m = mcpp::manifest::load(*root / "mcpp.toml"); + bool registryInvolved = false; + if (m) { + auto indices = mcpp::pm::effective_indices(*root); + mcpp::pm::IndexRoute route{ &indices, *root, &*cfg }; + for (auto& [_, spec] : m->dependencies) + if (mcpp::pm::routes_to_builtin(route, spec)) { registryInvolved = true; break; } + } + if (registryInvolved) { + auto xlEnv = mcpp::config::make_xlings_env(*cfg); + if (auto r = mcpp::pm::force_refresh(xlEnv); !r) { + mcpp::ui::error(r.error()); + return 1; + } + } + } else { + mcpp::ui::warning(cfg.error().message); + } + auto lockPath = *root / "mcpp.lock"; if (only) { // Targeted update — drop just that lock entry; next build will refetch. diff --git a/src/pm/index_management.cppm b/src/pm/index_management.cppm index 715cf738..60895fde 100644 --- a/src/pm/index_management.cppm +++ b/src/pm/index_management.cppm @@ -14,6 +14,7 @@ import mcpp.fetcher; import mcpp.fetcher.progress; import mcpp.lockfile; import mcpp.manifest; +import mcpp.platform; import mcpp.project; import mcpp.ui; import mcpp.xlings; @@ -25,6 +26,16 @@ export int search_packages(const std::string& keyword) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } + // THE ONLY TIME-DRIVEN REFRESH LEFT IN MCPP, and deliberately so. + // + // Everywhere else the question is "can the resolver work with what is on + // disk", which is answerable offline and therefore what #315 made the gate. + // Search has no such local answer to check: the user is asking what EXISTS + // upstream, so a stale index gives a confidently wrong answer ("no matches" + // for a package published last week) rather than a slow one. A TTL is the + // right instrument here — do not "unify" this with mcpp.pm.index_refresh. + // Offline is still honoured: update_index() returns without touching the + // network, and search reports whatever the local copy holds. auto xlEnv = mcpp::config::make_xlings_env(*cfg); if (!mcpp::xlings::is_index_fresh(xlEnv, cfg->searchTtlSeconds)) { mcpp::ui::status("Updating", "package index (auto-refresh)"); @@ -113,10 +124,24 @@ export int index_remove(const std::string& name) { } // `mcpp index update [name]` — empty filterName updates everything. +// +// NOTE on `name`: it only filters the PROJECT-level custom indices below. The +// global repos are always synced wholesale because `xlings update` has no +// per-index mode to call. Tracked as a follow-up; the help text says "if given, +// update only this index", which is not what happens for the global set. export int index_update(const std::string& filterName) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } + // Explicit command: offline must refuse audibly rather than no-op. Left to + // this layer because update_index() itself is a silent skip by design (it + // is called from paths where proceeding locally is the right answer). + if (mcpp::platform::env::offline_mode()) { + mcpp::ui::error("offline mode is on — cannot update the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + return 1; + } + // Update global index repos. mcpp::ui::status("Updating", "all index repos"); auto xlEnv = mcpp::config::make_xlings_env(*cfg); @@ -173,15 +198,20 @@ export int index_status() { std::string state = !st.present ? "missing" : st.fresh ? "fresh" : "stale"; - std::println(" {:<10} {:<8} {:<12} {}", - label, state, fmt_age(st.ageSeconds), st.dir.string()); + // The revision is the index's CONTENT identity (see index_revision): + // it is what actually tells you whether two machines — or a CI cache + // and a laptop — hold the same index, which the age never could. + std::println(" {:<10} {:<8} {:<12} {:<12} {}", + label, state, fmt_age(st.ageSeconds), + st.rev.value_or("-"), st.dir.string()); }; auto official = mcpp::xlings::official_index_status(xlEnv, cfg->searchTtlSeconds); auto deflt = mcpp::xlings::default_index_status(xlEnv, cfg->searchTtlSeconds); std::println(""); - std::println(" {:<10} {:<8} {:<12} {}", "index", "state", "refreshed", "path"); + std::println(" {:<10} {:<8} {:<12} {:<12} {}", + "index", "state", "refreshed", "revision", "path"); show("xim", official); show("mcpplibs", deflt); std::println(""); diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm new file mode 100644 index 00000000..eddd14b0 --- /dev/null +++ b/src/pm/index_refresh.cppm @@ -0,0 +1,354 @@ +// mcpp.pm.index_refresh — the single source of truth for "should mcpp touch +// the network to refresh a package index right now?" +// +// WHY THIS MODULE EXISTS +// +// The answer used to be derived independently in five places, and two of them +// disagreed. `mcpp.xlings`' xim gate (ensure_official_package_index_fresh) had +// been offline-first for a while — it refreshes only when a requested package +// is genuinely absent locally, and its comment says in so many words that a +// TTL must NOT trigger a network sync, because that is what hangs a build on a +// slow or blocked network. `mcpp.build.prepare` meanwhile refreshed on exactly +// that: a marker older than an hour, whether or not anything was missing. So +// the offline-first policy was real but unreachable — the TTL fired first, +// every hour, on every build with a registry dependency (#315). +// +// THE AXIS WAS WRONG, NOT JUST THE THRESHOLD +// +// "Is this index fresh enough?" is unanswerable, and mtime is a bad proxy for +// it: a restored CI cache, a clock skew, or a tar that preserved timestamps all +// produce a marker that lies in either direction. The answerable question is +// "can the resolver do its job with what is on disk?" — and every resolution +// input IS on disk (descriptors are files; `resolve_semver` parses a local +// xpkg.lua). So the decision below is a pure, offline, deterministic function +// of local state, and the marker is demoted to a debounce timer. +// +// THE TRAP THIS MODULE IS MOSTLY WRITTEN AROUND +// +// "Not found locally" must NEVER be read as "needs a refresh" on its own. A +// miss only means something when the index that would have answered is both +// readable and authoritative — `IndexRoute::authoritative_for` (#307) is what +// decides that. Notably, xim descriptors declare no `namespace`, so `(xim, x)` +// can never match the identity gate: treat that miss as real and every build +// with a toolchain-ish dependency refreshes EVERY TIME, which is strictly worse +// than the TTL this change removes. See `SuppressedInconclusive`. + +export module mcpp.pm.index_refresh; + +import std; +import mcpp.config; +import mcpp.log; +import mcpp.platform; +import mcpp.platform.axis; +import mcpp.pm.dep_spec; +import mcpp.pm.index_route; +import mcpp.pm.resolver; +import mcpp.ui; +import mcpp.xlings; + +export namespace mcpp::pm { + +// Why a refresh was, or was not, triggered. Every value is user-visible under +// `-v`, and the Suppressed* ones are the interesting half when diagnosing +// "why did/didn't mcpp go to the network". +enum class RefreshReason { + None, // resolvable locally — the steady state, zero network + IndexAbsent, // no local index at all (cold start) + DescriptorMiss, // package unknown locally, and the index can say so + VersionMiss, // package known, constraint unsatisfiable locally + SuppressedOffline, // --offline / MCPP_OFFLINE + SuppressedDisabled, // [index] auto_refresh = false + SuppressedDebounce, // refreshed moments ago; upstream simply lacks it + SuppressedInconclusive, // a miss here proves nothing (see header) +}; + +struct RefreshPolicy { + bool offline = false; + bool autoRefresh = true; + // Shared with the xim install gate — one constant, one rationale, in the + // leaf module both layers can see (mcpp::xlings). + std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors +}; + +// Human tail for the status line: "package index — ". +std::string_view reason_text(RefreshReason r); + +// flag > env > config. `offlineFlag` is the parsed `--offline`. +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); + +// Pure: no network, no filesystem writes, no side effects. The judgement table +// is the contract — `tests/unit/test_pm_index_refresh.cpp` locks it row by row. +RefreshDecision decide_for_dependency(const IndexRoute& route, + std::string_view depKey, + const DependencySpec& spec, + const mcpp::xlings::Env& env, + const mcpp::platform::PlatformKey& platform, + const RefreshPolicy& policy); + +// Is this dependency served by the shared registry (as opposed to a project +// `[indices]` entry, a path, or a git URL)? Refreshing the global index does +// nothing for the others, so callers use this to skip the sync entirely. +bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec); + +// For a caller that has ALREADY established a conclusive miss through +// `lookup_descriptor` (that is `mcpp add`): only the policy half of the +// judgement is left to make. Keeps the opt-outs in one place rather than +// re-tested at each call site. +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject); + +// Run the sync a decision asked for. At most ONE per process: a build whose +// deps miss for the same reason should pay one sync, not one per dep. +// Returns an error only when the sync itself failed; callers decide whether +// that is fatal (it is not, if the build can still resolve locally). +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env); + +// Explicit user intent (`mcpp update`): ignores debounce and the once-per- +// process guard. Still refuses when offline, and says so. +std::expected force_refresh(const mcpp::xlings::Env& env); + +// One line of advisory context for a resolution failure: what the index is and +// how old it is. Never a gate — only ever appended to an error the user is +// already seeing. +std::string staleness_note(const mcpp::xlings::Env& env); + +} // namespace mcpp::pm + +namespace mcpp::pm { + +namespace { + +// One sync per process, however many dependencies ask for one. +bool g_refreshed_this_process = false; + +// Echo back what the USER wrote (the `[dependencies]` key), not the resolved +// coordinate. `xim.nasm` is parsed into the candidate `(mcpplibs.xim, nasm)`, +// and reporting that spelling in a diagnostic sends the reader looking for a +// namespace they never typed. +std::string subject_of(std::string_view depKey, const DependencySpec& spec) { + std::string name{depKey}; + if (name.empty()) { + name = spec.shortName; + if (!spec.namespace_.empty()) + name = std::format("{}:{}", spec.namespace_, name); + } + return spec.version.empty() ? name : std::format("{}@{}", name, spec.version); +} + +// Does ANY candidate for this dependency resolve through the shared registry? +// A project `[indices] path = …` is whatever the user has on disk and a custom +// git index is synced by its own project-scoped path — refreshing the global +// index would do nothing for either, so those keep today's behaviour exactly. +bool coords_route_to_builtin(const IndexRoute& route, + const std::vector& coords) { + for (auto& c : coords) { + auto* idx = route.find_for_ns(c.namespace_); + if (!idx || idx->is_builtin()) return true; + } + return false; +} + +std::vector coords_of(std::string_view depKey, + const DependencySpec& spec) { + if (!spec.candidates.empty()) return spec.candidates; + // Specs built by hand (tests, older call sites) carry no candidate list. + return { DependencyCoordinate{ + .namespace_ = spec.namespace_, + .shortName = spec.shortName.empty() ? std::string(depKey) : spec.shortName, + } }; +} + +std::string age_phrase(std::int64_t s) { + if (s < 0) return "never refreshed"; + if (s < 90) return std::format("refreshed {}s ago", s); + if (s < 5400) return std::format("refreshed {}m ago", s / 60); + if (s < 172800) return std::format("refreshed {}h ago", s / 3600); + return std::format("refreshed {}d ago", s / 86400); +} + +} // namespace + +std::string_view reason_text(RefreshReason r) { + switch (r) { + case RefreshReason::None: return "resolvable locally"; + case RefreshReason::IndexAbsent: return "no local package index"; + case RefreshReason::DescriptorMiss: return "not found locally"; + case RefreshReason::VersionMiss: return "not satisfiable locally"; + case RefreshReason::SuppressedOffline: return "offline mode"; + case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; + case RefreshReason::SuppressedDebounce: return "index was just refreshed"; + case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + } + return ""; +} + +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { + RefreshPolicy p; + p.offline = mcpp::platform::env::offline_mode(); + p.autoRefresh = cfg.indexAutoRefresh; + return p; +} + +RefreshDecision decide_for_dependency(const IndexRoute& route, + std::string_view depKey, + const DependencySpec& spec, + const mcpp::xlings::Env& env, + const mcpp::platform::PlatformKey& platform, + const RefreshPolicy& policy) +{ + RefreshDecision d; + d.subject = subject_of(depKey, spec); + + // 1. Sources that never consult an index. + if (spec.isPath() || spec.isGit()) return d; + + // 2. Only the shared registry is refreshable from here. + auto coords = coords_of(depKey, spec); + if (!coords_route_to_builtin(route, coords)) return d; + + // 3. INV-3: a miss only counts when the index could have refuted it. + auto found = lookup_descriptor(route, coords); + if (!found.hit && !found.conclusive) { + d.reason = RefreshReason::SuppressedInconclusive; + return d; + } + + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + + // 4-6. The three ways local state fails to answer. + // + // Deliberately evaluated BEFORE the opt-outs below: this is all local file + // I/O, and knowing whether the answer was actually available is what makes + // the diagnostic worth reading. Short-circuiting on `--offline` first would + // report "offline mode" for a dependency that resolved perfectly well — + // which is precisely the case the user is trying to confirm. + if (!status.present) { + d.reason = RefreshReason::IndexAbsent; + d.shouldRefresh = true; + } else if (!found.hit) { + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + } else if (is_version_constraint(spec.version)) { + // Only CONSTRAINTS are checked here. An exact pin is deliberately not: + // version tables are per-OS, so "this host publishes no 1.2.3" is not + // "1.2.3 does not exist" (the same judgement `mcpp add` reaches — it + // flags an unpublished exact version rather than refusing it). Treating + // it as a miss would refresh on every build for a dependency that is + // simply not built for this platform. The install layer already covers + // the real version-miss case: it refreshes once when an install fails. + auto resolved = resolve_semver(found.hit->coord.namespace_, + found.hit->coord.shortName, + spec.version, route, platform); + if (!resolved) { + d.reason = RefreshReason::VersionMiss; + d.shouldRefresh = true; + } + } + + if (!d.shouldRefresh) return d; // nothing to suppress + + // 7. Opt-outs, in order of authority: the user's flag, the machine's + // config, then the "we just did this" guard. + if (policy.offline) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedDisabled; return d; } + + // 8. Debounce — but never for a cold start, where there is nothing to + // debounce against and the build cannot proceed without the data. + if (d.reason != RefreshReason::IndexAbsent + && status.ageSeconds >= 0 && status.ageSeconds < policy.debounceSeconds) { + d.shouldRefresh = false; + d.reason = RefreshReason::SuppressedDebounce; + } + + return d; +} + +bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec) { + if (spec.isPath() || spec.isGit()) return false; + return coords_route_to_builtin(route, coords_of(spec.shortName, spec)); +} + +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject) { + RefreshDecision d; + d.subject = std::string(subject); + if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } + // Same debounce as the build path: two `mcpp add` typos in a row should not + // buy two multi-repo syncs, for the same reason a build with two missing + // packages does not. + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + if (status.present && status.ageSeconds >= 0 + && status.ageSeconds < policy.debounceSeconds) { + d.reason = RefreshReason::SuppressedDebounce; + return d; + } + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + return d; +} + +namespace { + +std::expected run_sync(const mcpp::xlings::Env& env, + std::string_view banner) { + auto before = mcpp::xlings::default_index_status(env, 0).rev; + mcpp::ui::status("Refreshing", banner); + int rc = mcpp::xlings::update_index(env, /*quiet=*/true); + if (rc != 0) + return std::unexpected(std::format("package index refresh failed (rc {})", rc)); + auto after = mcpp::xlings::default_index_status(env, 0).rev; + if (before && after && *before != *after) + mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); + else if (after) + mcpp::log::verbose("index", std::format("package index still at {}", *after)); + return {}; +} + +} // namespace + +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env) +{ + if (!d.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: {}", d.subject, reason_text(d.reason))); + return {}; + } + if (g_refreshed_this_process) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: already refreshed in this run", d.subject)); + return {}; + } + g_refreshed_this_process = true; + return run_sync(env, std::format("package index — `{}` {} (one-time)", + d.subject, reason_text(d.reason))); +} + +std::expected force_refresh(const mcpp::xlings::Env& env) { + if (mcpp::platform::env::offline_mode()) + return std::unexpected( + "offline mode is on — cannot refresh the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + g_refreshed_this_process = true; + return run_sync(env, "package index (requested)"); +} + +std::string staleness_note(const mcpp::xlings::Env& env) { + auto st = mcpp::xlings::default_index_status(env, 0); + if (!st.present) return "no local package index"; + return st.rev + ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) + : std::format("local index {}", age_phrase(st.ageSeconds)); +} + +} // namespace mcpp::pm diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index b6e26403..8d5aecf5 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -15,6 +15,7 @@ export module mcpp.pm.package_fetcher; import std; import mcpp.config; import mcpp.log; +import mcpp.platform; // env::offline_mode import mcpp.manifest; // xpkg_lua_identity_matches — descriptor identity gate import mcpp.pm.compat; import mcpp.pm.dep_spec; @@ -1004,6 +1005,17 @@ Fetcher::resolve_xpkg_path(std::string_view target, // 3. Install via xlings (primary path). if (autoInstall) { + // Offline is checked HERE, after the "already installed" fast paths + // above: a build whose packages are all present must stay fully + // functional offline — that is the whole point. Only an actual + // download attempt is refused, and it names the package so the user + // knows what to fetch rather than seeing a generic network error. + if (mcpp::platform::env::offline_mode()) { + return std::unexpected(CallError{ std::format( + "offline mode: `{}@{}` is not installed and cannot be downloaded\n" + " run without --offline (or unset MCPP_OFFLINE) to fetch it", + parsed.packageName, parsed.version) }); + } if (parsed.indexName == "xim") { mcpp::xlings::Env xlEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; // quiet=false: this only ever prints when a dependency is missing diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index d21a4140..df831d60 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "2026.7.30.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.7.30.3"; struct FingerprintInputs { Toolchain toolchain; diff --git a/src/xlings.cppm b/src/xlings.cppm index c94a5384..d5d50c66 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -307,6 +307,15 @@ std::optional find_sandbox_nasm(const Env& env); // ─── Index freshness ──────────────────────────────────────────────── +// How long a just-completed index sync suppresses the next one. +// +// Re-running a multi-repo sync because a SECOND package is also missing cannot +// help: the first sync already fetched everything upstream had, so a package +// still absent afterwards is absent upstream. Lives here, in the leaf module, +// because both users need it and the policy layer (mcpp.pm.index_refresh) +// imports this module — the reverse would be a cycle. +inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; + // Check whether the default mcpplibs index data exists and is fresh // (within ttlSeconds). // Returns true if index is present and fresh, false otherwise. @@ -346,14 +355,26 @@ void ensure_official_package_index_fresh(const Env& env, // Snapshot of a local index directory — computed without touching the // network, for `mcpp index status`. struct IndexStatus { - std::filesystem::path dir; // on-disk index directory - bool present; // pkgs/ tree exists locally - bool fresh; // refreshed within ttlSeconds - std::int64_t ageSeconds; // since last refresh marker, -1 if unknown + std::filesystem::path dir; // on-disk index directory + bool present; // pkgs/ tree exists locally + bool fresh; // refreshed within ttlSeconds + std::int64_t ageSeconds; // since last refresh marker, -1 if unknown + std::optional rev; // index CONTENT identity, nullopt if absent }; IndexStatus default_index_status(const Env& env, std::int64_t ttlSeconds); IndexStatus official_index_status(const Env& env, std::int64_t ttlSeconds); +// The index's own content identity, as written by xlings into +// `/.xlings-index-version` when it materializes the tree. +// +// OPAQUE BY CONTRACT. Observed values are a 7-char short sha for the artifact- +// distributed indexes (`mcpp-index-8d67478.tar.gz` → `8d67478`) but a DATE +// VERSION for the sub-indexes (`xim-index-awesome-2026.7.30.1.tar.gz` → +// `2026.7.30.1`). Never parse it, never assume a length — only compare it for +// equality and print it. Returns nullopt when the file is missing (a local +// `path` index has none) or blank; callers must degrade, never hard-fail. +std::optional index_revision(const std::filesystem::path& indexDir); + // ─── run_capture utility ──────────────────────────────────────────── std::expected run_capture(const std::string& cmd); @@ -394,6 +415,10 @@ std::filesystem::path index_refresh_marker(const std::filesystem::path& indexDir return indexDir / ".mcpp-index-updated"; } +std::filesystem::path index_version_file(const std::filesystem::path& indexDir) { + return indexDir / ".xlings-index-version"; +} + std::filesystem::path official_package_file(const Env& env, std::string_view packageName) { if (packageName.empty()) return {}; std::string name(packageName); @@ -442,7 +467,12 @@ void mark_index_refreshed(const std::filesystem::path& indexDir) { } void mark_known_indexes_refreshed(const Env& env) { - if (!env.projectDir.empty()) return; + // Project-scoped envs used to return early here, which left the global + // indexes unmarked whenever a project with custom `[indices]` triggered the + // sync — so `mcpp index status` reported "unknown" forever on exactly the + // machines that refresh most. The marker is advisory (it debounces and it + // dates the status line; it is NOT what decides whether a refresh is + // needed), so marking it from either env is both safe and more accurate. mark_index_refreshed(default_index_dir(env)); mark_index_refreshed(official_index_dir(env)); } @@ -459,6 +489,12 @@ bool is_index_dir_fresh(const std::filesystem::path& indexDir, std::int64_t ttlS auto now = std::filesystem::file_time_type::clock::now(); auto age = std::chrono::duration_cast(now - newest); + // A marker stamped in the FUTURE yields a negative age, which the plain + // `age < ttl` test read as "fresh" — and stayed fresh until the wall clock + // caught up, potentially for years. Future timestamps are routine: clock + // skew in containers/VMs, a tar that preserved mtimes, a restored CI cache. + // An unusable timestamp means "unknown", and unknown must mean stale. + if (age.count() < 0) return false; return age.count() < ttlSeconds; } @@ -470,7 +506,27 @@ std::int64_t index_age_seconds(const std::filesystem::path& indexDir) { auto newest = std::filesystem::last_write_time(marker, ec); if (ec) return -1; auto now = std::filesystem::file_time_type::clock::now(); - return std::chrono::duration_cast(now - newest).count(); + auto age = std::chrono::duration_cast(now - newest).count(); + return age < 0 ? -1 : age; // future stamp → "unknown", same rule as above +} + +// Defined inside the anonymous namespace so `index_status_for` (also here) can +// use it; the exported `index_revision` below forwards to it. +std::optional read_index_revision(const std::filesystem::path& indexDir) { + std::ifstream is(index_version_file(indexDir), std::ios::binary); + if (!is) return std::nullopt; + std::string body((std::istreambuf_iterator(is)), {}); + // The observed files carry no trailing newline, but trim anyway: this value + // is compared for equality and printed, and a stray \r from a Windows-side + // writer would otherwise turn "same rev" into "changed rev" every run. + auto isSpace = [](unsigned char c) { return std::isspace(c) != 0; }; + while (!body.empty() && isSpace(static_cast(body.back()))) + body.pop_back(); + std::size_t b = 0; + while (b < body.size() && isSpace(static_cast(body[b]))) ++b; + body.erase(0, b); + if (body.empty()) return std::nullopt; + return body; } IndexStatus index_status_for(const std::filesystem::path& indexDir, @@ -482,6 +538,7 @@ IndexStatus index_status_for(const std::filesystem::path& indexDir, .present = present, .fresh = is_index_dir_fresh(indexDir, ttlSeconds), .ageSeconds = index_age_seconds(indexDir), + .rev = read_index_revision(indexDir), }; } @@ -558,6 +615,12 @@ struct LineScan { } // anonymous namespace +// ─── Index identity ───────────────────────────────────────────────── + +std::optional index_revision(const std::filesystem::path& indexDir) { + return read_index_revision(indexDir); +} + // ─── run_capture ──────────────────────────────────────────────────── std::expected run_capture(const std::string& cmd) { @@ -1356,6 +1419,33 @@ bool is_official_package_index_fresh(const Env& env, } int update_index(const Env& env, bool quiet) { + // Offline is absolute: no caller gets to reach the network by going around + // the decision layer. Reported as success so a build that can still resolve + // everything locally proceeds — the caller that genuinely needed the data + // fails on its own missing answer, with a message that says so. + if (mcpp::platform::env::offline_mode()) { + mcpp::log::verbose("index", "offline mode: skipping index update"); + return 0; + } + + // Cross-process mutex. The sync rewrites a tree several concurrent mcpp + // processes read (parallel CI jobs on one MCPP_HOME, two terminals, a + // workspace fan-out), and it also writes the refresh markers. The BMI cache + // has had this guard for a while; the index never did. + // + // NON-BLOCKING BY DESIGN: whoever holds the lock is already doing the exact + // work we wanted, so waiting buys nothing and a queue of stalled builds is + // precisely the symptom this whole change exists to remove. Skipping is + // also why this must not be reported as failure. + std::error_code lockEc; + std::filesystem::create_directories(paths::index_data(env), lockEc); + auto lock = mcpp::platform::fs::FileLock::try_acquire(paths::index_data(env)); + if (!lock) { + mcpp::log::verbose("index", + "another process is refreshing the index — skipping this one"); + return 0; + } + std::string cmd = build_command_prefix(env) + " update 2>&1"; // The index sync is a network git operation; a single transient blip (DNS, // TLS reset, a mirror hiccup) otherwise fails a cold `mcpp self env` / @@ -1420,8 +1510,7 @@ void ensure_official_package_index_fresh(const Env& env, // `xlings update` per package: if the index was refreshed moments ago and // the package is STILL missing, upstream simply lacks it; re-pulling won't // help. (A package added upstream before this run lands in that one pull.) - constexpr std::int64_t kJustRefreshedSeconds = 120; - if (is_official_index_fresh(env, kJustRefreshedSeconds)) return; + if (is_official_index_fresh(env, kIndexRefreshDebounceSeconds)) return; if (!quiet) print_status("Refreshing", diff --git a/tests/e2e/173_index_refresh_policy.sh b/tests/e2e/173_index_refresh_policy.sh new file mode 100755 index 00000000..344a5071 --- /dev/null +++ b/tests/e2e/173_index_refresh_policy.sh @@ -0,0 +1,162 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# #315: `mcpp build` must not refresh the package index just because time +# passed. It refreshes when — and only when — the local index cannot answer. +# +# Every case here is HERMETIC: the builtin registry is fabricated on disk and +# every path that could reach the network is either offline-gated or debounced, +# so this test never downloads anything and never depends on upstream state. +# +# The load-bearing case is INV-3 (step 3 below): a namespace the registry cannot +# refute — `xim`, whose descriptors declare no namespace, or any third-party ns +# — must NOT count as a miss. If it did, every build with such a dependency +# would sync EVERY TIME, which is strictly worse than the hourly TTL this +# change removes. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +INDEX="$MCPP_HOME/registry/data/mcpplibs" +mkdir -p "$INDEX/pkgs/w" + +cat > "$INDEX/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", + namespace = "mcpplibs", + name = "widget", + description = "Present in the local index", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { ["1.2.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + macosx = { ["1.2.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + windows = { ["1.2.0"] = { url = "https://example.invalid/w.zip", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + }, +} +EOF +echo "abc1234" > "$INDEX/.xlings-index-version" + +# Back-date the refresh marker well past any debounce window: under the old +# TTL policy this alone was enough to trigger a network sync on every build. +: > "$INDEX/.mcpp-index-updated" +touch -d '30 days ago' "$INDEX/.mcpp-index-updated" 2>/dev/null \ + || touch -A -300000 "$INDEX/.mcpp-index-updated" 2>/dev/null || true + +mkdir -p "$TMP/proj/src" +cd "$TMP/proj" +echo 'int main() { return 0; }' > src/main.cpp + +manifest() { # manifest + cat > mcpp.toml < + if grep -qE "$2" <<< "$1"; then + printf '%s\n' "$1" + echo "FAIL: $3" + exit 1 + fi +} +expect() { # expect + if ! grep -qE "$2" <<< "$1"; then + printf '%s\n' "$1" + echo "FAIL: $3" + exit 1 + fi +} + +# ── 1. Steady state: a resolvable dependency costs no network ───────────── +# The descriptor is present and the constraint is satisfiable, so the decision +# is "resolvable locally" no matter how old the marker is. The build is run +# offline so that a regression here surfaces as a missing/extra decision line +# rather than as a download. +manifest 'widget = "1.2.0"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'widget@1.2.0: resolvable locally' \ + 'a locally-resolvable dependency should be reported as such' +refute "$log" 'Refreshing +package index|Updating +package index' \ + 'a 30-day-old marker must NOT trigger a refresh when the index can answer' + +# A SemVer constraint that the local versions satisfy is equally steady. +manifest 'widget = "^1.2"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'widget@\^1.2: resolvable locally' \ + 'a satisfiable constraint should not be a version miss' +refute "$log" 'Refreshing +package index' \ + 'a satisfiable constraint must not trigger a refresh' + +# ── 2. A real miss is reported as one, and offline suppresses the sync ──── +manifest 'absentpkg = "1.0.0"' +log=$(MCPP_VERBOSE=1 MCPP_OFFLINE=1 "$MCPP" build 2>&1 || true) +expect "$log" 'absentpkg@1.0.0: offline mode' \ + 'offline mode must suppress the refresh and say so' +refute "$log" 'Refreshing +package index' \ + 'offline mode must not reach the network' +# The failure the user actually sees carries the index identity and age, so +# "not found" and "your index is from last month" are distinguishable. +expect "$log" 'index: local index abc1234' \ + 'the resolution failure should name the index revision' +expect "$log" 'mcpp index update' \ + 'the resolution failure should point at the explicit refresh command' + +# ── 3. INV-3: an unrefutable miss is not a miss ─────────────────────────── +# Neither of these may trigger a refresh. `xim` descriptors declare no +# namespace so `(xim, nasm)` can never match the identity gate, and a +# third-party namespace is outside the identities mcpp defines at all. +for dep in 'xim.nasm = "2.16.03"' 'somevendor.thing = "1.0.0"'; do + manifest "$dep" + log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline + expect "$log" 'no index can refute this' \ + "a miss under an unrefutable namespace must be inconclusive: $dep" + refute "$log" 'Refreshing +package index' \ + "an unrefutable miss must never trigger a refresh: $dep" +done + +# ── 4. Debounce: a second miss inside the window does not re-sync ───────── +: > "$INDEX/.mcpp-index-updated" # "just refreshed" +manifest 'absentpkg = "1.0.0"' +log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline +expect "$log" 'index was just refreshed' \ + 'a miss moments after a sync must be debounced, not re-synced' +refute "$log" 'Refreshing +package index' \ + 'debounce must prevent the second sync' + +# ── 5. [index] auto_refresh = false is honoured ─────────────────────────── +touch -d '30 days ago' "$INDEX/.mcpp-index-updated" 2>/dev/null \ + || touch -A -300000 "$INDEX/.mcpp-index-updated" 2>/dev/null || true +printf '\n[index]\nauto_refresh = false\n' >> "$MCPP_HOME/config.toml" +log=$(MCPP_VERBOSE=1 "$MCPP" build 2>&1 || true) # deliberately NOT offline +expect "$log" 'auto_refresh = false' \ + 'the config opt-out must suppress the refresh and name itself' +refute "$log" 'Refreshing +package index' \ + 'auto_refresh = false must prevent the sync' + +# ── 6. `mcpp index update` refuses offline instead of silently no-opping ── +log=$(MCPP_OFFLINE=1 "$MCPP" index update 2>&1 || true) +expect "$log" 'offline mode is on' \ + 'an explicit index update must refuse audibly when offline' + +# ── 7. `mcpp index status` reports the index revision ───────────────────── +log=$(MCPP_OFFLINE=1 "$MCPP" index status 2>&1 || true) +expect "$log" 'revision' 'index status should have a revision column' +expect "$log" 'abc1234' 'index status should show the local index revision' + +echo "PASS: index refresh is driven by local resolvability, not by the clock" diff --git a/tests/unit/test_pm_index_refresh.cpp b/tests/unit/test_pm_index_refresh.cpp new file mode 100644 index 00000000..a4a912a9 --- /dev/null +++ b/tests/unit/test_pm_index_refresh.cpp @@ -0,0 +1,336 @@ +// The judgement table of mcpp.pm.index_refresh, row by row. +// +// This file is the contract for #315: a build refreshes the package index when +// — and only when — the local copy cannot answer. The rows worth staring at: +// +// InconclusiveNamespaceNeverRefreshes — the one that decides whether this +// whole change is an improvement or a regression. xim descriptors declare +// no `namespace`, so `(xim, nasm)` can never satisfy the identity gate; if +// that miss counted, every build with a xim dependency would sync EVERY +// TIME, which is worse than the hourly TTL being removed. +// ExactVersionAbsentIsNotAVersionMiss — version tables are per-OS, so "this +// host publishes no 1.2.3" is not "1.2.3 does not exist". +// ColdStartIgnoresDebounce — an absent index has nothing to debounce against. + +#include + +import std; +import mcpp.config; +import mcpp.pm.dep_spec; +import mcpp.pm.index_refresh; +import mcpp.pm.index_route; +import mcpp.pm.index_spec; +import mcpp.platform.axis; +import mcpp.xlings; + +namespace { + +using mcpp::pm::RefreshReason; + +// A throwaway MCPP_HOME whose layout matches what the builtin registry reader +// walks: /data//pkgs//.lua +struct FakeRegistry { + std::filesystem::path home; + + explicit FakeRegistry(std::string_view tag) { + home = std::filesystem::temp_directory_path() + / std::format("mcpp-index-refresh-{}-{}", tag, + std::filesystem::hash_value( + std::filesystem::path(std::string(tag)))); + std::filesystem::remove_all(home); + std::filesystem::create_directories(home / "data" / "mcpplibs" / "pkgs"); + } + ~FakeRegistry() { std::error_code ec; std::filesystem::remove_all(home, ec); } + + std::filesystem::path index_dir() const { return home / "data" / "mcpplibs"; } + + void publish(std::string_view name, std::string_view versions) { + auto dir = index_dir() / "pkgs" / std::string(1, name.front()); + std::filesystem::create_directories(dir); + std::ofstream(dir / std::format("{}.lua", name)) << std::format(R"( +package = {{ + spec = "1", + namespace = "mcpplibs", + name = "{}", + type = "package", + xpm = {{ + linux = {{ {} }}, + macosx = {{ {} }}, + windows = {{ {} }}, + }}, +}} +)", name, versions, versions, versions); + } + + // Age the refresh marker by writing it and back-dating it. + void mark_refreshed(std::chrono::seconds ago = std::chrono::seconds{0}) { + auto marker = index_dir() / ".mcpp-index-updated"; + std::ofstream(marker) << "ok\n"; + std::error_code ec; + std::filesystem::last_write_time( + marker, std::filesystem::file_time_type::clock::now() - ago, ec); + } + + void remove_index() { std::filesystem::remove_all(index_dir()); } + + mcpp::xlings::Env env() const { return mcpp::xlings::Env{ .home = home }; } + + mcpp::config::GlobalConfig config() const { + mcpp::config::GlobalConfig cfg; + cfg.registryDir = home; + return cfg; + } +}; + +mcpp::pm::DependencySpec version_dep(std::string_view ns, + std::string_view shortName, + std::string_view version) { + mcpp::pm::DependencySpec spec; + spec.namespace_ = std::string(ns); + spec.shortName = std::string(shortName); + spec.version = std::string(version); + spec.candidates.push_back(mcpp::pm::DependencyCoordinate{ + .namespace_ = std::string(ns), .shortName = std::string(shortName) }); + return spec; +} + +RefreshReason decide(const mcpp::pm::IndexRoute& route, + const mcpp::pm::DependencySpec& spec, + const mcpp::xlings::Env& env, + mcpp::pm::RefreshPolicy policy = {}) { + return mcpp::pm::decide_for_dependency( + route, spec.shortName, spec, env, + mcpp::platform::TargetPlatform::for_os("linux"), policy).reason; +} + +bool refreshes(const mcpp::pm::IndexRoute& route, + const mcpp::pm::DependencySpec& spec, + const mcpp::xlings::Env& env, + mcpp::pm::RefreshPolicy policy = {}) { + return mcpp::pm::decide_for_dependency( + route, spec.shortName, spec, env, + mcpp::platform::TargetPlatform::for_os("linux"), policy).shouldRefresh; +} + +constexpr std::string_view kOneVersion = R"(["1.2.0"] = { url = "u", sha256 = "s" },)"; + +} // namespace + +// ─── Rows that never reach the filesystem ──────────────────────────────── + +TEST(PmIndexRefresh, PathAndGitDependenciesNeverRefresh) { + FakeRegistry reg("srcdeps"); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + mcpp::pm::DependencySpec pathDep; + pathDep.path = "../sibling"; + EXPECT_EQ(decide(route, pathDep, reg.env()), RefreshReason::None); + + mcpp::pm::DependencySpec gitDep; + gitDep.git = "https://example.invalid/x.git"; + EXPECT_EQ(decide(route, gitDep, reg.env()), RefreshReason::None); +} + +TEST(PmIndexRefresh, OfflineSuppressesEvenAGenuineMiss) { + FakeRegistry reg("offline"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + mcpp::pm::RefreshPolicy offline; offline.offline = true; + + EXPECT_EQ(decide(route, spec, reg.env(), offline), RefreshReason::SuppressedOffline); + EXPECT_FALSE(refreshes(route, spec, reg.env(), offline)); +} + +TEST(PmIndexRefresh, OfflineDoesNotMisreportAResolvableDependency) { + // The opt-outs are applied AFTER the local analysis, so a dependency that + // resolved fine is reported as such even offline. Short-circuiting on the + // flag first would answer "offline mode" for every dependency — hiding + // exactly what an offline user is checking. + FakeRegistry reg("offlinesteady"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + mcpp::pm::RefreshPolicy offline; offline.offline = true; + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "1.2.0"), reg.env(), offline), + RefreshReason::None); +} + +TEST(PmIndexRefresh, AutoRefreshOptOutSuppressesAGenuineMiss) { + FakeRegistry reg("optout"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + mcpp::pm::RefreshPolicy off; off.autoRefresh = false; + + EXPECT_EQ(decide(route, spec, reg.env(), off), RefreshReason::SuppressedDisabled); +} + +// ─── INV-3: a miss must be refutable to mean anything ──────────────────── + +TEST(PmIndexRefresh, InconclusiveNamespaceNeverRefreshes) { + FakeRegistry reg("inconclusive"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // `xim` and any third-party namespace: the builtin registry is what would + // answer, but it is not authoritative for identities mcpp does not define, + // so "absent" here is not evidence of anything. + for (auto ns : { "xim", "somevendor" }) { + auto spec = version_dep(ns, "thing", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::SuppressedInconclusive) + << "namespace: " << ns; + EXPECT_FALSE(refreshes(route, spec, reg.env())) << "namespace: " << ns; + } +} + +TEST(PmIndexRefresh, LazyGitIndexNeverRefreshesTheGlobalOne) { + FakeRegistry reg("lazygit"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + indices["acme"] = mcpp::pm::IndexSpec{ .name = "acme", .url = "https://x/y" }; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // Served by a custom index; syncing the shared registry would not help. + auto spec = version_dep("acme", "widget", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::None); +} + +// ─── The three ways local state fails to answer ────────────────────────── + +TEST(PmIndexRefresh, PresentDescriptorIsTheSteadyStateAndCostsNothing) { + FakeRegistry reg("steady"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); // old, and irrelevant + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "1.2.0"), reg.env()), + RefreshReason::None); + EXPECT_EQ(decide(route, version_dep("mcpplibs", "widget", "^1.2"), reg.env()), + RefreshReason::None); +} + +TEST(PmIndexRefresh, MissingDescriptorTriggersARefresh) { + FakeRegistry reg("descmiss"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::DescriptorMiss); + EXPECT_TRUE(refreshes(route, spec, reg.env())); +} + +TEST(PmIndexRefresh, UnsatisfiableConstraintTriggersARefresh) { + FakeRegistry reg("vermiss"); + reg.publish("widget", kOneVersion); // only 1.2.0 exists + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "widget", "^9.9"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::VersionMiss); +} + +TEST(PmIndexRefresh, ExactVersionAbsentIsNotAVersionMiss) { + FakeRegistry reg("exactver"); + reg.publish("widget", kOneVersion); // only 1.2.0 exists + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // Version tables are per-OS: an exact pin the local table does not list may + // simply not be built for this host. Refreshing on it would fire on EVERY + // build for such a dependency. The install layer covers the real case. + auto spec = version_dep("mcpplibs", "widget", "9.9.9"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::None); +} + +// ─── Debounce ──────────────────────────────────────────────────────────── + +TEST(PmIndexRefresh, DebounceSuppressesTheSecondMissInAWindow) { + FakeRegistry reg("debounce"); + reg.publish("widget", kOneVersion); + reg.mark_refreshed(std::chrono::seconds{5}); // just synced + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "absent", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::SuppressedDebounce); + EXPECT_FALSE(refreshes(route, spec, reg.env())); +} + +TEST(PmIndexRefresh, ColdStartIgnoresDebounce) { + FakeRegistry reg("coldstart"); + reg.mark_refreshed(std::chrono::seconds{5}); + reg.remove_index(); // marker gone with it; no local index at all + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto spec = version_dep("mcpplibs", "widget", "1.0.0"); + EXPECT_EQ(decide(route, spec, reg.env()), RefreshReason::IndexAbsent); + EXPECT_TRUE(refreshes(route, spec, reg.env())); +} + +// ─── decide_for_miss (the `mcpp add` half) ─────────────────────────────── + +TEST(PmIndexRefresh, DecideForMissHonoursTheSameOptOutsAndDebounce) { + FakeRegistry reg("formiss"); + reg.mark_refreshed(std::chrono::hours{99}); + auto env = reg.env(); + + mcpp::pm::RefreshPolicy allowed; + EXPECT_TRUE(mcpp::pm::decide_for_miss(allowed, env, "pkg").shouldRefresh); + + mcpp::pm::RefreshPolicy offline; offline.offline = true; + EXPECT_EQ(mcpp::pm::decide_for_miss(offline, env, "pkg").reason, + RefreshReason::SuppressedOffline); + + mcpp::pm::RefreshPolicy disabled; disabled.autoRefresh = false; + EXPECT_EQ(mcpp::pm::decide_for_miss(disabled, env, "pkg").reason, + RefreshReason::SuppressedDisabled); + + // Two `mcpp add` typos in a row must not buy two multi-repo syncs. + reg.mark_refreshed(std::chrono::seconds{5}); + EXPECT_EQ(mcpp::pm::decide_for_miss(allowed, env, "pkg").reason, + RefreshReason::SuppressedDebounce); +} + +TEST(PmIndexRefresh, SubjectEchoesWhatTheUserWrote) { + FakeRegistry reg("subject"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + // The manifest key `xim.nasm` parses into the candidate (mcpplibs.xim, + // nasm); a diagnostic naming that spelling sends the reader hunting for a + // namespace they never typed. + auto spec = version_dep("mcpplibs.xim", "nasm", "2.16.03"); + auto d = mcpp::pm::decide_for_dependency( + route, "xim.nasm", spec, reg.env(), + mcpp::platform::TargetPlatform::for_os("linux"), {}); + EXPECT_EQ(d.subject, "xim.nasm@2.16.03"); +} diff --git a/tests/unit/test_xlings.cpp b/tests/unit/test_xlings.cpp index 01a54a1b..d7afd3d8 100644 --- a/tests/unit/test_xlings.cpp +++ b/tests/unit/test_xlings.cpp @@ -454,3 +454,70 @@ TEST(XlingsSeed, ArtifactAndSourceEmittedWhenSet) { std::string::npos) << text; std::filesystem::remove_all(dir); } + +// ─── Index identity + clock-skew hardening (#315) ──────────────────────── + +TEST(XlingsIndexFreshness, FutureMarkerIsStaleNotEternallyFresh) { + auto home = make_tempdir("mcpp-xlings-index-future"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + auto marker = dir / ".mcpp-index-updated"; + std::ofstream(marker) << "ok\n"; + // Clock skew in a container, a tar that preserved mtimes, a restored CI + // cache: a marker in the future used to yield a negative age, which read as + // "fresh" and stayed that way until the wall clock caught up. + std::error_code ec; + std::filesystem::last_write_time( + marker, std::filesystem::file_time_type::clock::now() + std::chrono::hours{72}, ec); + + mcpp::xlings::Env env{.home = home}; + auto st = mcpp::xlings::default_index_status(env, 3600); + + EXPECT_FALSE(st.fresh); + EXPECT_EQ(st.ageSeconds, -1); // unusable timestamp reports as unknown + + std::filesystem::remove_all(home); +} + +TEST(XlingsIndexRevision, ReadsTrimmedValueAndDegradesToNullopt) { + auto home = make_tempdir("mcpp-xlings-index-rev"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + + // Absent: a local `path` index has no such file — must not be an error. + EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + + // Present. The observed files carry no trailing newline, but a writer that + // added one (or a \r from Windows) must not change the value, or every run + // would look like the index had changed. + std::ofstream(dir / ".xlings-index-version") << " 8d67478\r\n"; + auto rev = mcpp::xlings::index_revision(dir); + ASSERT_TRUE(rev.has_value()); + EXPECT_EQ(*rev, "8d67478"); + + // Blank is not an identity. + std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "\n"; + EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + + // The value is OPAQUE: sub-indexes carry a date version, not a sha. + std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "2026.7.30.1"; + EXPECT_EQ(mcpp::xlings::index_revision(dir).value(), "2026.7.30.1"); + + std::filesystem::remove_all(home); +} + +TEST(XlingsIndexRevision, StatusCarriesTheRevision) { + auto home = make_tempdir("mcpp-xlings-index-rev-status"); + auto dir = home / "data" / "mcpplibs"; + std::filesystem::create_directories(dir / "pkgs"); + std::ofstream(dir / ".mcpp-index-updated") << "ok\n"; + std::ofstream(dir / ".xlings-index-version") << "abc1234"; + + mcpp::xlings::Env env{.home = home}; + auto st = mcpp::xlings::default_index_status(env, 3600); + + ASSERT_TRUE(st.rev.has_value()); + EXPECT_EQ(*st.rev, "abc1234"); + + std::filesystem::remove_all(home); +}