Skip to content

Preserve PostgreSQL checkpoint history and recover backups beyond 20,000 commits - #8759

Open
loopx-agent wants to merge 78 commits into
mainfrom
codex/pg-authority-checkpoint-seed-20261114
Open

loopx-agent wants to merge 78 commits into
mainfrom
codex/pg-authority-checkpoint-seed-20261114

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

PostgreSQL v1 retains a complete, verifiable journal with append-only checkpoints for restricted runtime roles. The offline backup verifier also reads through a captured frontier above 20,000 commits.

Review fixes

  • Existing v0 databases. Setup carries the old journal's table-level SELECT and INSERT grants, including group grants, into the checkpoint table. Optional runtime-role provisioning and the format change share the existing PostgreSQL transaction. The released v0 client can still read its original receipt after an intentionally failed role provision, and the same restricted worker can cross cursor 65 after the successful upgrade. History changes and schema changes remain denied to the runtime role.
  • Seed retries. The CLI accepts an existing regular output file only when its bytes match the requested seed. It publishes complete new files without clobbering the destination. Conflicting files and symbolic links fail before the target connection. Concurrent and sequential replays through File and SQLite returned the same original PostgreSQL receipt and one commit at cursor 1.

The retained journal and recovery archive continue to carry history, events and receipts. The administrative seed command operates on a sanitized synthetic head. See the exact-head review and schema upgrade, rollback and seed instructions.

Same-workload local PostgreSQL run

PostgreSQL 16.2, 20,001 actual commits per format, an 8,312-byte projection, one original event and one original receipt per commit. The same native archive owner exported and audited both formats through cursor 20,001 and obtained the same final projection digest.

Observation Released v0 Checkpoints and deltas
Journal relations, including indexes and TOAST 172.3 MiB 12.9 MiB
Current-head table, including retained physical row versions 148.5 MiB 138.7 MiB
20,001 ordinary appends 37.29 s 48.86 s
Latest head read p95, 200 samples 0.84 ms 0.65 ms
Receipt read p95, 200 samples 0.82 ms 0.57 ms
Tail page of 32 rows p95, 50 samples 2.49 ms 2.97 ms
Complete export and verification 4.65 s 5.26 s
Complete source audit 12.53 s 6.28 s

An isolated target restored all 20,001 commits from the 10.9 MiB archive through ordinary writes in 50.22 s, then independently matched the full journal and original receipts in 5.70 s. The entire client run used 55.61 CPU-seconds and 291.9 MiB maximum RSS. This was one local sample. Database process CPU and RSS, multi-tenant throughput, deployment capacity, and the earlier PostgreSQL 16.15 sample have separate measurement boundaries.

Validation at 2aae8f435ea8

398 real PostgreSQL store tests, 11 service tests, 47 remote authority tests, 4 real archive interchange tests, 73 Python CLI and recovery tests, and 17 diff-selected premerge validators passed. The exact-head TypeScript typecheck, Ruff, mypy, bilingual RFC and public-diff checks also passed. The 73 Python cases include full SQLite backup verification at 20,001 commits. Manifest counts retain exact positive-decimal validation, and the independent archive audit verifies the claimed history.

The full TypeScript suite passed 6,544 tests with 89 expected skips and one duplicate digest-owner failure. The full Python suite reported a similar architecture failure after 4,327 passes and was stopped for diagnosis. The same failing checks reproduce at the unchanged base f7e3f427197b: one native child result matcher in TypeScript and duplicate Chat/Lark matchers in Python. These failures hold the managed quality gate. The Python suite beyond that point remains unverified.

Fresh independent review and hosted CI are pending. Joint PostgreSQL service and cutover qualification remains with the related delivery program.

…rk-source-image-return-20261114

Signed-off-by: LoopX Agent <[email protected]>
…rk-source-image-return-20261114

Signed-off-by: LoopX Agent <[email protected]>
…-pg-seed exporter

- Add authority_checkpoints table DDL and tenant-scoped RLS policies.
- Store incremental delta in authority_commits and bounded checkpoints every 64 commits and on genesis cursor 1, resolving table bloat and the 20,000 commit recovery ceiling.
- Implement scanCommitted bounded delta replay from covering checkpoint via AuthorityStateReplay.
- Add setupPostgreSqlAuthoritySchemaAndRole for explicit schema installation and least-privilege restricted role assembly.
- Add exportLocalToPgGenesisSeed with sanitization (expired leases, local absolute paths, .local/ references, Feishu IDs, private materials) and fail-closed privacy boundary scan.
- Add loopx postgres setup and loopx postgres local-to-pg-seed CLI commands.
- Add test suites for checkpoints, sanitization, role assembly, and CLI invocation.

Signed-off-by: LoopX Agent <[email protected]>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

审阅提交 bda7463f66f39324960fb9f468d7f8ef3b43c818,GitHub 目标基线 859a23ee9534539536343206e23452253528dd07,实际合并基点 859a23ee9534539536343206e23452253528dd07。

[P1] 新格式写入后无法重建已提交的历史。 对隔离 PG 16.15 同一 Goal 提交三笔,用生产 scanCommitted 读回;再在旧库升级一次、新写一笔后重试。 旧库历史在迁移后、新写入之前可读,写入第四笔后返回 provider_read_unavailable;新库三笔同样无法扫历史。真实 store suite 旧版 397/397,新版 386/397。共享 AuthorityStateReplay 只提供 apply,当前调用了不存在的 applyDelta;最新 head/原回执仍存在,所以只核对 latest 会误判。 改为调用现有 apply 并补上原有完整游标、缺行拒绝与迁移/恢复测试。运行 npm run typecheck:control-plane 和 npm run test:postgresql-authority-store;覆盖旧库升级后新写入,再从原游标回放。 代码位置

[P1] 按新 setup 建成的受限角色无法提交第一笔事务。 实际运行 loopx postgres setup --execute 创建受限运行角色;使用该角色给空 Goal 提交首笔,游标 1 会写检查点。 setup 显示 installed,角色对检查点只有 SELECT/INSERT。首笔 SQL 使用 ON CONFLICT DO UPDATE,即使没有冲突也需要 UPDATE,PostgreSQL 返回 42501,事务回滚且没有 head。 复用现有锁和唯一约束,检查点走只追加的 INSERT,避免给普通运行角色修改过去检查点的权限。让真实 setup 生成的受限角色完成首次写入、普通更新和跨检查点写入,并确认跨租户与改删历史仍被拒绝。 代码位置

动机 / Motivation

管理员在隔离 PostgreSQL 中准备用最小权限运行共享权威存储,并要在更新后读回历史工作和回执。 一个已有三笔提交的 Goal 升级后还能读回原有历史,但以新格式写入第四笔时,提交成功,整段历史却读不出来。 迁移和恢复依赖可读的历史;如果只核对当前 head 或只读当前回执,会把不可恢复的状态当成已经交付。 引入有界检查点能减少重复存储;实测现有 seed 的正常提交可读回,但新格式历史回放和受限角色首笔提交仍失败。 本轮只评估 PG 检查点、管理员设置、隔离种子导出和受影响的现有读写;部署、切流及整组三切片验收仍待独立工作。

改动思路 / Approach

PostgreSqlAuthorityStore 持有事务与租户过滤;AuthorityStateReplay 管增量格式;仅显式管理员准备 schema。 CLI 校验管理员或导出参数 → effect router → PostgreSQL 事务读写/File 导出 → 同事务 head、历史与回执;恢复阅读旧或新记录。 迁移只变数据库存储格式;检查点、增量、事件和回执同时提交。种子 preview 不连接目标数据库。 维持现有全投影可保读回但存储继续增长;复用共享回放和现有事务 owner 可达到目标。独立发布紧耦合的写入与读回会留下不可恢复的存量。 让 PostgreSQL 直接沿用 File 和 SQLite 的现有回放接口,并把普通写入保持为追加;管理员设库应与运行时种子提交拥有不同权限。 本轮只评估 PG 检查点、管理员设置、隔离种子导出和受影响的现有读写;部署、切流及整组三切片验收仍待独立工作。

具体改动 / Changes

规范依据:docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md,修订 859a23ee9534539536343206e23452253528dd07。按已有存储与服务约束核对,不把隔离 PG 中的局部成功当成部署资格。 +1538/-21(7 个运行时文件、3 个测试文件)。 四个 CLI/facade 文件走同一 effect handler;两个 TS owner 承担设库、授权、检查点回放、种子导出;三个文件是单测和 CLI 回归。

  • history_exact 未满足。每笔已提交的历史投影和回执可完整读回,迁移前后同语义。 对应代码或规范
  • restricted_runtime 未满足。管理员建库,受限 runtime 可以提交但不能改历史或跨租户。 对应代码或规范
  • service_isolation 已验证。现有服务继续按租户和数据库身份拒绝错配。 对应代码或规范

关键代码讲解

  • installPostgreSqlAuthorityStoreSchema:安装或升级检查点表、RLS 和增量字段。事务中核实原 incarnation,旧历史行保留;不匹配回滚。
  • commitAuthority:受限事务在首个游标写检查点。首笔即命中 ON CONFLICT UPDATE 所需权限;SQL 42501 时首笔整事务回滚。
  • scanCommitted:从近处检查点向前重建历史。共享回放接口名错误,任何新的非检查点 delta 无法处理;TypeError 被包装为 provider_read_unavailable。
  • setupPostgreSqlAuthoritySchemaAndRole:创建受限的 NOLOGIN 组角色和表级授权。检查点表只有 SELECT/INSERT;保持追加语义,而非赋予改历史权限。
  • handle_postgres_command:导出或显式提交种子。未要求写入时不向目标传 URL;无需自动迁移或配置变更。

对主干的风险 / Risks

  • PASS · 旧版真实 PG 权威契约。基线 397/397,0 skip。
  • PASS · 服务鉴权 / 轮转 / 远程事务和正常 CLI。11+40 项真实 PG 服务与远程测试、19 项 Python、21 项 Node 单测通过;Node 单测含一项独立跳过。普通 seed preview、显式提交和重复目标拒绝实测。
  • FAIL · 历史 Todo、迁移、扫描、受限调用。386 通过、11 失败,0 skip。另用真实旧库与新库重现;当前 head 可读但历史 unavailable。
  • FAIL · 仅用 setup 生成角色的首笔写入。42501 权限错误,零 head;最小权限合同未达。
  • FAIL · 共享回放和 effect runtime 类型。基线通过;当前 4 处类型错误,包括不存在的回放方法和 handler 输入类型。
  • FAIL · 差异空白。两个新增测试在 EOF 有空行。

语义与 CI 对齐

本地类型检查另报四处新错误:回放接口不存在、缺少事务类型、seed 局部收窄丢失,以及 effect handler 入参不满足通用边界。请修类型而非把未知输入直接断言成 seed 请求。 在同一 PG 16.15 服务器中,旧版 397 项全部通过,新版 386 通过、11 失败;首次新格式写入后历史扫描 unavailable,且受限角色首笔写入因缺少检查点 UPDATE 权限失败并回滚。 在现有回放 owner 改用实际接口,验证游标连续和历史恢复;检查点使用单次 INSERT,由同事务、主键和原有回执处理重复;再让管理员 setup 和受限连接分开验收。 仅隔离 PostgreSQL 16.15 与源码 CLI;未验证已安装分发、生产部署、三切片联合、20,000+ 历史容量或托管 CI。当前实测失败足以保留合入阻断。

我的整体评价 / Verdict

REQUEST_CHANGES。当前新历史会阻断长期恢复,受限运行角色会阻断用户首次写入。两项都有真实 PG 反例。回放修复与最小权限写入通过后再做联合 PG 检查。 已有 PostgreSQL 部署执行 schema 更新后会写入新历史格式;旧版对新格式的读取与回滚未建立。PR 描述覆盖新功能,但低估了既有 PG 恢复调用面。 让 PostgreSQL 直接沿用 File 和 SQLite 的现有回放接口,并把普通写入保持为追加;管理员设库应与运行时种子提交拥有不同权限。 本轮只评估 PG 检查点、管理员设置、隔离种子导出和受影响的现有读写;部署、切流及整组三切片验收仍待独立工作。

English verdict: REQUEST_CHANGES at bda7463. On PostgreSQL 16.15 the new delta path cannot replay committed history, and the role created by setup cannot append its first checkpoint. Repair both on the real store and keep runtime privileges append-only.

projection = checkpointProjection;
} else {
if (raw.delta !== null && raw.delta !== undefined) {
replay.applyDelta(raw.delta as unknown as AuthorityStateDelta);

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] 新格式写入后无法重建已提交的历史。 对隔离 PG 16.15 同一 Goal 提交三笔,用生产 scanCommitted 读回;再在旧库升级一次、新写一笔后重试。 旧库历史在迁移后、新写入之前可读,写入第四笔后返回 provider_read_unavailable;新库三笔同样无法扫历史。真实 store suite 旧版 397/397,新版 386/397。共享 AuthorityStateReplay 只提供 apply,当前调用了不存在的 applyDelta;最新 head/原回执仍存在,所以只核对 latest 会误判。 改为调用现有 apply 并补上原有完整游标、缺行拒绝与迁移/恢复测试。运行 npm run typecheck:control-plane 和 npm run test:postgresql-authority-store;覆盖旧库升级后新写入,再从原游标回放。

`INSERT INTO loopx_control_plane.authority_checkpoints
(tenant_id, goal_id, cursor, projection, projection_digest)
VALUES ($1, $2, $3::bigint, $4::jsonb, $5)
ON CONFLICT (tenant_id, goal_id, cursor) DO UPDATE

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] 按新 setup 建成的受限角色无法提交第一笔事务。 实际运行 loopx postgres setup --execute 创建受限运行角色;使用该角色给空 Goal 提交首笔,游标 1 会写检查点。 setup 显示 installed,角色对检查点只有 SELECT/INSERT。首笔 SQL 使用 ON CONFLICT DO UPDATE,即使没有冲突也需要 UPDATE,PostgreSQL 返回 42501,事务回滚且没有 head。 复用现有锁和唯一约束,检查点走只追加的 INSERT,避免给普通运行角色修改过去检查点的权限。让真实 setup 生成的受限角色完成首次写入、普通更新和跨检查点写入,并确认跨租户与改删历史仍被拒绝。

loopx-agent and others added 27 commits November 15, 2026 13:57
Signed-off-by: lusendong.6789 <[email protected]>
Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>
…t-20261115

Defer report providers during Turn reads
…reviewed-20261115

test(core): require installed provenance before qualification
Load the optional PR reconciler only after the invocation-time observation permission check, inside the existing provider failure boundary. Preserve original permission, cadence, result and unknown-effect owners, and document the bounded default-closure and timing-window change.

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>
* Skip automatic usage observers for supported opt-outs

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>

* Record installed Core and C1 observer qualification

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>

---------

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>
…comparison (#8760)

* docs(blog): add EdgeBench controlled Context ON vs OFF 10-task score comparison

Signed-off-by: LoopX Agent <[email protected]>

* docs(blog): use declarative mechanism-led opening in zh context-delivery score section

Signed-off-by: LoopX Agent <[email protected]>

---------

Signed-off-by: LoopX Agent <[email protected]>
* Package the manager discovery projection

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>

* Record selected manager discovery distribution qualification

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>

---------

Signed-off-by: lusendong.6789 <[email protected]>
Co-authored-by: TRAE CLI <[email protected]>
Port the probe fixture and native Windows regression coverage from e6eb92d and 8d200aa. Decode the generated PowerShell literals into argv before launching the pinned reader.

Signed-off-by: LoopX Agent <[email protected]>
…moke-20261115

Keep creation-only intent in the saved Goal browser check
…s into one owner (#8826)

reviewer_recommendation.py restated the trust and freshness sets that
repository_context.py already owns. Import both as aliases of the owner
objects so the decision has a single source, and add an architecture guard
that pins single ownership by value, by identity, and by wiring through both
public builders. REVIEWER_SOURCE_KINDS stays local (deliberately narrower).

Signed-off-by: psychologister <[email protected]>
Co-authored-by: psychologister <[email protected]>
…llow-up route (#8708)

* fix(scheduler): reject superseded heartbeat receipts on the Python follow-up route

The Python compatibility route for scheduler-ack-current/scheduler-fail-current
checked only that the receipt for --turn-instance-id exists; a receipt already
superseded by a newer heartbeat Turn still committed scheduler state, while the
native TypeScript transaction rejects it as heartbeat_receipt_stale.

Thread the Turn from build_scheduler_followup_payload through
record_quota_scheduler_ack / record_quota_scheduler_ack_for_decision /
record_quota_scheduler_failure_for_decision into _run_native_followup, which now
sends turn_instance_id with require_heartbeat_receipt=true. The TypeScript
transaction remains the single owner of the missing/stale/superseded fence (it
also rechecks under the scheduler state lock); optional parameters keep the
in-process API, which has no Turn, unchanged.

Fixes #8605

Signed-off-by: Lihua <[email protected]>

* Keep scheduler follow-up hint reads out of legacy migration

Signed-off-by: LoopX Agent <[email protected]>

---------

Signed-off-by: Lihua <[email protected]>
Signed-off-by: LoopX Agent <[email protected]>
Co-authored-by: LoopX Agent <[email protected]>
* fix(replan): resume settlement after lifecycle review writeback

Signed-off-by: LoopX Agent <[email protected]>

* Recover committed replan settlement before rebuilding its review

Signed-off-by: LoopX Agent <[email protected]>

---------

Signed-off-by: LoopX Agent <[email protected]>
* Recheck captured work before launching an external Turn host

Signed-off-by: LoopX Agent <[email protected]>

* Keep complete-source metadata assembly outside Turn planning

Signed-off-by: LoopX Agent <[email protected]>

---------

Signed-off-by: LoopX Agent <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants