-
-
Save denniswon/c9e73e39e14f8de1230d4253f46f6bda to your computer and use it in GitHub Desktop.
| ⏺ You're right — I was stuck in the wrong substrate model. Let me course-correct. | |
| ★ Insight ───────────────────────────────────── | |
| - I was treating "state tree" as the private-data substrate and "EVM contract storage" as the public-data substrate, as if Sub-project A's | |
| fabric didn't apply here. It does apply. The state tree is a content-addressed key-value store with on-chain root commitments — it has nothing | |
| inherently private about it. Encrypted leaves = private data (Sub-project A's SecretsRegistry); plaintext leaves = public data (Sub-project B's | |
| artifacts). The substrate is the same, only leaf encryption differs. | |
| - On-chain, we only ever need the hash + root commitments. For a specific task, NewtonPolicy.policyCodeHash already commits to the Rego bytes — | |
| any fetcher verifies keccak256(bytes) == policyCodeHash and that closes the integrity loop regardless of where bytes live. For registry-level | |
| questions ("is this plugin still approved?"), the 120s BLS-signed state root + inclusion proof does the job. SSTORE2 never enters the picture | |
| because EVM itself never consumes the bytes. | |
| - This collapses Sub-project B's infrastructure cost dramatically. The state-tree + EigenDA + BLS-commit + bootnode + gateway-orchestration | |
| fabric is already being built in Sub-project A. Sub-project B adds more registries on top of that fabric — different data types, different leaf | |
| semantics, but the same plumbing. It's additive, not a parallel track. | |
| ───────────────────────────────────────────────── | |
| Corrected model — where each thing lives | |
| ┌──────────────────────────────┬────────────────────────────────────────────────┬────────────────────────────┬────────────────────────────┐ | |
| │ What │ Where │ Who writes │ Who reads │ | |
| ├──────────────────────────────┼────────────────────────────────────────────────┼────────────────────────────┼────────────────────────────┤ | |
| │ keccak256(rego_bytes) → Rego │ Operator-local state tree (plaintext leaf), │ Gateway-orchestrated │ Operators / challengers / │ | |
| │ source │ backed by EigenDA delta blob │ operator broadcast on │ SP1 prover, via signed RPC │ | |
| │ │ │ policy deploy │ + inclusion proof │ | |
| ├──────────────────────────────┼────────────────────────────────────────────────┼────────────────────────────┼────────────────────────────┤ | |
| │ keccak256(schema_bytes) → │ │ │ Same — signed RPC from │ | |
| │ data.params / data.wasm │ Same — state tree leaf │ Same flow │ operator │ | |
| │ schema │ │ │ │ | |
| ├──────────────────────────────┼────────────────────────────────────────────────┼────────────────────────────┼────────────────────────────┤ | |
| │ keccak256(wasm_bytes) → WASM │ Same — state tree leaf (MB-scale leaf is fine; │ Same │ Same │ | |
| │ plugin binary │ redb handles arbitrary value sizes) │ │ │ | |
| ├──────────────────────────────┼────────────────────────────────────────────────┼────────────────────────────┼────────────────────────────┤ | |
| │ │ Per-registry BLS-signed state root, committed │ │ EVM — used for │ | |
| │ On-chain commitment │ every 120s to the registry contract │ Operator quorum (same as │ inclusion-proof │ | |
| │ │ (implementing IStateRootCommittable) │ Sub-project A) │ verification in challenge │ | |
| │ │ │ │ paths │ | |
| ├──────────────────────────────┼────────────────────────────────────────────────┼────────────────────────────┼────────────────────────────┤ | |
| │ │ NewtonPolicy.policyCodeHash continues to point │ │ SP1 circuit / challenger / │ | |
| │ Existing on-chain hashes │ at the registered Rego hash; NewtonPolicyData │ Policy deployer │ operator (already the │ | |
| │ │ gets a wasmHash field replacing wasmCid │ │ four-site commitment │ | |
| │ │ │ │ pattern) │ | |
| └──────────────────────────────┴────────────────────────────────────────────────┴────────────────────────────┴────────────────────────────┘ | |
| What goes on-chain: hashes (as they already do via policyCodeHash) + per-registry state roots + registry metadata (deprecation flags, deploy | |
| timestamps). That's it. | |
| What goes in operator state + EigenDA: the actual bytes. | |
| What disappears: IPFS dependency, SSTORE2, MB-scale EVM storage cost. The "on-chain" guarantee in the brief is satisfied by the committed state | |
| root, not by bytes-in-EVM. | |
| What I was misunderstanding | |
| I was holding onto the EVM smart-contract analogy from your brief ("like how smart contracts are deployed/stored onchain") too literally — | |
| treating it as "bytes physically in contract code." You were using it as "protocol-committed content with on-chain trust anchors, fetchable | |
| from the protocol without external dependency." Under that second reading (which is actually what the brief requires), the state-tree substrate | |
| is a cleaner fit than SSTORE2. | |
| What this resolves | |
| - Q3 (storage substrate): state tree on the Sub-project A fabric, not SSTORE2. Applies uniformly to all four artifacts. | |
| - E5 for all four: confirmed — this is literally what the state-tree read path does (gateway-orchestrated for writes, operator signed-RPC reads | |
| for fetches, EigenDA as the blob-availability layer for snapshot/replay). | |
| - Q2 (addressing): content-hash remains the primary key. In state-tree terms: the tree key is the content hash. | |
| - Privacy extension (R5): free — encrypted leaves work the same as SecretsRegistry. The on-chain registry stores (cleartext_hash, | |
| TEE_attestation_requirement_flag) in its metadata; the leaf stores ciphertext. | |
| - Infrastructure reuse: Sub-project B becomes "three or four more IStateRootCommittable registries" (RegoRegistry + SchemaRegistry(ies) + | |
| WasmPluginRegistry), reusing the state-tree crate, EigenDA client, bootnode, gateway orchestration, and BLS commit flow from Sub-project A | |
| wholesale. | |
| What opens up next | |
| A few concrete decisions remain: | |
| 1. One registry vs several — single unified PolicyArtifactRegistry with typed entries (kind: Rego / Schema / WasmPlugin) sharing one state | |
| root, or separate RegoRegistry / SchemaRegistry / WasmPluginRegistry each with its own state root? Sub-project A chose separate (Identity / | |
| Confidential / Secrets) to keep trust boundaries clean. Same logic probably applies here. | |
| 2. Registry on which chain(s)? Sub-project A's three registries live on source chain (per your Linear issue decomposition). Policy artifacts | |
| probably same — but worth confirming because destination-chain challengers need to verify artifact presence too, which means either dest-chain | |
| registries or cross-chain root relay. | |
| 3. State-root commit cadence for artifact registries — Sub-project A uses 120s across all three registries. Do we lock-step all six+ registries | |
| (Sub-project A's three + Sub-project B's new ones) at the same 120s cadence? It simplifies operators' commit loop significantly. | |
| 4. Model X vs Model Y for WASM leaves — leaf stores bytes directly (X), or leaf stores metadata + EigenDA blob pointer (Y)? Model X makes state | |
| tree a single source of truth but grows the tree storage footprint; Model Y keeps the tree light but adds a fetch indirection. Sub-project A | |
| uses Model X for identity/confidential data (KB-scale leaves); for MB-scale WASM, Y might be preferable. | |
| Which of these do you want to dig into first? Q4 (single vs several registries) feels like the most foundational — it decides the on-chain | |
| contract surface and the trust boundaries between artifact kinds. | |
| You're right — I got Sub-project A's topology wrong in two places. Let me correct both before recommending on Q4. | |
| ★ Insight ───────────────────────────────────── | |
| - Sub-project A's registry topology is unified, not split. The V2 design collapsed IdentityRegistry / ConfidentialDataRegistry / | |
| SecretsRegistry (V1) into one StateCommitRegistry per chain with 1-byte namespace prefixes (0x01 identity / 0x02 confidential / 0x03 | |
| secrets) addressing a single unified JMT root (spec §3, lines 14, 68, 106, 194, 465). The spec explicitly rejects per-namespace | |
| operator-set partitioning (§S line 444): "it was the sole benefit of the V1 three-registry shape, and the cross-namespace policy-eval | |
| dependency made that partitioning useless in practice." | |
| - Sub-project A's registries deploy to every chain, not source-only. Spec §line 412 — "Three registries deploy per chain Newton | |
| supports"; §line 415 — "Destination chains (Base, L2s): full registry — operators maintain per-chain subtrees and post commits per | |
| chain"; §line 465 — "One StateCommitRegistry MUST be deployed per chain Newton supports. Each chain's registry is authoritative for | |
| that chain's unified state root and sequenceNo." No cross-chain state-root sync in Phase 1. | |
| - The grill-me context line at the top of this session ("Three registries with shared IStateRootCommittable interface — not a single | |
| PrivateDataStore") is stale — it reflects an earlier iteration of Sub-project A's design. The written spec supersedes it. I should | |
| have verified against the doc before anchoring recommendations to that line. | |
| ───────────────────────────────────────────────── | |
| Answers | |
| (2) Chain placement — you were right. Sub-project A's StateCommitRegistry lives on every chain Newton supports (source AND | |
| destinations), one per chain, each authoritative for its own unified state root. No cross-chain sync in Phase 1. So for Sub-project | |
| B: same default — one policy registry per chain. Dest-chain challengers get the inclusion-proof substrate for free. Flag for Q5 | |
| confirmation later, but this is the baseline. | |
| (3) 120s lockstep — locked in. All Sub-project A + Sub-project B registries commit on the same 120s boundary per chain. | |
| (1) Q4 recommendation — don't split. Unify into either one or two state trees. | |
| My prior "Sub-project A chose separate, same logic applies" was based on the wrong premise. The correct analogue is the opposite: | |
| Sub-project A unified, so Sub-project B should unify too. Two sub-options remain: | |
| Option: 1A. Extend existing StateCommitRegistry | |
| What it is: Add namespace tags 0x04 Rego / 0x05 Schema / 0x06 WasmPlugin to the same JMT. One root, one BLS sig, one commit tx per | |
| chain per 120s. | |
| Trust boundary: Same operator quorum already signs across identity/confidential/secrets. Extending to policy artifacts is additive. | |
| Plumbing cost: Zero new contracts. One more namespace prefix per artifact kind. | |
| ──────────────────────────────────────── | |
| Option: 1B. Second unified registry (PolicyArtifactRegistry) | |
| What it is: Separate JMT for policy artifacts, same IStateRootCommittable interface, parallel 120s commit. | |
| Trust boundary: Separate state root lets you bound "I trust the private-data quorum but not the policy-artifact quorum" — useful if | |
| the operator sets ever diverge. | |
| Plumbing cost: One new contract per chain. One more BLS aggregation + commit tx per 120s. Possible to run on the same operator set | |
| ❯ 1A agreed. Also, for (4), for cost, performance, reliability, what do you recommend? | |
| ⏺ 1A locked in. StateCommitRegistry extends with namespace tags 0x04 Rego / 0x05 Schema / 0x06 WasmPlugin. Zero new contracts, same | |
| operator quorum, same 120s lockstep commit. | |
| --- | |
| Model X vs Model Y on cost / performance / reliability | |
| Recommend Model X with per-commit size guard on all three axes. Reliability is the decisive one. | |
| Cost | |
| ┌──────────────────────────┬───────────────────────────────────┬──────────────────────────────────────────┐ | |
| │ Dimension │ Model X (bytes in leaf) │ Model Y (hash in leaf, bytes in EigenDA) │ | |
| ├──────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Per-operator disk │ ~200 MiB–1 GiB for 20–50 policies │ ~200 KiB (just hashes + metadata) │ | |
| ├──────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────┤ | |
| │ EigenDA write cost │ Bytes in delta blob │ Bytes in separate artifact blob (~same) │ | |
| ├──────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Bootnode/challenger disk │ Full bytes (~1 GiB) │ Hashes + ephemeral cached bytes │ | |
| ├──────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Network replication │ N operators × 1 GiB │ 1× EigenDA + per-operator cache │ | |
| └──────────────────────────┴───────────────────────────────────┴──────────────────────────────────────────┘ | |
| Y saves disk, but at Phase 1 scale (20–50 policies × 10–30 MiB ≈ <2 GiB total network-wide), savings are trivial. Modern SSDs are | |
| TB-scale; redb handles it fine. Neither model is cost-constrained at Phase 1. | |
| Performance | |
| ┌────────────────────────────┬──────────────────────────┬────────────────────────────────────┐ | |
| │ Path │ Model X │ Model Y │ | |
| ├────────────────────────────┼──────────────────────────┼────────────────────────────────────┤ | |
| │ Eval hot path (cache hit) │ 1 redb read │ 1 redb read + cache hit │ | |
| ├────────────────────────────┼──────────────────────────┼────────────────────────────────────┤ | |
| │ Eval hot path (cache miss) │ local bytes (~1ms) │ network fetch (~10–200ms) │ | |
| ├────────────────────────────┼──────────────────────────┼────────────────────────────────────┤ | |
| │ Cold start / new operator │ snapshot = bulk transfer │ snapshot + N parallel WASM fetches │ | |
| ├────────────────────────────┼──────────────────────────┼────────────────────────────────────┤ | |
| │ First-run wasmtime compile │ 100ms–1s (dominates) │ 100ms–1s (same) │ | |
| └────────────────────────────┴──────────────────────────┴────────────────────────────────────┘ | |
| Hot path is equivalent (both cache). Bootstrap favors X (single bulk transfer). First-run compilation dominates in both. Performance | |
| difference is not decisive. | |
| Reliability (decisive axis) | |
| Model X: | |
| - Single substrate: redb + JMT | |
| - Delta blob = transport only (ephemeral, pruneable after apply) | |
| - Tree = authoritative storage | |
| - Inclusion proof verifies bytes directly | |
| - No dangling-reference risk | |
| - Matches Sub-project A's existing transport/storage split exactly | |
| Model Y: | |
| - Two-tier substrate: JMT (for hash) + EigenDA (for bytes) | |
| - EigenDA becomes a storage medium, not just transport | |
| - New retention invariant: "WASM artifacts live in EigenDA indefinitely" (today delta blobs can be pruned after consumption) | |
| - Two-step verification: prove hash via JMT, then verify fetched bytes | |
| - Dangling-reference risk if artifact blob is pruned | |
| - Cache eviction + EigenDA outage = unreadable policy = cannot evaluate tasks | |
| ★ Insight ───────────────────────────────────── | |
| The core architectural question here is whether EigenDA is transport or storage. Sub-project A treats it as transport — delta blobs | |
| are consumed by bootnode/operators/challengers, applied to the authoritative tree, and become pruneable. Model Y promotes EigenDA to | |
| a storage medium with forever-retention requirements, which is a fundamentally different reliability posture. Once that invariant | |
| exists for one artifact class, it creates pressure to reuse it for others, and a new substrate responsibility has quietly been added | |
| to the system. That's worth avoiding unless the cost/performance case is overwhelming — and at Phase 1 scale, it isn't. | |
| ───────────────────────────────────────────────── | |
| Recommendation | |
| Model X + per-commit size guard (~50 MiB cap per commit, roll excess to next commit). | |
| - Cost: trivial at Phase 1 scale | |
| - Performance: equivalent hot path, slight bootstrap edge | |
| - Reliability: clearly superior — single substrate, no new EigenDA retention invariant, no dangling references | |
| Two caveats worth flagging | |
| 1. Delta blob size limit: EigenDA has a per-blob cap (currently 16 MiB decoded, I think — needs verification before finalizing the 50 | |
| MiB commit cap). If a commit's total leaf bytes exceed the blob limit, we either (a) lower the per-commit cap, or (b) split the | |
| delta across multiple blobs. Worth nailing down before writing the spec. | |
| 2. Future WASM opcode fee mechanism (the TODO at crates/core/src/config/data_provider.rs:28-30): If/when pay-per-byte pricing lands, | |
| Model Y becomes attractive again (bytes live once in EigenDA, fees charged per-byte, no replication cost). But for Phase 1 (no fee | |
| mechanism), Model X wins on reliability and the cost savings from Y are marginal. | |
| --- | |
| Model X + 50 MiB per-commit cap looks solid — want to move on to Q5 (chain placement) next, or dig into one more Q4 sub-branch first | |
| (e.g., the policy-artifact write path: who authors a new Rego deployment, which gateway endpoint, what BLS-quorum validation on | |
| registration)? | |
| ✻ Sautéed for 1m 51s | |
| ※ recap: Designing Sub-project B (on-chain policy storage for Rego/WASM/schema). Locked Option 1A (extend StateCommitRegistry with | |
| namespaces 0x04/0x05/0x06) and Model X with 50 MiB per-commit cap. Next: move to Q5 on chain placement, or dig into the | |
| policy-artifact write path first. (disable recaps in /config) | |
| ❯ let's do cost projection when going with model X | |
| ⏺ Cost projection: Model X, Phase 1 | |
| Bottom line up front: ~$10–50/month network-wide across all dimensions. At Phase 1 scale, the Newton AVS pays approximately the same | |
| as it currently pays for IPFS pinning — the replicated redb storage is not the constraint. | |
| Let me walk through the dimensions with assumptions so you can challenge any that don't match your mental model. | |
| Assumptions (pls correct if off) | |
| ┌──────────────────────┬─────────────────────────────────────────┬───────────────────────────────────────────────────────────────┐ | |
| │ Parameter │ Phase 1 estimate │ Notes │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Policy count │ 20–50 │ Early Phase 1 steady state │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Rego source per │ ~5 KiB avg │ Typical Rego ~100–500 lines │ | |
| │ policy │ │ │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Schema JSON per │ ~3 KiB avg │ Small │ | |
| │ policy │ │ │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ WASM binary per │ ~15 MiB avg (5–30 MiB) │ Based on observed sizes; max_cache_size 100 MiB is hard upper │ | |
| │ policy │ │ bound │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Operators per chain │ ~10 │ Based on current stagef/prod patterns │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Chains supported │ 3 │ Sepolia + Base Sepolia + one L2 (or mainnet eq.) │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Replicas per chain │ 12 │ 10 operators + 1 bootnode + 1 challenger │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Commit cadence │ 120 s │ 720 commits/day/chain │ | |
| ├──────────────────────┼─────────────────────────────────────────┼───────────────────────────────────────────────────────────────┤ | |
| │ Policy deployment │ 1–2/day during launch → ~0/day steady │ Rego updates rare post-launch │ | |
| │ rate │ state │ │ | |
| └──────────────────────┴─────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘ | |
| 1. On-chain gas cost (the biggest variable) | |
| Each 120 s commit is one tx: commitStateRoot(newRoot, sig) → one SSTORE + one event emission + BLS cert verification inline. | |
| ┌───────────────────────────┬────────────────┬─────────────┬─────────┬─────────────┬──────────────┐ | |
| │ Chain │ Gas per commit │ Commits/day │ Gas/day │ Daily cost* │ Monthly cost │ | |
| ├───────────────────────────┼────────────────┼─────────────┼─────────┼─────────────┼──────────────┤ | |
| │ Sepolia (1 gwei) │ ~80k │ 720 │ 57.6M │ $0.14 │ $4.20 │ | |
| ├───────────────────────────┼────────────────┼─────────────┼─────────┼─────────────┼──────────────┤ | |
| │ Base Sepolia (~0.01 gwei) │ ~80k │ 720 │ 57.6M │ $0.01 │ $0.30 │ | |
| ├───────────────────────────┼────────────────┼─────────────┼─────────┼─────────────┼──────────────┤ | |
| │ Mainnet L1 (30 gwei) │ ~80k │ 720 │ 57.6M │ $4.30 │ $130 │ | |
| ├───────────────────────────┼────────────────┼─────────────┼─────────┼─────────────┼──────────────┤ | |
| │ Mainnet L2 (~0.1 gwei) │ ~80k │ 720 │ 57.6M │ $0.05 │ $1.50 │ | |
| └───────────────────────────┴────────────────┴─────────────┴─────────┴─────────────┴──────────────┘ | |
| *ETH price $2,500 assumed. | |
| This is the dominant line item at scale, not storage. If we launch on all three mainnet chains (Ethereum + Base + Arbitrum), commit | |
| gas lands around $140/month; testnet-only deployments are <$5/month. | |
| Mitigations if this becomes a constraint: | |
| - Skip no-op commits (don't commit if delta is empty — only commit when state advances) | |
| - Stretch commit cadence on idle chains (already contemplated in §S.13 safety) | |
| - Batch multiple namespace updates into single tx (already the case with 1A) | |
| 2. Off-chain storage (redb per replica) | |
| Per-policy artifact set: | |
| ┌─────────────────────────────────────────┬──────────────────────────┐ | |
| │ Item │ Size │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ Rego leaf │ ~5 KiB │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ Schema leaf │ ~3 KiB │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ WASM leaf │ ~15 MiB │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ JMT internal nodes per leaf (amortized) │ ~500 B │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ Leaf header (LeafHeaderV1) │ 20 B │ | |
| ├─────────────────────────────────────────┼──────────────────────────┤ | |
| │ Per-policy total │ ~15 MiB (WASM dominates) │ | |
| └─────────────────────────────────────────┴──────────────────────────┘ | |
| At 50 policies: | |
| - Raw artifacts: 750 MiB | |
| - JMT overhead (~20%): 150 MiB | |
| - redb metadata + compaction slack: ~100 MiB | |
| - Per replica: ~1 GiB | |
| Network-wide: | |
| - 3 chains × 12 replicas × 1 GiB = 36 GiB aggregate | |
| - At AWS EBS gp3 (~$0.08/GB/month): $2.90/month across entire network | |
| This is trivial. Even at 10× the policy count (500 policies → 10 GiB/replica × 36 replicas = 360 GiB), monthly cost is ~$29 | |
| network-wide. | |
| 3. EigenDA delta blob cost | |
| Delta blob composition per 120 s commit: | |
| - Steady state (no artifact changes): ~2 KiB (sequence number bump + JMT internal node changes) | |
| - Policy deployment: ~15 MiB (new WASM) + ~10 KiB (Rego + schema + headers) | |
| ┌───────────────────────────────────┬─────────────────────────────┬───────────────┐ | |
| │ Day type │ Per-chain EigenDA bytes/day │ 3-chain total │ | |
| ├───────────────────────────────────┼─────────────────────────────┼───────────────┤ | |
| │ Steady state │ 720 × 2 KiB = 1.4 MiB │ 4.3 MiB/day │ | |
| ├───────────────────────────────────┼─────────────────────────────┼───────────────┤ | |
| │ Launch week (1–2 deployments/day) │ ~30 MiB │ ~90 MiB/day │ | |
| ├───────────────────────────────────┼─────────────────────────────┼───────────────┤ | |
| │ Heavy deployment day │ ~100 MiB │ ~300 MiB/day │ | |
| └───────────────────────────────────┴─────────────────────────────┴───────────────┘ | |
| At EigenDA Phase 1 indicative pricing (~$0.001/MiB, dropping with volume): | |
| - Steady state: ~$0.15/month network-wide | |
| - Heavy deployment month: ~$10/month | |
| Basically noise. | |
| 4. RAM / compute (no delta vs today) | |
| - Moka WASM cache: already max_cache_size: 100 MiB per operator (existing config, no new demand) | |
| - wasmtime compile: 100 ms–1 s per WASM on first use — same as today (no regression) | |
| - JMT proof generation during eval: <1 ms per proof (proof is ~600 B for depth-20 JMT) | |
| - Delta apply per 120 s commit: ~50 ms on operator side (negligible) | |
| - BLS commit sig: ~20 ms per operator per commit | |
| Total per-operator compute tax from state commits: ~70 ms × 720/day = 50 seconds of CPU/day. Trivial. | |
| 5. Bandwidth | |
| Per operator per day: | |
| - Inbound delta fetches: 720 × 2 KiB + occasional 15 MiB bursts ≈ 30–50 MiB/day | |
| - Outbound BLS sig publish: 720 × 96 B = 70 KiB/day | |
| Per-month per-operator bandwidth: ~1.5 GiB → <$0.15/month at AWS egress rates. | |
| Snapshot bootstrap (new operator joining): | |
| - 1 GiB one-shot pull from bootnode | |
| - <$0.10 per event, rare | |
| 6. Aggregate Phase 1 cost projection | |
| ┌────────────────────────┬─────────────────────────────┬──────────────────────────────────────────┐ | |
| │ Category │ Monthly cost (testnet only) │ Monthly cost (mainnet: ETH + Base + Arb) │ | |
| ├────────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤ | |
| │ On-chain commit gas │ $5 │ $135 │ | |
| ├────────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Off-chain redb storage │ $3 │ $3 │ | |
| ├────────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤ | |
| │ EigenDA delta blobs │ $1 │ $1 │ | |
| ├────────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Bandwidth │ $5 (10 ops) │ $5 │ | |
| ├────────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤ | |
| │ Total │ ~$14 │ ~$144 │ | |
| └────────────────────────┴─────────────────────────────┴──────────────────────────────────────────┘ | |
| For comparison, current IPFS pinning cost (assuming Pinata or similar at $20 / 1 TB / month tier) is in the same order of magnitude. | |
| ★ Insight ───────────────────────────────────── | |
| The real "cost" of Model X isn't a dollar figure — it's the per-commit gas tax amortized across idle chains. A chain with no policy | |
| activity still pays ~$4/day on L1 for committing a state root that didn't move. That becomes the dominant line item as chain count | |
| grows. The two paths to mitigate are (a) skip commits when delta is empty, which preserves the 120 s guarantee window but breaks the | |
| deterministic cadence (and is a §S.13 design choice), or (b) stretch cadence on idle chains. Both are adjustments to the commit | |
| cadence policy, not to Model X itself. | |
| ───────────────────────────────────────────────── | |
| Scale sensitivities | |
| What makes the projection move in non-linear ways: | |
| ┌──────────────────────────────────┬──────────────────────────────────────┬──────────────────────────────────────────────────────┐ | |
| │ Dimension │ Impact of 10× growth │ Becomes problematic when │ | |
| ├──────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────┤ | |
| │ Policy count (50 → 500) │ redb: 1 GiB → 10 GiB/replica │ Never at Phase 1 scale; operators on small VMs might │ | |
| │ │ │ notice │ | |
| ├──────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────┤ | |
| │ WASM size (15 MiB → 150 MiB avg) │ Breaks the 100 MiB cache; commit cap │ Need to revisit the max_cache_size config + │ | |
| │ │ blocks │ per-commit cap │ | |
| ├──────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────┤ | |
| │ Chain count (3 → 10) │ Gas cost scales linearly │ Mainnet × 10 chains = ~$450/month just for commits │ | |
| ├──────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────┤ | |
| │ Policy deployment rate (2/day → │ EigenDA: $1 → $25/month │ Still manageable; per-commit cap protects against │ | |
| │ 50/day) │ │ bursts │ | |
| ├──────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────┤ | |
| │ Commit cadence (120 s → 30 s) │ Gas × 4 │ Would drive mainnet cost to ~$540/month; probably │ | |
| │ │ │ not justified │ | |
| └──────────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────────────────────┘ | |
| Biggest unknown | |
| Actual WASM binary size distribution. I used 15 MiB avg as a placeholder. Want to pull actual numbers from current deployments? just | |
| admin-run --bin cli --command wasm-size-report or similar — if we have one, the projection tightens up. If most policies are <1 MiB | |
| WASM, everything above is ~10× too high. | |
| --- | |
| Want me to (a) move to Q5 (chain placement details — confirming artifact replication semantics across chains), (b) dig into the | |
| on-chain gas optimization question (skip-if-empty commits vs. deterministic cadence), or (c) pull actual WASM size numbers from the | |
| current codebase to tighten the projection? | |
| ✻ Cooked for 1m 46s | |
| ❯ the current wasm cache limits are not set. they were just stop gap. we needed wasm cache to prevent fetching the same wasm from ipfs | |
| every single task evaluation. Now that we are storing wasm bytes on newton protocol chain, do we still need cache? | |
| ⏺ Good catch — the cache's original rationale dissolves under Model X. Let me verify the current cache's role and then walk through | |
| what the new architecture should look like. | |
| Searched for 2 patterns, read 2 files (ctrl+o to expand) | |
| ⎿ PreToolUse:Read hook error ⎿ Failed with non-blocking status code: [TokenEfficiency] Hard | |
| rule violation: Re-reading wasm_cache.rs (2x) — file unchanged | |
| since last read. Consider using cached knowledge. | |
| ⎿ Loaded crates/CLAUDE.md | |
| ⎿ Loaded crates/data-provider/src/data_sources/CLAUDE.md | |
| ⏺ Confirmed what the current cache is: a disk-bytes cache at ~/.cache/newton-prover/wasm/* with LRU eviction. Sole purpose: avoid IPFS | |
| re-fetch. Under Model X, that purpose evaporates. | |
| But I found something more interesting: WasmExecutor::execute_wasm_bytes recompiles the WASM on every eval (compile sites at | |
| crates/data-provider/src/wasm/executor.rs:381,508). There's no Module cache today. The bytes cache was saving us from IPFS fetches, | |
| but wasmtime compilation — which dominates cost at 100ms–1s per WASM — happens every single task evaluation. That's a much bigger | |
| missed optimization. | |
| So the answer is nuanced: you're right that the bytes cache goes away, but we should replace it with a Module cache, not nothing. | |
| Three-tier view of caching | |
| ┌──────────┬─────────────────────────────┬───────────────────┬───────────────────────────────────────────┬───────────────────────┐ | |
| │ Layer │ What it holds │ Lookup cost │ Produce cost │ Currently cached? │ | |
| ├──────────┼─────────────────────────────┼───────────────────┼───────────────────────────────────────────┼───────────────────────┤ | |
| │ Bytes │ Raw Vec<u8> WASM │ redb read: ~0.1–1 │ IPFS fetch: 50–500 ms (today) / redb read │ Yes — disk LRU │ | |
| │ │ │ ms │ (Model X) │ │ | |
| ├──────────┼─────────────────────────────┼───────────────────┼───────────────────────────────────────────┼───────────────────────┤ | |
| │ Module │ Compiled wasmtime::Module │ HashMap lookup: │ Module::new(): 100 ms – 1 s │ No (this is the gap) │ | |
| │ │ │ ~1 µs │ │ │ | |
| ├──────────┼─────────────────────────────┼───────────────────┼───────────────────────────────────────────┼───────────────────────┤ | |
| │ Instance │ Instantiated │ — │ Instance::new(): ~1 ms │ No (too stateful to │ | |
| │ │ wasmtime::Instance │ │ │ reuse) │ | |
| └──────────┴─────────────────────────────┴───────────────────┴───────────────────────────────────────────┴───────────────────────┘ | |
| Under Model X: | |
| - The bytes layer's "produce cost" collapses from IPFS fetch to redb read (200–500× cheaper). Disk cache becomes obsolete. | |
| - The Module layer's produce cost is unchanged (wasmtime compile is WASM-shape-dependent, not source-location-dependent). This | |
| becomes the dominant hot-path cost — and it's also where the largest caching win lives. | |
| Recommendation | |
| Remove the WasmCache; add a Module cache. Net is simpler architecture AND better hot-path performance. | |
| ┌───────────────┬──────────────────────────────┬─────────────────────────────────────────────────────────┐ | |
| │ Dimension │ Today (WasmCache) │ Recommended (ModuleCache) │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ What's cached │ Bytes on disk │ Compiled Module in RAM │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ Keyed by │ IPFS CID │ wasmHash (keccak256 of bytes) │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ Hit behavior │ Return bytes → still compile │ Return ready-to-instantiate Module │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ Miss behavior │ IPFS fetch + disk write │ redb read + wasmtime compile + HashMap insert │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ Eviction │ LRU by byte size │ LRU by cached entry count or est. RAM size │ | |
| ├───────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ | |
| │ Invalidation │ Manual │ Automatic (content-addressed: new wasmHash = new entry) │ | |
| └───────────────┴──────────────────────────────┴─────────────────────────────────────────────────────────┘ | |
| Eval hot path becomes: | |
| eval(policy_id, task): | |
| wasm_hash = read_leaf(policy_id).hash # redb, ~0.5 ms | |
| module = module_cache.get_or_insert(wasm_hash, || { | |
| bytes = read_leaf_bytes(wasm_hash) # redb, ~1 ms | |
| verify_jmt_inclusion(bytes, state_root) # ~1 ms | |
| Module::new(&engine, &bytes) # 100 ms – 1 s, one-time | |
| }) | |
| Instance::new(&module, &store) # ~1 ms | |
| run(instance, task) | |
| Hot path after warm-up is ~3 ms end-to-end vs today's ~100+ ms (compile-every-eval). | |
| ★ Insight ───────────────────────────────────── | |
| This is a case where removing one layer exposes a different, more valuable layer that was always there but invisible. The disk-bytes | |
| cache papered over the "fetch" cost and left "compile" as the silent hot-path tax. Model X + Module cache flips the priority: bytes | |
| are now effectively free, so compilation — always the real bottleneck — becomes the thing worth caching. Content-addressed keying (by | |
| wasmHash) means the cache is self-consistent: when a policy upgrades, its new wasmHash is a fresh cache key, old entry ages out via | |
| LRU, no explicit invalidation logic needed. This pattern shows up elsewhere in crypto systems (Merkle-addressed state, | |
| content-addressable storage) and the property is always the same: content identity = cache identity. | |
| ───────────────────────────────────────────────── | |
| Sizing the Module cache | |
| wasmtime compiled Module size is typically 2–5× the WASM byte size (native code + metadata + relocations). | |
| ┌────────────────┬────────────────────┬─────────────────┬───────────────┐ | |
| │ WASM size │ Module size (est.) │ Cache N modules │ RAM footprint │ | |
| ├────────────────┼────────────────────┼─────────────────┼───────────────┤ | |
| │ 5 MiB │ ~15 MiB │ 20 │ 300 MiB │ | |
| ├────────────────┼────────────────────┼─────────────────┼───────────────┤ | |
| │ 15 MiB (avg) │ ~45 MiB │ 20 │ 900 MiB │ | |
| ├────────────────┼────────────────────┼─────────────────┼───────────────┤ | |
| │ 30 MiB (large) │ ~90 MiB │ 20 │ 1.8 GiB │ | |
| └────────────────┴────────────────────┴─────────────────┴───────────────┘ | |
| At 50 policies/chain × 15 MiB avg WASM, the full working set is ~2–3 GiB RAM if we cache everything. Realistically, eval traffic | |
| follows power-law distribution — top-10 policies handle >90% of tasks — so caching ~20 hot Modules per chain covers the working set | |
| at <1 GiB operator RAM. | |
| Proposed config (replaces max_cache_size): | |
| pub struct DataProviderConfig { | |
| // ...existing fields... | |
| /// Maximum number of compiled wasmtime::Module entries to keep in RAM. | |
| /// LRU-evicted. Each entry is typically 2-5x the source WASM byte size. | |
| pub max_module_cache_entries: usize, // default: 32 | |
| /// Optional soft cap on total Module cache RAM footprint (MiB). | |
| /// Acts as a secondary constraint; whichever limit hits first wins. | |
| pub max_module_cache_ram_mib: Option<u64>, // default: Some(1024) | |
| } | |
| Gotchas to flag | |
| 1. wasmtime Engine is already shared across Modules — good. No duplication of JIT state. | |
| 2. Module compilation is single-threaded by default per call. If a cold-miss storm hits (e.g., new operator joining, bootstrap), | |
| first-eval latency spikes. Options: pre-warm cache during bootstrap, or use Engine::precompile_module in a background task right | |
| after state-tree apply. | |
| 3. Serialized Module persistence — wasmtime supports Module::serialize + Module::deserialize_file to persist compiled modules to | |
| disk, skipping re-compile on restart. Probably not worth it for Phase 1 (operators restart rarely; cold start is ≤30 s for 20 | |
| modules). Worth revisiting if operator restart latency becomes a pain point. | |
| 4. wasmtime version-bumps invalidate serialized Modules. If we ever add disk-persisted Module cache, key by (wasmHash, | |
| wasmtime_version) to avoid loading stale compiled code. | |
| 5. Security angle. Since the Module cache key is wasmHash = keccak256(bytes), and the leaf in redb commits to those exact bytes via | |
| JMT inclusion proof, no additional integrity check is needed on cache hits. Content addressing carries the integrity guarantee. This | |
| is cleaner than today's IPFS-fetched bytes where we had to verify CID → bytes mapping separately. | |
| What to delete when implementing | |
| - crates/data-provider/src/wasm_cache.rs entire file | |
| - WasmCache struct and all call sites | |
| - DataProviderConfig::max_cache_size field | |
| - IPFS fetch logic in data_sources/wasm.rs::fetch_data (bytes come from redb now) | |
| - ProjectDirs disk-cache path plumbing in multiple places | |
| Gives us less code + faster hot path + clearer architecture. Clean win. | |
| --- | |
| Agree on the direction? If so, this likely belongs as Q4-sub-branch-c → caching layer in our spec. Next decisions after this: | |
| - (a) Move to Q5 (chain placement) — artifact replication semantics across chains | |
| - (b) Drill down on Module cache sizing / pre-warm strategy (important for operator RAM planning) | |
| - (c) Zoom out to the policy deployment write path — the authoring side of this new storage (which we deferred from Q4) | |
| Which? | |
| ✻ Cogitated for 2m 0s |
All substantive fixes applied. Spec is now 587 lines, internally consistent across struct names and pointer references.
★ Insight ─────────────────────────────────────
- The bigger structural change here was making currentPolicyDataInstance a mapping(address => mapping(bytes32 => bytes32)) instead of
an array. This bakes a key design property into the data model: clients deploy PD instances against template IDs, and the operator's
eval loop walks the policy template's policyDataTemplateRefs to look up instances by slot — never by index. That's how schema evolution
stays clean: rotating one PD template doesn't shift array indices for unrelated PD slots. - The duplicate-registration revert vs. silent no-op decision is a small choice with a big consequence for tooling. Reverting forces
every caller (CLI, admin scripts, Foundry deploy) to handle the case explicitly. A silent no-op would let buggy deploy scripts believe
they registered something fresh when they actually re-keyed against an existing template — a source of "phantom deployment" bugs. - expireAfter living on the instance, not the template, also means challenger verification at G3 needs to read the instance state at
the dispute block. The spec's verifier pseudocode already does both reads — but make sure the implementation plan keeps these two reads
atomic across the same RPC call to avoid race conditions on registry upgrades.
─────────────────────────────────────────────────
Summary of fixes applied (A–D + E–K)
┌─────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Fix │ What changed │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ A │ Added full PolicyRegistry and PolicyDataRegistry struct definitions; currentPolicyDataInstance keyed by (client, │
│ │ policyDataTemplateId) mapping │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ B │ expireAfter: uint64 declared on both PolicyInstance and PolicyDataInstance │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ C │ policyDataTemplateRefs: bytes32[] declared on PolicyTemplate (sorted) │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ D │ Added ### Duplicate registration semantics section: TemplateAlreadyRegistered and InstanceAlreadyDeployed reverts; CLI │
│ │ short-circuits via G3 │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ E │ Hashing parenthetical rewritten to clarify raw-vs-hash storage │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ F │ Unified to leaf_key[0] convention; dropped LeafHeaderV1.namespace references │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ G │ show-secrets metadata enumerated: epoch_id, schema ref, ciphertext size, upload timestamp, uploader │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ H │ Added ArtifactVerifier trait abstraction; documented SHA2-256 (IPFS) vs KZG (EigenDA) paths │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ I │ Citation ranges tightened: secrets.rs:65-89, multichain_routing.rs:253-261, added handler/mod.rs:1665-1708 │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ J │ secretsSchema validation runs in single operator (cryptographically sufficient); defense-in-depth multi-validator │
│ │ deferred to Phase 1b │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ K │ Client owner row updated to mention currentPolicyInstance + currentPolicyDataInstance[templateId] + setSecrets │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ (bonus) │ Narrative pointer language at line 94 updated to match new field names │
└─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Remaining items for discussion (L–O)
These are smaller-stakes items I held back per your instruction. Each is a one-line decision:
L. L4 cache tier hit rate is qualitative, not numeric
Current: L0=95%+, L1=99%+, L2=99.5%+, L3="Tail", L4="Tail emergency".
Options:
- Leave qualitative — operationally honest (we don't have measured L4 rates)
- Specify "fallback only — no SLA" for L3 and L4 — clearer that these are edge cases
- Pick numbers (e.g., L3 99.99%, L4 99.999%) — gives a target to design against
My take: option 2 — the L4 path is genuinely best-effort, and committing to numbers without measurement is the kind of fiction that
breaks in prod review.
M. Single-snapshot section omits PD instance params
Spec says "rego bytes, WASM bytes, and all three schemas read from the same snapshot". Should also list PD instance params (which are
stored in EigenDA blobs but referenced from the registry — also need to read at the snapshot block).
My take: add it — listing all five artifact classes (rego, WASM, 3 schemas, instance params) makes the invariant exhaustive rather than
approximate.
N. Multi-chain example forces an order despite "any-order" Q8 lock
Example registers PD templates first, then policy templates that reference them. Q8 says any-order works thanks to pre-flight.
Options:
- Leave as-is and add a one-line note: "Order is illustrative; pre-flight permits any order"
- Reorder the example to register policy template first to demonstrate the any-order property
- Both — note + reordered
My take: option 1 — the dependency-first order is the default mental model for the reader, and a single sentence is clearer than
reverse-order pedagogy.
O. "Off-chain documentation" hand-wave for deprecation
Spec says: "Off-chain documentation can mark templates as discouraged." No location specified.
Options:
- Drop the line entirely — Phase 1 has no deprecation, full stop
- Point at a real channel (Newton docs site? docs/POLICY_DEPRECATION.md? The runbook?)
- Keep as-is — leaves room for ad-hoc docs
My take: option 1 — the line is currently a hint at a process that doesn't exist yet, which is exactly the kind of phrase that becomes
a future "we said we'd document this somewhere but didn't" issue.
Pick L/M/N/O choices and I'll apply them. Or if you'd rather hold them and review the full spec end-to-end first, I can stage them.
✻ Brewed for 5m 39s
※ recap: Drafted Sub-project B on-chain policy storage spec at docs/superpowers/specs/2026-04-22-onchain-policy-storage-design.md and
just applied substantive fixes A through K from my self-review. Next action is your decision on the four remaining items L through O
before user review.
⏺ ★ Insight ─────────────────────────────────────
Recall search confirmed no prior Sub-B entries — the closest hits are Sub-A "Private Data Storage" memories. Sub-B needs its own memory
file and tagged recall entries; this isn't an update of an existing record. Local memory dir has project_private_data_storage.md for
Sub-A (52.9K, 5 days old), so the natural pattern is a sibling project_onchain_policy_storage.md for Sub-B with a parallel entry in
MEMORY.md.
Session search recovered all Q1-Q10 + G1-G3 context, so nothing was lost to compaction.
─────────────────────────────────────────────────
Preview — what I'll save across all 3 memory systems
Project/Topic: Newton Private Data Storage — Sub-project B (On-chain Policy + Policy Data Migration)
Linear: TBD — no Linear project created yet (Sub-A's Private Storage Layer project covers Sub-A only). Sub-B Linear decomposition is a
follow-up task.
Summary (2-3 sentences)
Completed architectural design and implementation plan for Sub-project B, which migrates policy Rego bytecode + policy-data WASM + 3
schemas from IPFS-only addressing to on-chain commitment + EigenDA bytes (validium pivot). Locked 10 architectural decisions Q1-Q10 plus
3 consistency-dry-run gaps G1-G3 (shared HPKE clarification, ChallengeVerifier re-derivation, CLI pre-flight checks). Final 3,860-line
implementation plan saved with 47 TDD tasks across 6 streams (A:12, B:9, C:9, D:11, E:6, X:4); awaiting execution-mode choice.
Key Decisions
- Validium pivot LOCKED (Q1) — bytes via EigenDA, only commitments on-chain. State-tree namespaces 0x04-0x08 multiplex through
leaf_key[0] prefix in Sub-A's unified JMT. - Two-tier identity formulas (Q2):
- policyTemplateId = keccak256(rego ‖ policyParamsSchema ‖ canonical(policyDataTemplateRefs)) (sorted refs)
- policyDataTemplateId = keccak256(wasm ‖ policyDataParamsSchema ‖ secretsSchema)
- policyInstanceId = keccak256(chainId ‖ clientAddr ‖ policyTemplateId ‖ paramsHash)
- policyDataInstanceId = keccak256(0x01 ‖ chainId ‖ clientAddr ‖ policyDataTemplateId ‖ paramsHash) — domain separator byte - Three schemas (Q3) — policyParamsSchema (0x05), policyDataParamsSchema (0x07), secretsSchema (0x08); JSON Schema Draft 2020-12.
- Template/Instance split (Q4) — Templates immutable + content-addressed (Newton-vetted allowlist); instances mutable + client-owned.
currentPolicyInstance[client] (single) and currentPolicyDataInstance[client][templateId] (per-slot mapping). - Three-layer drift defense (Q5) — L1 deploy-time, L2 eval-time (single-snapshot invariant), L3 challenge-time (re-derivation).
- Five-tier cache (Q6) — L0 enclave RAM, L1 daemon RAM, L2 daemon disk, L3 EigenDA fallback, L4 bootnode fallback.
- CLI surface (Q7) — policy template/instance, policy-data template/instance, set-secrets, describe, with profile config 4-lane
resolver. - Per-client mapping shape (Q8) — single + per-slot mapping (not array).
- Duplicate registration (Q9) — TemplateAlreadyRegistered / InstanceAlreadyDeployed reverts; CLI G3 pre-flight short-circuits.
- Allowlists (Q10) — templateAuthors + instanceDeployers via AllowlistGuard mixin (Safe multisig per chain).
- G1 (gap fix) — Production reuses ONE shared HPKE keypair via OPERATOR__ENCRYPTION__PRIVATE_KEY; Sub-B only adds secretsSchema
namespace 0x08. - G2 (gap fix) — ChallengeVerifier re-reads on-chain policyTemplateRef / policyDataTemplateRef at task's reference block; never trusts
challenger-supplied hash. - G3 (gap fix) — Every CLI deploy command pre-flights all on-chain assertions before broadcasting any tx.
- expireAfter per-instance — confirmed at instance level, > 0 enforced at deployInstance.
- Atomicity — Phase 1 ships any-order (Option B); atomic bundle multicall (Option A) deferred to Phase 2.
Implementation Progress
- Spec: docs/superpowers/specs/2026-04-22-onchain-policy-storage-design.md (586 lines, gitignored, accepted)
- Plan: docs/superpowers/plans/2026-04-22-onchain-policy-storage-plan.md (3,860 lines, 127K, gitignored, complete with self-review)
- 47 TDD tasks across 6 streams:
- A. Foundation (12 tasks) — namespaces 0x04-0x08, artifact-verifier crate (IPFS+EigenDA), policy-id crate (template/instance ID
formulas), schema-validator crate - B. Contracts (9 tasks) — IPolicyRegistry/IPolicyDataRegistry, AllowlistGuard mixin, ChallengeVerifier G2 re-derivation, deployment +
bindings - C. CLI (9 tasks) — profile config, G3 pre-flight, policy template/instance, policy-data template/instance, set-secrets, describe
- D. Service refactors (11 tasks) — chainio writers/readers, LayeredArtifactCache, pre-warm subscriber, WASM compile cache, schema
cache, single-snapshot invariant, gateway secretsSchema validation, challenger G2 binding, data-provider hot path - E. E2E (6 tasks) — feature flag, deploy/eval/challenge/secrets-validation scenarios, 14 quickstart policy migration
- X. Cross-cutting (4 tasks) — metrics, docs, Phase 1.5 IPFS-default-off, Phase 2 IPFS deletion
- A. Foundation (12 tasks) — namespaces 0x04-0x08, artifact-verifier crate (IPFS+EigenDA), policy-id crate (template/instance ID
- Self-review inlined in plan: spec coverage table, placeholder scan, type consistency table, risk callouts (B7 reference-block stashing
deferred, pre-warm subscriber bootstrapping)
Learnings
- Sub-A primitives reuse pattern — Sub-B reuses every Sub-A primitive unchanged (StateCommitRegistry, IStateRootCommittable, JMT state
tree, EigenDaClient, BootnodeServer, NamespaceId enum). Net-new is contracts, CLI, and service refactors only — no new infrastructure
crates. - ArtifactVerifier trait abstracts SHA2-256 (IPFS) vs KZG (EigenDA) — both return keccak256(bytes) as canonical commitment. Decouples
on-chain commitment from off-chain availability layer. - Stream parallelism — Streams A and B are fully independent (no shared symbols), can fan out 2 concurrent subagents in week 1. Stream C
blocks on A; Stream D blocks on A+B; Stream E blocks on D. - expireAfter > 0 enforcement — moved from policy implementation to deployInstance revert (ExpireAfterZero); avoids the historical
AttestationExpired failure mode where expireAfter=0 produced expiration == reference_block DOA attestations. - Three-layer drift defense lesson generalized — applies to any class of artifact where on-chain commitment must bind cryptographically
to off-chain bytes. Sub-A applies it to private data; Sub-B applies it to policy artifacts. Both reuse the same ChallengeVerifier
re-derivation pattern.
Current Issues / Limitations
- B7 reference-block stashing deferred — Phase 1 reads current instance from registries since clients deploy a new instance per config
change. If a client races setCurrent mid-task, mitigation is task-creation-time instanceId snapshot + verifier reads immutable instance.
Phase 1.5 hardening. - Pre-warm subscriber bootstrapping — A new operator joining mid-stream must replay all TemplateRegistered events from genesis via
from_block backfill before live subscription. May take minutes at high event volume. - No Linear decomposition yet — 47 tasks not yet captured as Linear issues. Sub-A's pattern was 50 issues across NEWT-1003..1052 created
post-spec; Sub-B's equivalent is pending. - docs/superpowers/ is gitignored — both spec and plan are local-only. Cannot be committed; teammates need this memory file + recall
entries to track Sub-B context.
Remaining TODOs
- Choose execution mode — Subagent-Driven (recommended for parallel A+B streams) vs Inline. User must pick before plan execution
begins. - Decompose into Linear issues — Sub-A pattern: 47 tasks → ~47 issues with crate-shaped titles, tags, depends-on/blocks, acceptance
criteria. Mirror NEWT-1003..1052 conventions. - Begin Stream A Task A1 — Extend NamespaceId with PolicyRego (0x04), PolicyParamsSchema (0x05), PolicyDataWasm (0x06),
PolicyDataParamsSchema (0x07), SecretsSchema (0x08). - Begin Stream B Task B1 in parallel — IPolicyRegistry interface with BlobReference Solidity type.
- Phase 1.5 hardening — instanceId snapshot at task creation; switch use_ipfs_resolver default to false.
- Phase 2 cleanup — delete IPFS resolver code; consider atomic bundle multicall (Option A).
Aligned Directions (discussed, not implemented)
- Sub-A primitive reuse — Sub-B will share Sub-A's StateCommitRegistry, IStateRootCommittable, JMT, EigenDaClient, BootnodeServer,
NamespaceId infrastructure unchanged. - Three-layer drift defense as protocol pattern — both Sub-A and Sub-B implement it; future on-chain-bytes work should follow the same
L1-deploy / L2-eval / L3-challenge pattern. - Five-tier cache as standard for content-addressed artifacts — LayeredArtifactCache<L0,L1,L2,L3,L4> reusable for any future
committed-bytes class.
Open Questions
- Linear project name for Sub-B — likely Onchain Policy Storage or similar; not yet created.
- Atomic bundle multicall priority — Phase 2 deferral assumes atomic deploy is not blocking for Phase 1 launches. Re-evaluate after
first Phase 1 client onboards. - Quickstart policy migration ordering — 14 quickstart policies; should they migrate one batch or staggered? Sub-A pattern was one batch
via deploy script.
Relevant Links
- Spec (gitignored): docs/superpowers/specs/2026-04-22-onchain-policy-storage-design.md
- Plan (gitignored): docs/superpowers/plans/2026-04-22-onchain-policy-storage-plan.md
- Sub-A spec (PR #570 merged): docs/PRIVATE_DATA_STORAGE.md
- Sub-A plan (gitignored): docs/superpowers/plans/2026-04-20-private-data-migration-plan.md
- Sub-A memory: project_private_data_storage.md (sibling file in this dir)
Anything to add, correct, or remove before I save this across all 3 systems?
✻ Cooked for 3m 5s
25 tasks (17 done, 1 in progress, 7 open)
◼ Transition to writing-plans skill for implementation plan
◻ Propose 2-3 approaches with trade-offs
◻ Present design sections for approval
◻ Apply 10-item delta report (scope C)
◻ Verification pass 2
… +3 pending, 17 completed
Open Questions Q.1–Q.7 in the document need resolution before stream-by-stream execution starts — most notably the JSON Schema
validator library choice (Q.2), quickstart-policy migration ordering (Q.3), and overlap with NEWT-1036 (Q.4). Implementation plan
execution mode (Subagent-Driven vs Inline) is still to be picked. --> Let's actually discuss this and align on this
⏺ Good call — let's actually work through these. Let me re-read the exact Q.1–Q.7 framing from the doc so we discuss what's actually
written, not what I remember.
Searched for 1 pattern, read 1 file (ctrl+o to expand)
⏺ Let's work through them. I'll give my read on each plus the question I'd ask back. We're discussing 8 items total (the deferred Q.0
execution-mode pick + Q.1–Q.7 in the doc).
★ Insight ─────────────────────────────────────
- Q.2, Q.3, and Q.4 are the only blockers. Q.0/Q.1 are workflow choices, Q.5/Q.6 are deferral decisions (default to "no, ship Phase 1"),
Q.7 is documentation hygiene. The three real blockers all need an answer before stream A starts. - Two of the three blockers are people-coordinated, not technical. Q.3 is "how does Newton ops sequence the Safe multisig sessions" and
Q.4 is "when does NEWT-1036's deletion land relative to Sub-B's adds." Those answers come from you/Aaron and the team, not from spec
analysis. Q.2 is the only one that can be resolved by benchmark. - AddressesProviderConsumer immutable-caching means Q.4 is more interesting than it looks. If any contract on chain consumes
IdentityRegistry/ConfidentialDataRegistry IDs from the provider today, it cached those addresses as immutables — so NEWT-1036's deletion
doesn't hot-swap them. Sub-B's new contracts are net-new consumers with no prior immutable caching, so Q.4 is mostly about ordering
registrar updates, not hot-swapping live consumers.
─────────────────────────────────────────────────
Q.0 — Execution mode for the 47-task plan
My recommendation: Subagent-Driven.
47 tasks × ~3-5 steps each won't fit one context window even with checkpoints. The two-stage review (subagent + me) catches drift on
Sub-B's intricate identity formulas (the 0x01 domain-separator byte on policyDataInstanceId, the chainId ‖ clientAddr ordering) — those
are exactly the kinds of details that get fuzzy when context is full. Inline only wins for <10-task plans where ceremony overhead
exceeds the benefit.
Question back: any reason to prefer Inline? (e.g., wanting to watch each task land in real-time, or tighter feedback on early tasks
before parallelizing?)
Q.1 — Linear project name
My recommendation: ship the provisional "Onchain Policy Storage."
This is naming bikeshed territory — both "On-chain" and "Onchain" are correct. Newton ops decides; doesn't block code. Mirroring Sub-A's
"Private Data Storage" (no hyphen in URLs) is reasonable.
Question back: want me to also pre-register the 47 Linear issues mirroring NEWT-1003..1052 conventions, or do that later when the plan
is closer to execution?
Q.2 — JSON Schema validator library (boon vs jsonschema)
My recommendation: benchmark first; default to jsonschema if inconclusive.
boon is lighter (~150-200KB smaller binary) but newer. jsonschema is mature with broader Draft 2020-12 coverage. The 14 quickstart
schemas are the natural test bed. Concrete proposal:
┌────────────┬───────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────┐
│ Library │ Strengths │ Risks │
├────────────┼───────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ boon │ Pure Rust, smaller deps, │ Newer (less battle-tested), some Draft 2020-12 features may be │
│ │ no_std-friendlier │ incomplete │
├────────────┼───────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ jsonschema │ Mature, broad coverage, active │ ~400KB heavier, may pull in async runtime indirectly │
│ │ maintenance │ │
└────────────┴───────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────┘
Stream A early: write the 14 quickstart schemas, run both validators, pick whichever passes 100% of the suite with the cleanest API. If
both pass, take boon for the smaller footprint.
Question back: is binary footprint (operator daemon size) actually a constraint we care about right now, or is correctness/maturity the
only axis?
Q.3 — Quickstart migration ordering
My recommendation: staged 3 + 5 + 6, NOT one batch.
This is the most consequential ops decision in the list. Reasoning:
- Single batch (14 in one Safe session): faster, but a single hash-mismatch or schema-validation surprise rolls back 14 deploys. The L1
drift-detection layer (registerTemplate rejects keccak256 mismatch) is new code — not yet field-tested at scale. - Staged (3 + 5 + 6): first batch should be the simplest policies (single-domain or read-only). If something surfaces, we catch it on 3
instead of 14. Safe multisig overhead per batch is ~10-20 min — three sessions across a week is cheap insurance.
The G2 ChallengeVerifier re-derivation hook is also new; staging gives Newton ops a real-world signal that on-chain policyTemplateRef
values match what challengers expect before all 14 are exposed.
Question back: which of the 14 quickstart policies are the simplest / most tested today? That informs the first batch.
Q.4 — NEWT-1036 overlap
My recommendation: NEWT-1036 lands first, Sub-B's adds land after.
NEWT-1036 (deletes IdentityRegistry + ConfidentialDataRegistry) is independent of Sub-B's net-new PolicyRegistry + PolicyDataRegistry.
They could ship in either order, but landing NEWT-1036 first keeps the provider directory smaller and avoids a transitional window where
the provider has both the deprecated entries and Sub-B's adds.
The immutable-caching concern only matters if there are existing consumers of the deleted IDs — which there will be (the registries'
direct consumers). NEWT-1036 has to handle redeployment of those consumers regardless of Sub-B. Sub-B's consumers are net-new, so they
only see the post-deletion provider state.
Question back: is NEWT-1036 ready to ship, or is its blocker the same as Sub-B's (waiting for Sub-A primitives to soak)? If they're both
gated on the same thing, sequencing them serially is the safer call.
Q.5 — Atomic-bundle priority (Phase 2 multicall)
My recommendation: defer to Phase 2 backlog as documented.
The "deploy-then-update-pointer" race window is exactly one block on most chains and only matters if an observer queries
currentPolicyInstance[client] between the two txs. For Phase 1 — where clients control their own deploys — this is acceptable. Pull it
forward only if real-world ops surface frequent friction.
Question back: any known external dependency that observes currentPolicyInstance[client] mid-deploy and would break under the race
window?
Q.6 — policyParams on-chain mirror
My recommendation: ship Phase 1 with the on-chain mirror.
The 14 quickstart policyParams payloads are small (likely <1KB each, mostly string addresses + numeric thresholds). On-chain storage
cost is bounded. Moving to EigenDA-backed policyParamsRef adds challenger-side fetch + KZG verification complexity — unjustified at MVP
scale.
Threshold to reconsider: any single policyParams exceeding 8KB, or storage cost on Base/L2s becoming visible in operator economics.
Question back: is there a known upcoming policy with policyParams >2KB? If so, that flips the calculus.
Q.7 — Secrets schema ↔ HPKE envelope versioning
My recommendation: keep independent (as Phase 1 ships); document the orthogonality.
Coupling them would force an envelope rotation on every schema update — costly entanglement for no security benefit. The envelope_format
byte (0x03 namespace) describes how bytes are encrypted; the secrets schema (0x08 namespace) describes what fields are in the
plaintext. These are genuinely orthogonal — a _v2 schema with new fields can live under the same SECRETS_ENVELOPE_V1 envelope as long as
the new fields fit in the existing encrypted blob.
The doc already describes the failure mode (WASM eval fails deterministically when a required field is absent) — that's enough.
Question back: none from me on this one. It's a "default to keeping things uncoupled" decision unless someone has a concrete pairing
reason.
Summary table
┌─────┬─────────────────────────────────────┬────────────────┬──────────────────────────────────────┐
│ Q │ My pick │ Confidence │ Needs you to confirm? │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.0 │ Subagent-Driven │ High │ Yes │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.1 │ "Onchain Policy Storage" │ Low (bikeshed) │ Yes │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.2 │ Benchmark first; default jsonschema │ Medium │ No (handled in stream A) │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.3 │ Staged 3 + 5 + 6 │ High │ Yes │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.4 │ NEWT-1036 first, Sub-B after │ Medium │ Yes (depends on NEWT-1036 readiness) │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.5 │ Defer to Phase 2 │ High │ Yes (sanity check) │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.6 │ On-chain mirror │ High │ Yes (sanity check) │
├─────┼─────────────────────────────────────┼────────────────┼──────────────────────────────────────┤
│ Q.7 │ Keep independent │ High │ Yes (sanity check) │
└─────┴─────────────────────────────────────┴────────────────┴──────────────────────────────────────┘
What's your read on each? Which of my recommendations are wrong, which want different framing, and which are you fine with as-is?
✻ Crunched for 2m 7s
※ recap: Goal: lock Sub-B (on-chain policy storage) protocol decisions before implementing the 47-task plan. Current task: align on Q.0
execution mode plus Q.1–Q.7 from the doc. Next action: react to my recommendations in the summary table.
Spec is intact at 510 lines / 31.4K. Now doing the self-review pass for placeholders, consistency, scope, and ambiguity.
★ Insight ─────────────────────────────────────
"is the design right". The four review axes (placeholders, consistency, scope, ambiguity) map to four classes of post-acceptance bugs:
TODO drift, contradiction-driven rework, scope creep, and "we thought we agreed" reinterpretation.
policyTemplateId is reproducible from local files alone — a property the CLI exploits via compute-id for offline scripting and the
contract exploits via deterministic dedup. Catching where the spec under-specifies this dual semantics is a high-value review target.
file), readers lose the ability to verify the spec's claims against the code. Each citation in this spec is a load-bearing claim about
the existing system.
─────────────────────────────────────────────────
Self-review findings
MUST FIX — substantive gaps
A. currentInstanceIds shape for PolicyDataRegistry is undefined (lines 178, 407)
Line 178 says "clients can have multiple policy-data instances per policy" and uses currentInstanceIds (plural). The Solidity
pseudocode at lines 167-176 shows only PolicyRegistry with clientCurrent: mapping(address => bytes32). The PD parallel layout is not
shown. Implementer ambiguity: is it mapping(address => mapping(bytes32 policyId => bytes32[])), or mapping(address => bytes32[]), or
keyed by something else?
B. expireAfter field is referenced but never declared (line 393)
▎ "registry rejects expireAfter == 0 at deployInstance time"
But PolicyInstance struct shape isn't shown anywhere. The implementer cannot tell from the spec whether expireAfter lives on the
instance, on the template, or in params.
C. policyDataTemplateRefs storage is implied but not declared (line 82, 90)
The formula canonical(policyDataTemplateRefs) is part of policyTemplateId. For policy describe to walk the DAG (line 228-231), the
template must store these references on-chain explicitly — but PolicyTemplate struct shape isn't shown.
D. Contract behavior on duplicate registerTemplate is unspecified (lines 190 vs 271)
short-circuits to a no-op"
But the contract behavior is undefined — does registerTemplate revert on duplicate, no-op, or update? Critical for retry semantics if a
CLI pre-flight is bypassed (admin tool, direct contract interaction).
SHOULD FIX — clarity
E. Hashing parenthetical is confusing (line 89)
▎ "arguments are encoded with abi.encodePacked for hashing only (commitments stored as raw bytes, never abi.encode'd at registry-write
▎ time)"
The "stored as raw bytes" phrase is ambiguous — is it talking about the registry storing bytes32 results, or storing the input
rego_bytes? Re-read suggests it means the on-chain commitment is the 32-byte hash, not abi-encoded. Worth re-stating without the
parenthetical.
F. Namespace tag terminology drifts (lines 109 vs 441)
Sub-A uses leaf_key[0] as the 1-byte namespace prefix. Pick one term and use it consistently — LeafHeaderV1.namespace field vs
leaf_key[0] prefix are not the same shape.
G. show-secrets metadata semantics undefined (line 226)
▎ "show-secrets # show metadata only"
What metadata? Upload timestamp? Schema reference? Envelope size? Encryption epoch? The implementer needs the field list.
H. CID-vs-KZG integrity abstraction unclear (line 396-398)
The spec says "Sub-B replaces IPFS CID integrity verification with EigenDA blob KZG-commitment verification" — but SHA-256 multihash
and KZG pairing are categorically different checks. Is the operator hot path wrapped behind a trait that abstracts both? Or are they
two distinct code paths gated by the feature flag?
I. Off-by-one in code citation (line 504)
▎ "integration-tests/src/multichain_routing.rs:253-260"
Actual range was 253-261. Same kind of issue may be in the secrets.rs:1-120 citation (line 503) — that's a whole-file range, not a
function range.
J. secretsSchema validation locality (lines 449, 462)
Line 449 says validation "runs inside any operator the gateway delegates to" — does "any" mean exactly one, or all decrypters? Given
the shared key model, one is sufficient cryptographically; redundancy is a defense-in-depth choice that should be explicit.
K. Client owner role description misses plural case (line 407)
▎ "Point their client's currentInstanceId to a deployed instance they own"
For PD, it's currentInstanceIds. The role description should accommodate both registries.
NICE TO HAVE — style
L. L4 hit rate qualitative outlier (line 331)
L0–L3 have target percentages or "Tail"; L4 has "Tail emergency" with no target. Would be more uniform to say "fallback only — no SLA"
or pick a numeric target.
M. Single-snapshot mentions only 5 artifacts (line 389)
"rego bytes, WASM bytes, and all three schemas" — but PD instance params are also read at eval time. Implicit but worth listing for
completeness.
N. Multi-chain example forces an order despite "any-order" Q8 lock (lines 297-303)
The example registers PD templates first, then policy templates referencing them. Q8.3 says any-order works thanks to pre-flight. A
one-line note ("Order is illustrative; pre-flight permits any order") would prevent reader confusion.
O. "Off-chain documentation" hand-wave (line 419-420)
▎ "Off-chain documentation can mark templates as discouraged"
What channel? Newton docs site? GitHub README? A specific runbook? Either point to the channel or remove the line.
ALL CLEAR
Recommendation
Substantive fixes A–D should land before user review. The SHOULD-FIX items E–K are individually small but together drive ~15 lines of
edits. NICE-TO-HAVE items L–O are polish.
Want me to apply A–D + E–K now, or stage them for your review pass first?
✻ Worked for 2m 27s
※ recap: Drafting the Sub-project B on-chain policy storage spec; finished writing it to
docs/superpowers/specs/2026-04-22-onchain-policy-storage-design.md and ran a self-review surfacing four substantive gaps. Next action
is your call on whether to apply the fixes now or after your review.
❯ yes for A: currentInstanceIds refers to the "deployed policy data instances" not templates right? conceptually, policy ref (template)
to policy Data refs (template) are mapped, and each poilcy client (address) has policy instance id (policy registry) + policy data
instance ids (policy data registry)? confirm my understanding and correct me if im wrong. B. i need to confirm, but think expire After
is set at the policy/policy data "instance" level? So first, yes, let's apply fixes for A-D + E-K, and for the rest, let's review /
discuss as well
⏺ Confirming your mental model first, then applying fixes.
Your mental model — correct, with one refinement
▎ "currentInstanceIds refers to the 'deployed policy data instances' not templates"
✅ Correct. currentInstanceIds (plural) tracks policy-data instances the client has deployed, not templates.
▎ "conceptually, policy ref (template) to policy Data refs (template) are mapped, and each policy client (address) has policy instance
▎ id (policy registry) + policy data instance ids (policy data registry)"
✅ Correct. The relationship is:
policyTemplate ──declares──→ policyDataTemplateRefs = [pd1, pd2, pd3] (1-to-N at template level)
policyInstance ──references──→ policyTemplate (client deploys 1 policy instance)
policyDataInstance ──references──→ policyDataTemplate (client deploys 1 PDI per referenced PD template)
One refinement on the keying: the per-client PD instance map should be keyed by (client, policyDataTemplateId), not a flat bytes32[]
array. So:
// PolicyRegistry — one policy instance per client
mapping(address client => bytes32 instanceId) public currentPolicyInstance;
// PolicyDataRegistry — one PD instance per (client, PD template) slot
mapping(address client => mapping(bytes32 policyDataTemplateId => bytes32 instanceId))
public currentPolicyDataInstance;
Why mapping not array: the operator's eval loop walks the policy template's policyDataTemplateRefs and asks the registry "for client C,
which PD instance fills slot pd1?". A keyed mapping gives O(1) per slot and decouples client-side instance ordering from policy-author
template-ref ordering. An array would force ordering coupling and make swapping a single PD instance harder.
▎ "expireAfter is set at the policy/policy data 'instance' level"
✅ Correct. It's per-instance because two clients deploying against the same template can choose different validity windows. This
matches today's NewtonPolicy.PolicyConfig.expireAfter.
★ Insight ─────────────────────────────────────
pd1 instance independently of pd2, which lets clients hot-swap a single data domain (e.g., "use a fresh sanctions list") without
redeploying their policy. Array-keyed would force re-pointing the entire policy any time one PD changes.
the verifier re-derivation flow at G2: the verifier must read both the template (for hash commitments) AND the instance (for
expireAfter) from the registry at the task's reference block.
verify that every PD template the policy requires has a corresponding instance pointed by the client. A missing slot is a deploy-time
invariant violation that pre-flight (G3) must catch — otherwise eval-time will see a zero-instance lookup and need to fail loudly.
─────────────────────────────────────────────────
Now applying fixes A–D + E–K.