Schema fixes by daywalker90 · Pull Request #9537 · ElementsProject/lightning

daywalker90 · 2026-09-21T14:42:33Z

I made one commit per schema file and included the corresponding code path for ease of review.

Whenever developer-only fields came up i chose to omit/remove them since we do that almost anywhere else with a few exceptions, but we can have a more general discussion about that.

Andezion

in 8f0d5bc, why does enumerate_enum() use str(typename) while enumvar2number() uses str(typename.name)? Given TypeName.__str__() strips dashes, was this intentional, or an oversight?

Maybe we can use str(typename.name) on line 117 to match enumvar2number, or just add one small helper that both places call, so the two can't drift apart again??

daywalker90 · 2026-09-23T14:42:46Z

in 8f0d5bc, why does enumerate_enum() use str(typename) while enumvar2number() uses str(typename.name)? Given TypeName.__str__() strips dashes, was this intentional, or an oversight?

Maybe we can use str(typename.name) on line 117 to match enumvar2number, or just add one small helper that both places call, so the two can't drift apart again??

Fixed

Andezion · 2026-09-30T15:29:50Z

      },
      "recurrence_start": {
-        "type": "number",
+        "type": "u32",


same wire type problem as in waitsendpay.json - this changes CancelrecurringinvoiceRequest.recurrence_start (field 4) from double to uint32 with the same field number?

Andezion · 2026-09-30T15:30:59Z

      },
      "completed_at": {
-        "type": "number",
+        "type": "u32",


nGoline · 2026-11-12T11:55:13Z

Hi @daywalker90 this is marked for v26.12, but it's still in draft. Let us know when it's ready for review, otherwise we'll push this to v27.03.

The meta-schema only accepted schemas carrying an `rpc` or `notification` key, and `schemaResponse` did not allow response-level `if`/`then`/`else`. As a result every doc/schemas/hook/*.json failed the pre-commit `check-jsonschema` hook, and the four hook responses which encode conditional requiredness (htlc_accepted.json, invoice_payment.json, openchannel.json and openchannel2.json) were rejected outright. Add the `hook` key with a matching top-level `oneOf` branch, and allow `if`/`then`/`else` in `schemaResponse`. Also list string_map, json_object_or_array, json_scalar and proof_field, which msggen and the pyln-testing schema validator already understand (contrib/msggen/msggen/model.py:425-461, contrib/pyln-testing/pyln/testing/fixtures.py:482-511). Changelog-None

lightningd emits this field with json_add_s32 (the difference between the HTLC's cltv_expiry and the current blockheight can be negative): lightningd/peer_htlcs.c:1200 s32 expiry = hin->cltv_expiry, blockheight = ... lightningd/peer_htlcs.c:1252 json_add_s32(s, "cltv_expiry_relative", expiry - blockheight); Type it as `integer` rather than `u32`. Changelog-None

The hook payload is built from the u16 fields in the openchannel payload: lightningd/opening_control.c:663 u16 to_self_delay; lightningd/opening_control.c:664 u16 max_accepted_htlcs; lightningd/opening_control.c:691-692 json_add_num(stream, "to_self_delay", payload->to_self_delay); json_add_num(stream, "max_accepted_htlcs", payload->max_accepted_htlcs); The sibling openchannel2.json hook already types the same BOLT fields as u16. Changelog-None

When the funder plugin contributes funds and the peer requested a lease, it appends the lease rates to its response: plugins/funder.c:464-465 if (current_policy->rates && !amount_sat_is_zero(info->requested_lease)) json_add_lease_rates(response, current_policy->rates); dualopend scans those fields back from the hook result: lightningd/dual_open_control.c:789-802 json_scan(payload, buffer, toks, "{lease_fee_base_msat:%" ",lease_fee_basis:%" ",channel_fee_max_base_msat:%" ",channel_fee_max_proportional_thousandths:%" ",funding_weight:%}", ...) but the response schema had additionalProperties:false and did not list them, so the documented payload was incomplete. lease_fee_basis, funding_weight and channel_fee_max_proportional_thousandths are u16 in the wire struct (wire/peer_wire.csv:215-217). Changelog-None

`example_hooks` is not part of the rpc meta-schema and is not consumed by tools/fromschema.py (its FOOTER_KEY_SEQUENCE only knows `examples` and `example_notifications`), so it was never rendered. The same file already carries an `examples` block. Changelog-None

lightningd copies the plugin's `return.result` token verbatim without imposing a type: lightningd/jsonrpc.c:965-970 custom_return = json_get_member(buffer, tok, "result"); if (custom_return) { p->custom_result = json_strdup(p, buffer, custom_return); return true; } The test plugin returns an array (tests/plugins/rpc_command_1.py:19), which the old `type: object` rejected. Type it `json_object_or_array` so msggen keeps the field as a JsonObjectOrArray instead of dropping it (as it does for `untyped`). Changelog-None

forward_event shares json_add_forwarding_fields() with listforwards, which emits created_index, updated_index, in_htlc_id and preimage: lightningd/notification.c:395 json_add_forwarding_fields(stream, cur, &in->payment_hash, in->preimage); lightningd/forwards.c:95-143 json_add_u64(response, "created_index", cur->created_index); json_add_u64(response, "updated_index", cur->updated_index); json_add_u64(response, "in_htlc_id", cur->htlc_id_in); json_add_preimage(response, "preimage", preimage); Add the missing fields to the notification schema. created_index and updated_index are only emitted when non-zero, so they stay optional. (out_htlc_id is deliberately omitted: forward_event_notification_serialize() sets cur->htlc_id_out = NULL at lightningd/notification.c:385.) Changelog-None

Notifications are generated under a renamed protobuf message (notification_typename_overrides), but their field numbers were looked up in grpc-field-map under the unrenamed name. A type shared by an RPC response and a notification (ConnectAddress) therefore got one number space: a field only the notification has was reserved on the response. Key grpc-field-map by the protobuf message name, the same name the contract uses, and move the connect notification entries to match. Generated output is unchanged. Changelog-None

When a peer is dialed through a proxy (or with DNS disabled) the address is ADDR_INTERNAL_FORPROXY and json_add_address_internal serializes it as type "unresolved" with name/port: common/json_stream.c:582-587 case ADDR_INTERNAL_FORPROXY: json_add_string(response, "type", "unresolved"); json_add_string(response, "name", addr->u.unresolved.name); json_add_num(response, "port", addr->u.unresolved.port); That address type is reachable from connect (see common/wireaddr.c:774-792 and lightningd/connect_control.c:187-208). Add the enum value and the required name/port fields. Changelog-None

json_add_payment_fields() emits created_at and completed_at with json_add_u32, matching struct wallet_payment: lightningd/pay.c:158 json_add_u32(response, "created_at", t->timestamp); lightningd/pay.c:160 json_add_u32(response, "completed_at", *t->completed_at); wallet/wallet.h:363-364 u32 timestamp; u32 *completed_at; Changelog-None

sendpay_failure serializes the payment via json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (struct wallet_payment keeps them as u32): lightningd/pay.c:158,160 wallet/wallet.h:363-364 Changelog-None

The command does not exist at tag v25.09 (the plugin only gained json_cancelrecurringinvoice in the v25.12 cycle), so the added annotation is v25.12, not v25.09. This also required updating the recorded annotation in .msggen.json. recurrence_counter and recurrence_start are parsed with param_number (unsigned int) and are tu32 on the wire: plugins/fetchinvoice.c:867,873-874 u32 *recurrence_counter, *recurrence_start; p_req("recurrence_counter", param_number, &recurrence_counter), p_opt("recurrence_start", param_number, &recurrence_start), wire/bolt12_wire.csv:93,95 ...invreq_recurrence_counter,counter,tu32, ...invreq_recurrence_start,period_offset,tu32, They were typed u64 and number; both are u32. Changelog-None

The connect RPC serializes its address with the same json_add_address_internal() as the connect notification: lightningd/connect_control.c:63 json_add_address_internal(response, "address", addr); which emits type "unresolved" with name/port for ADDR_INTERNAL_FORPROXY (common/json_stream.c:582-587). Add the enum value, the websocket value already present on the notification, and the required name/port branch. Changelog-None

delpay serializes each payment with json_add_payment_fields(), which emits both timestamps with json_add_u32 (struct wallet_payment keeps them as u32): lightningd/pay.c:158,160 json_add_u32(response, "created_at", t->timestamp); json_add_u32(response, "completed_at", *t->completed_at); lightningd/pay.c:2406 json_add_payment_fields(response, payments[i]); wallet/wallet.h:363-364 u32 timestamp; u32 *completed_at; Changelog-None

The title was copy-pasted from fundchannel; dev-splice moves funds into or out of a channel (plugins/spender/splice.c). Changelog-None

lease_fee_basis, funding_weight and channel_fee_max_proportional_thousandths are u16 on the wire (wire/peer_wire.csv:215-217) and the plugin range-checks the request values into u16 (plugins/funder.c:1315-1317). Match the openchannel2 hook response and listnodes. Changelog-None

getemergencyrecoverdata returns an object with filedata, can_create_penalty and backed_up_channel_ids (plugins/chanbackup.c:1034-1043), but the generator locked a bare string: tests/autogenerate-rpc-examples.py:1509 response='emergencyrecoverdata' + ('01' * 827) Replace it with a correctly-shaped canned response and regenerate the stored examples so check-doc-examples passes. Changelog-None

log_to_json never emits a SKIPPED entry and num_skipped does not exist anywhere in the tree: lightningd/log.c:1185-1190 level == LOG_BROKEN ? "BROKEN" : ... : "IO_IN" Drop the SKIPPED enum value and its num_skipped branch. The previous msggen commit preserves SKIPPED's proto3 wire value as deprecated, so existing gRPC clients are unaffected. Also drop the num_skipped rewrite from the example generator: num_skipped is not emitted by lightningd at all. Changelog-None

injectpaymentonion reports success via injectonion_succeed(), which emits both timestamps with json_add_u32 from the u32 wallet_payment struct members: lightningd/pay.c:1769-1770 json_add_u32(response, "created_at", payment->timestamp); json_add_u32(response, "completed_at", *payment->completed_at); wallet/wallet.h:363-364 u32 timestamp; u32 *completed_at; Changelog-None

closed_channel.c serializes it with json_add_txid, not a generic hash: lightningd/closed_channel.c:76 json_add_txid(response, "last_commitment_txid", &txid); Changelog-None

The listconfigs canonicalisation was a copy-paste of the recoverchannel one and injected a bogus `scb` request parameter into listconfigs#1; point it at recoverchannel. Regenerate the stored examples so check-doc-examples passes. Changelog-None

The plugin registers the method as `listcurrencyrates`: plugins/currencyrate-plugin/src/main.rs:77 RpcMethodBuilder::new("listcurrencyrates", listcurrencyrates) but the schema declared `"rpc": "currencyrates"`, so the generated synopsis was `**currencyrates** *currency*`. Changelog-None

For outputs from our own unilateral close listfunds emits csv_lock and spendable_at: wallet/walletrpc.c:406-411 if (utxo->close_info && utxo->close_info->csv > 1) { json_add_num(response, "csv_lock", utxo->close_info->csv); if (utxo->blockheight) json_add_u32(response, "spendable_at", *utxo->blockheight + utxo->close_info->csv); } Changelog-None

The request parses id with param_node_id: wallet/walletrpc.c:1365 p_opt("id", param_node_id, &specific_id), and the matching response field peer_id is already typed pubkey (doc/schemas/listnetworkevents.json response.properties.peer_id). Changelog-None

The lease_rates wire fields are u16 (wire/peer_wire.csv:215-217) and json_add_lease_rates serializes them from the u16 struct members (common/json_stream.c:671-683). Match the openchannel2 hook response types. Changelog-None

The xpay listpays helper emits created_at with json_add_u32 (from u32 pm->timestamp): plugins/xpay/listpays.c:168 json_add_u32(ret, "created_at", pm->timestamp); The legacy listpays path serializes both timestamps through json_add_payment_fields(), which uses json_add_u32 from the u32 wallet_payment struct members: lightningd/pay.c:158,160 json_add_u32(response, "created_at", t->timestamp); json_add_u32(response, "completed_at", *t->completed_at); wallet/wallet.h:363-364 u32 timestamp; u32 *completed_at; Changelog-None

Drop the dead next_fee_step field: json_add_channel only emits initial_feerate, last_feerate and next_feerate (lightningd/peer_control.c:1061-1082). Type to_self_delay and max_accepted_htlcs as u16: these are the same BOLT channel_config fields as the openchannel/openchannel2 hooks (common/channel_config.h:69). Changelog-None

log_to_json only ever emits BROKEN, UNUSUAL, INFO, DEBUG, TRACE, IO_IN and IO_OUT; num_skipped appears nowhere in the tree: lightningd/log.c:1185-1190 level == LOG_BROKEN ? "BROKEN" : ... : "IO_IN" so the SKIPPED enum value and its num_skipped branch were dead. Also add "broken" to the level parameter enum, which is accepted by log_level_parse() (common/status_levels.c:30-49). Changelog-None

listsendpays serializes each payment with json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (lightningd/pay.c:158,160; wallet/wallet.h:363-364). Changelog-None

sendpay serializes the payment with json_add_payment_fields(), which emits both timestamps with json_add_u32 (struct wallet_payment keeps them as u32): lightningd/pay.c:158,160 json_add_u32(response, "created_at", t->timestamp); json_add_u32(response, "completed_at", *t->completed_at); wallet/wallet.h:363-364 Changelog-None

sendonion calls send_payment_core(), which reports via json_sendpay_in_progress() -> json_add_payment_fields(). That function emits created_at with json_add_u32 from the u32 wallet_payment struct member: lightningd/pay.c:158 json_add_u32(response, "created_at", t->timestamp); lightningd/pay.c:301 json_add_payment_fields(response, payment); lightningd/pay.c:1384 return send_payment_core(ld, cmd, ...); wallet/wallet.h:363 u32 timestamp; Changelog-None

The `tx` field is only emitted on the dev-only wetrun path: plugins/spender/splice.c:123-131 if (splice_cmd->wetrun) { ... json_add_hex(response, "tx", tx_bytes, tal_bytelen(tx_bytes)); but splicein always disables wetrun: plugins/spender/splice.c:2243 splice_cmd->wetrun = false; The normal success path returns only psbt and txid (plugins/spender/splice.c:1762-1769). Drop the dead field. Changelog-None

As with splicein, `tx` is only emitted on the dev-only wetrun path, which spliceout disables: plugins/spender/splice.c:123-131 if (splice_cmd->wetrun) { ... json_add_hex(response, "tx", ...); } plugins/spender/splice.c:2293 splice_cmd->wetrun = false; The success path returns only psbt and txid. Changelog-None

waitsendpay reports the payment via json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (lightningd/pay.c:158,160; wallet/wallet.h:363-364). completed_at was typed number. Changelog-None

checkPluginJSON only validated notifications. Load the hook schemas too, so hook_in and hook_out payloads recorded in plugin-io are checked. Changelog-None

funder.c sends the lease fields in its rbf_channel response when the peer requested a lease, but the schema rejected them because additionalProperties is false. Add the same fields openchannel2 already documents, with the same types and descriptions. Changelog-None

daywalker90 · 2026-11-13T12:48:18Z

The proto files should be stable after #9678 while cln-rpc gets the fixes in the next version.

daywalker90 requested a review from cdecker as a code owner September 21, 2026 14:42

daywalker90 marked this pull request as draft September 21, 2026 15:50

daywalker90 force-pushed the docs-schema-fixes-scan branch 4 times, most recently from 2b542c3 to b691223 Compare September 22, 2026 16:34

daywalker90 marked this pull request as ready for review September 23, 2026 07:20

daywalker90 requested a review from Andezion September 23, 2026 07:20

Andezion requested changes Sep 23, 2026

View reviewed changes

daywalker90 force-pushed the docs-schema-fixes-scan branch from b691223 to df831c5 Compare September 23, 2026 14:41

daywalker90 requested a review from Andezion September 23, 2026 14:43

Andezion reviewed Sep 30, 2026

View reviewed changes

daywalker90 marked this pull request as draft October 7, 2026 15:19

daywalker90 added this to the v26.12 milestone Oct 26, 2026

daywalker90 force-pushed the docs-schema-fixes-scan branch 2 times, most recently from 19b4437 to 7323d05 Compare October 30, 2026 13:11

madelinevibes added the Status::Ready for Review The work has been completed and is now awaiting evaluation or approval. label Nov 11, 2026

daywalker90 mentioned this pull request Nov 12, 2026

grpc: freeze the gRPC wire format and enforce required fields up to v26.06 and more #9678

Merged

daywalker90 added 10 commits November 13, 2026 13:25

daywalker90 added 26 commits November 13, 2026 13:33

docs: sendpay_failure timestamps are u32

bdaea47

sendpay_failure serializes the payment via json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (struct wallet_payment keeps them as u32): lightningd/pay.c:158,160 wallet/wallet.h:363-364 Changelog-None

docs: fix dev-splice title

7b334b7

The title was copy-pasted from fundchannel; dev-splice moves funds into or out of a channel (plugins/spender/splice.c). Changelog-None

docs: listclosedchannels last_commitment_txid is a txid

75af3e5

closed_channel.c serializes it with json_add_txid, not a generic hash: lightningd/closed_channel.c:76 json_add_txid(response, "last_commitment_txid", &txid); Changelog-None

docs: listnodes option_will_fund lease fields are u16

40e765d

The lease_rates wire fields are u16 (wire/peer_wire.csv:215-217) and json_add_lease_rates serializes them from the u16 struct members (common/json_stream.c:671-683). Match the openchannel2 hook response types. Changelog-None

docs: listsendpays timestamps are u32

97ded8d

listsendpays serializes each payment with json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (lightningd/pay.c:158,160; wallet/wallet.h:363-364). Changelog-None

docs: waitsendpay timestamps are u32

9700756

waitsendpay reports the payment via json_add_payment_fields(), which emits created_at and completed_at with json_add_u32 (lightningd/pay.c:158,160; wallet/wallet.h:363-364). completed_at was typed number. Changelog-None

pyln-testing: validate plugin-io hook payloads against doc/schemas/hook

1f46f02

checkPluginJSON only validated notifications. Load the hook schemas too, so hook_in and hook_out payloads recorded in plugin-io are checked. Changelog-None

daywalker90 force-pushed the docs-schema-fixes-scan branch from 7323d05 to 5b334d1 Compare November 13, 2026 12:46

daywalker90 marked this pull request as ready for review November 13, 2026 12:47

daywalker90 requested a review from Andezion November 13, 2026 12:48

Navigation Menu

Repository navigation

Schema fixes - #9537

Schema fixes#9537
daywalker90 wants to merge 36 commits into
ElementsProject:masterfrom
daywalker90:docs-schema-fixes-scan

daywalker90 commented Sep 21, 2026

Uh oh!

Andezion left a comment

Uh oh!

daywalker90 commented Sep 23, 2026

Uh oh!

Andezion Sep 30, 2026

Uh oh!

Andezion Sep 30, 2026

Uh oh!

nGoline commented Nov 12, 2026

Uh oh!

daywalker90 commented Nov 13, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

daywalker90 commented Sep 21, 2026

Uh oh!

Andezion left a comment

Choose a reason for hiding this comment

Uh oh!

daywalker90 commented Sep 23, 2026

Uh oh!

Andezion Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

Andezion Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

nGoline commented Nov 12, 2026

Uh oh!

daywalker90 commented Nov 13, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants