hsmtool: recover HTLC outputs after channel state loss by sangbida · Pull Request #9730 · ElementsProject/lightning

sangbida · 2026-11-12T00:01:32Z

The issue

This came out of #9200.
When we have lost channel state, the peer may close with a commitment transaction we have never seen. We can recover our main output, but we cannot recognise the HTLC outputs because we no longer have the peer's per-commitment point or the HTLC information.
This adds:
lightning-hsmtool recoverhtlc <bundle.json> <hsm_secret>
The bundle contains the commitment transaction, channel information and the HTLCs we want to recover. The tool rebuilds the HTLC scripts and checks that they actually match outputs in the commitment transaction before signing anything.
It supports:

HTLCs we offered, which we reclaim after the CLTV expiry;
HTLCs the peer offered, which we claim with the preimage;
anchor and non-anchor channels;
identical MPP outputs, which are swept together.
If one HTLC was trimmed or does not match, we skip it and continue with the others.

This prints one public commitment point without exposing the shaseed or any commitment secrets, so a cooperative peer can safely give us what we need.

This is not general recovery from only an hsm_secret. We still need the channel information, HTLC details, commitment transaction and help from the peer. It is intended for cases like #9200, where that information can still be recovered.

The channel-seed derivation is moved into common/hsm_secret so hsmd and the recovery tools all use the same implementation.

Limitations

This is not recovery from hsm_secret alone. The operator still needs the peer ID, channel DBID, peer HTLC basepoint, commitment transaction, HTLC details, and the peer's per-commitment point.

Andezion · 2026-11-12T13:52:13Z

 }

+/* Unlike atoll, refuses anything which is not entirely a number. */
+static bool parse_u64(const char *arg, u64 *val)


parse_u64 accepts leading whitespace as i see, so myabe we could add !cisdigit(arg[0]) (from ccan/str)

Andezion · 2026-11-12T13:54:18Z


+@unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "sqlite3-specific DB rollback")
+@pytest.mark.parametrize("anchors", [False, True])
+def test_dataloss_protection_hsmtool_recoverhtlc(node_factory, bitcoind, tmp_path, anchors):


maybe we should add cases like a trimmed (dust) HTLC, an encrypted hsm_secret, bad --point-only arguments and small test of parse_bundle?

Andezion · 2026-11-12T13:55:08Z

+#include <common/amount.h>
+
+struct node_id;
+struct ripemd160;


are we using this?

Andezion · 2026-11-12T13:58:17Z

+	size_t num_recovered = 0;
+
+	/* Transaction and signature helpers require libwally's tal allocator. */
+	common_setup(argv0);


common_setup() replaces the secp256k1 context created in main(), so maybe call common_setup(argv[0]) once at the top of main() for every command. The other commands would also get a real tmpctx. Or call it in the recoverhtlc branch of main() and remove the argv0 parameter

Andezion · 2026-11-12T13:59:36Z

+		    bundle->peer_id, bundle->channel_dbid,
+		    &bundle->commitment_point, &bundle->remote_htlc_basepoint,
+		    hsm_secret_path, keys))
+		errx(ERROR_USAGE, "Could not derive HTLC recovery keys");


maybe we can move ERROR_USAGE define to tools/lightning-hsmtool.h and use ERROR_KEYDERIV there

Andezion · 2026-11-12T14:01:02Z

+    `bitcoin-cli getrawtransaction`.
+  * *feerate\_perkw* is optional, in satoshi per 1000 weight: 250 is
+    1 sat/vB.  The default is 2500.
+  * Each of *htlcs* with *direction* `out` is an HTLC which we offered: it


should we add there a short note on where to get payment_hash and cltv_expiry - the peer s listhtlcs/listpeerchannels, logs, or decoding the commitment tx outputs with the peer?

nGoline

This is a useful tool for the #9200 situation. I went through the derivation and scripts against BOLT #3 and it looks right to me.

Two things needed before this can go green:

There is no Changelog-Added: line in any commit or the PR body, so "Pre-build checks" fails at "Check changelog" and none of the Linux jobs have run yet. Please add one to the recoverhtlc commit.
Inline on test_connection.

Two smaller points, not blocking inline.

nGoline · 2026-11-12T14:30:45Z

    assert (closetxid, "confirmed") in set([(o['txid'], o['status']) for o in l2.rpc.listfunds()['outputs']])


+@unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "sqlite3-specific DB rollback")


The full-integration matrix has a liquid-regtest job, and recoverhtlc refuses Elements networks, so this test will fail there. Please add @unittest.skipIf(TEST_NETWORK != 'regtest', "hsmtool recoverhtlc does not support Elements").

nGoline · 2026-11-12T14:32:18Z

+		return "its value does not cover the fee";
+	if (amount_sat_less(amounts.output, chainparams->dust_limit))
+		return "its value after the fee is below the dust limit";
+	if (amount_sat_greater(amounts.fee, amount_sat_div(amounts.input, 4)))


Not blocking: a fee above a quarter of the HTLC only warns, and the signed tx is still printed. A mistyped feerate_perkw (an extra zero, or sat/vB vs sat/kw) gives a valid tx that hands most of the HTLC to miners. I would either refuse above a cap unless explicitly overridden, or drop "a wrong value cannot produce a wrong spend" from the man page, since it is not true for feerate_perkw or the destination.

nGoline · 2026-11-12T14:32:56Z

 #define PASSPHRASE_HASH_LEN 32
 #define HSM_SECRET_PLAIN_SIZE 32

+void derive_channel_seed(struct secret *channel_seed,


Not blocking: The move dropped the /*~ comments from hsmd explaining "peer seed"/"per-peer seed" and the native-endian dbid. Since this is now the only copy, it would be good to keep them there.

Allow callers to derive one public commitment point without printing the shaseed or any per-commitment secrets. Co-authored-by: Cursor <[email protected]>

Move the historical channel-seed calculation into common/hsm_secret and reuse it from hsmd, checkchannels and lightning-hsmtool. Co-authored-by: Cursor <[email protected]>

The recovery tool is long, so it is best kept in its own file. Expose load_hsm_secret in a header so it can be shared with the recovery tool. Co-authored-by: Cursor <[email protected]>

Derive the channel basepoints and commitment-specific keys needed to recover HTLC outputs without contacting lightningd or hsmd. Co-authored-by: Cursor <[email protected]>

Build offered and received HTLC scripts for the peer’s commitment, including anchor channels and CLTV expiry. Co-authored-by: Cursor <[email protected]>

Match reconstructed P2WSH scripts against commitment outputs and retain every matching index so identical MPP parts can be recovered together. Co-authored-by: Cursor <[email protected]>

Build both anchor variants of the requested HTLC script and use the commitment transaction to determine which one contains the outputs. Co-authored-by: Cursor <[email protected]>

Build a single transaction spending every matched HTLC output and sign each input with the appropriate timeout or preimage witness. Co-authored-by: Cursor <[email protected]>

Sum the matched HTLC outputs and calculate the fee and destination amount from the estimated signed transaction weight. Co-authored-by: Cursor <[email protected]>

Read versioned recovery data from JSON so the command line only needs operational inputs such as the destination and fee rate. Co-authored-by: Cursor <[email protected]>

Read a recovery bundle, validate its external inputs, and print signed transactions for each matching HTLC claim. Changelog-Added: tools: lightning-hsmtool can recover HTLC outputs after channel state loss. Co-authored-by: Cursor <[email protected]>

Show the recovered inputs, output amount, fee, transaction ID, and verification and broadcast instructions. Co-authored-by: Cursor <[email protected]>

Reproduce stale channel state with pending HTLCs and prove recoverhtlc signs broadcastable anchor and non-anchor spends. Cover identical MPP outputs, same-hash opposite directions, strict JSON validation, and skipped unmatched claims. Co-authored-by: Cursor <[email protected]>

Document the recovery bundle, where each input comes from, the point-only peer workflow, transaction timing, and operator verification steps. Co-authored-by: Cursor <[email protected]>

sangbida added this to the v26.12 milestone Nov 12, 2026

sangbida mentioned this pull request Nov 12, 2026

UTXO missing from wallet after a restart and an unwanted local force-close #9200

Open

sangbida requested review from Andezion, jaonoctus and nGoline and removed request for Andezion November 12, 2026 05:34

Andezion reviewed Nov 12, 2026

View reviewed changes

nGoline requested changes Nov 12, 2026

View reviewed changes

sangbida force-pushed the hsmtool/recover-htlc branch from d287dad to 659ccdf Compare November 14, 2026 00:53

sangbida and others added 14 commits November 14, 2026 11:28

hsmtool: add point-only dumpcommitments mode

c23fc00

Allow callers to derive one public commitment point without printing the shaseed or any per-commitment secrets. Co-authored-by: Cursor <[email protected]>

hsm: share channel seed derivation

ce29532

Move the historical channel-seed calculation into common/hsm_secret and reuse it from hsmd, checkchannels and lightning-hsmtool. Co-authored-by: Cursor <[email protected]>

hsmtool: make hsm secret loading reusable

1336f2d

The recovery tool is long, so it is best kept in its own file. Expose load_hsm_secret in a header so it can be shared with the recovery tool. Co-authored-by: Cursor <[email protected]>

hsmtool: derive keys for HTLC recovery

8aacd67

Derive the channel basepoints and commitment-specific keys needed to recover HTLC outputs without contacting lightningd or hsmd. Co-authored-by: Cursor <[email protected]>

hsmtool: reconstruct HTLC witness scripts

73201d5

Build offered and received HTLC scripts for the peer’s commitment, including anchor channels and CLTV expiry. Co-authored-by: Cursor <[email protected]>

hsmtool: match HTLC commitment outputs

adffead

Match reconstructed P2WSH scripts against commitment outputs and retain every matching index so identical MPP parts can be recovered together. Co-authored-by: Cursor <[email protected]>

hsmtool: find recoverable HTLC outputs

e84b859

Build both anchor variants of the requested HTLC script and use the commitment transaction to determine which one contains the outputs. Co-authored-by: Cursor <[email protected]>

hsmtool: sign HTLC recovery transactions

4ee3ed5

Build a single transaction spending every matched HTLC output and sign each input with the appropriate timeout or preimage witness. Co-authored-by: Cursor <[email protected]>

hsmtool: calculate HTLC recovery fees

a2b462d

Sum the matched HTLC outputs and calculate the fee and destination amount from the estimated signed transaction weight. Co-authored-by: Cursor <[email protected]>

hsmtool: parse HTLC recovery bundles

a390c6f

Read versioned recovery data from JSON so the command line only needs operational inputs such as the destination and fee rate. Co-authored-by: Cursor <[email protected]>

hsmtool: add recoverhtlc command

008c0e8

Read a recovery bundle, validate its external inputs, and print signed transactions for each matching HTLC claim. Changelog-Added: tools: lightning-hsmtool can recover HTLC outputs after channel state loss. Co-authored-by: Cursor <[email protected]>

hsmtool: explain HTLC recovery transactions

64e635f

Show the recovered inputs, output amount, fee, transaction ID, and verification and broadcast instructions. Co-authored-by: Cursor <[email protected]>

docs: explain hsmtool HTLC recovery

d36e8de

Document the recovery bundle, where each input comes from, the point-only peer workflow, transaction timing, and operator verification steps. Co-authored-by: Cursor <[email protected]>

sangbida force-pushed the hsmtool/recover-htlc branch from 659ccdf to d36e8de Compare November 14, 2026 01:36

Navigation Menu

Repository navigation

hsmtool: recover HTLC outputs after channel state loss - #9730

hsmtool: recover HTLC outputs after channel state loss#9730
sangbida wants to merge 14 commits into
ElementsProject:masterfrom
sangbida:hsmtool/recover-htlc

sangbida commented Nov 12, 2026 •

edited

Loading

Uh oh!

Andezion Nov 12, 2026

Uh oh!

Andezion Nov 12, 2026

Uh oh!

Andezion Nov 12, 2026

Uh oh!

Andezion Nov 12, 2026

Uh oh!

Andezion Nov 12, 2026

Uh oh!

Andezion Nov 12, 2026

Uh oh!

nGoline left a comment

Uh oh!

nGoline Nov 12, 2026

Uh oh!

nGoline Nov 12, 2026

Uh oh!

nGoline Nov 12, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert (closetxid, "confirmed") in set([(o['txid'], o['status']) for o in l2.rpc.listfunds()['outputs']])


		@unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "sqlite3-specific DB rollback")

Conversation

sangbida commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The issue

Limitations

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nGoline left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sangbida commented Nov 12, 2026 •

edited

Loading