[ripplebiz]

This is an attempt to carve out a brand new Direct Mode by utilising the as-yet-unused TRANSPORT_DIRECT route type (0x03). ie. Packets with Direct + transport code data.

transport_code[0] is used as such:

• bits 0..7: sequence number
• bits 8..13: path cursor (0..63)
• bits 14..15: DM_MODE_SEQ_* (sub-mode: one of NORMAL, WIDE, WIDE_MODIFIED, LINK_ID)

transport_code[1] is used as such:

• bits 0..13: session-id
• bits 14..15: crypto-mode (CRYPTO_LEGACY, CRYPTO_NEW_DIRECT, RESERVED1, RESERVED2)

Some of the things this new encoding/scheme is attempting is:

• path is not consumed as it propagates along path. Dest node can see what the Full Path was. Instead, a simple 'cursor'/index is incremented at each hop, and path remains untouched (except in 'wide' mode).
• 8-bit sequence numbers incremented by sender. Dest nodes should be able to do Link Quality calcs, ie. determine how many packets were lost.
• A new 'wide' / 'fuzzy' sub-mode (DM_MODE_SEQ_WIDE).
• Possibility for a number of different encryption methods. (a new placeholder one has a minimal impl for the CRYPTO_NEW_DIRECT slot, which attempts to use the seq-no/session-id fields as IV data)

NOTE: The 'wide' mode is inspired from @redxdev and this PR: #2648

Wide sub-mode is fairly simple, where (at each hop) the designated repeater with matching hash forwards, but also all direct neighbors of that repeater also forward. This could be a much more resilient direct mode. New clients could have a strategy, when sending, to use DM_MODE_SEQ_NORMAL (say first 2 attempts), then DM_MODE_SEQ_WIDE (for next attempt), then revert to FLOOD.

Also, if a neighbor repeater forwards it scribbles into the path its own hash, then edits the sub-mode to DM_MODE_SEQ_WIDE_MODIFIED. Dest node can then see if an alternate path was actually taken and use that info.

The _LINK_ID sub-mode is still effectively a reserved sub-mode, and hasn't been properly fleshed-out.

There are still a number of aspects that are yet to be done, like acting on the Link Quality calcs. (just placeholder updateLinkQualityStats() methods ATM)

[mikecarper]

Is this a use case for the blockchain path tracking that you were looking at? Allow one to mine the exact path taken using the discovery db of adverts. First 16 char is all you'd need to add to the blockchain

[ripplebiz]

Is this a use case for the blockchain path tracking that you were looking at? Allow one to mine the exact path taken using the discovery db of adverts. First 16 char is all you'd need to add to the blockchain

No, this is unrelated.

[VoxMesh]

I think there's a case where WIDE can break an otherwise working route.

Suppose the path starts with A, B, .... Both A and B can hear the sender, and B has A in its neighbor table.

B can forward on behalf of A, replacing A with its own hash and advancing the cursor. This leaves [B, B, ...], with B still expected to handle the next hop.

However, B has already marked the packet as seen. When A forwards its copy, B will discard it as a duplicate, since calculatePacketHash() doesn't include the cursor. The packet can get stuck even though the original route was valid.

Instead of always advancing the cursor by one, could we check whether the forwarding repeater's own hash appears again later in the remaining path? If it does, we could advance the cursor past that occurrence and continue forwarding, effectively skipping the intermediate hops.

We'd also need to account for the skipped hops in the recorded path so it doesn't misrepresent the actual route.

Am I missing anything in the duplicate handling here?

[ripplebiz]

I think there's a case where WIDE can break an otherwise working route.

Suppose the path starts with A, B, .... Both A and B can hear the sender, and B has A in its neighbor table.

B can forward on behalf of A, replacing A with its own hash and advancing the cursor. This leaves [B, B, ...], with B still expected to handle the next hop.

However, B has already marked the packet as seen. When A forwards its copy, B will discard it as a duplicate, since calculatePacketHash() doesn't include the cursor. The packet can get stuck even though the original route was valid.

Instead of always advancing the cursor by one, could we check whether the forwarding repeater's own hash appears again later in the remaining path? If it does, we could advance the cursor past that occurrence and continue forwarding, effectively skipping the intermediate hops.

We'd also need to account for the skipped hops in the recorded path so it doesn't misrepresent the actual route.

Am I missing anything in the duplicate handling here?

Yes, I think you're correct. That scenario would fail. Seems like removing duplicates of 'B' (in that example) before forwarding could be the fix, although it's a little sketchy if it's a 1-byte path hash. It could be conservative and only remove duplicate path hash (ahead of cursor) only if it's the next hop or the one after that(?) rather than scanning whole path.

[VoxMesh]

Yes, I think you're correct. That scenario would fail. Seems like removing duplicates of 'B' (in that example) before forwarding could be the fix, although it's a little sketchy if it's a 1-byte path hash. It could be conservative and only remove duplicate path hash (ahead of cursor) only if it's the next hop or the one after that(?) rather than scanning whole path.

Agreed, with 1-byte path hashes any fix seems like a trade-off.

I also considered keeping the repeated B in the path. When B substitutes for A in [A, B, C], giving [B, B, C], it could first forward with the cursor at C, then send a backup copy pointing at the second B.

That would cover both possibilities: the second B is the same repeater, or a different one with a matching prefix.

The downside is that duplicate handling would need to distinguish these forwarding stages, including on downstream repeaters. Plus the extra airtime.

Might be more complexity than this case warrants.

[redxdev]

I've been out of the loop lately so I'm glad to see the path cursor and wide mode are still being considered!

An additional thing that's worth considering is having isHashNeighbor return false if the neighbor has too low of an SNR (I think I required >= 0, but it being configurable probably makes sense) - I did that in my original proposal to try to limit wide paths to neighbors that would likely still be heard by the next hop. But maybe it's not necessary, haven't fully thought through it.

[ripplebiz]

I've been out of the loop lately so I'm glad to see the path cursor and wide mode are still being considered!

An additional thing that's worth considering is having isHashNeighbor return false if the neighbor has too low of an SNR (I think I required >= 0, but it being configurable probably makes sense) - I did that in my original proposal to try to limit wide paths to neighbors that would likely still be heard by the next hop. But maybe it's not necessary, haven't fully thought through it.

Good idea. It would be easy to add that as tuning config item, eg. "set wide.snr.thresh 1.0", but have it default to something like -8.0

[VoxMesh]

Both sendRequest() overloads shadow the outer pkt with auto pkt. The null check and encodeDatagram() use the inner pointer, but stdSend() gets the uninitialized outer one.

Looks like auto just needs to be dropped in both places.

[ripplebiz]

Thanks for spotting that. Both fixed now

[VoxMesh]

This early return also drops responses to public REGIONS, OWNER and BASIC requests. Those handlers don't create an ACL entry, so a valid request from an unknown client gets no reply.

Could we preserve the reply path for these requests without requiring an ACL entry?

[ripplebiz]

OK, should be handled now

[VoxMesh]

Adding crypto_type here leaves SensorMesh::onPeerPathRecv() with the old 8-argument signature in both the header and implementation.

The override now fails to compile. Looks like the sensor override needs updating too. simple_sensor isn't covered by the current PR build matrix.

[ripplebiz]

now fixed

[VoxMesh]

Found another WIDE case, this time at the destination repeater.

Suppose S sends a REQ to repeater R via [A]. R can also hear S directly and has A in its neighbor table.

In WIDE, R can forward for A, call markSeen() and return without processing the request locally. When A later forwards its copy with the cursor at the end, R drops it at wasSeen() before checking the destination or MAC.

NORMAL delivers the same request. A local test using the actual Mesh::onRecvPacket() reproduces this.

Seems like forwarding and local delivery need to be handled separately, or an addressed-to-self packet should be checked before WIDE substitution.

[sethoscope]

Did you consider having repeaters scan the entire path for neighbors, rather than only looking where the cursor is? This would create a lot more flexibility and robustness.

Currently, if the path is specified as
A→B→C→D, it could travel
A→W→C→D if W is a neighbor of B.

But here are things it can't do:
A→X→C→D if X neighbors C but not B
A→Y→B→C→D
A→Z→D if Z neighbors C, but not B

It's a bit more work to scan through the whole path, and to shift bytes around when making the path longer or shorter, and there's always a chance a path is already at maximum length. But I think the current algorithm as described is much too limited and we'd be missing a lot of opportunities.

Scanning the whole path and allowing repeaters to insert themselves or skip ahead is much more likely to find a viable route than by only making substitutions. That would make it a narrow flood tunnel, described by the path, with a 1-hop radius, where the message could travel any viable route within that described area. It's a different flood-path compromise, using more airtime but having a higher likelihood of success.

[marcelverdult]

Some notes on the CRYPTO_NEW_V1 slot. I know the current code is a placeholder, so these are meant for whoever fills in the real scheme later.

1. The IV is short. Direct packets derive the key from the 14-bit session-id plus the 8-bit seq-no, so within one session the IV repeats every 256 packets. TRANSPORT_FLOOD packets only use the session-id, so all flood packets of a session get the same derived key. Client and repeater also share one session-id and each count their own seq-no, so packet n in each direction gets the same IV.

   With the current ECB on top of HKDF this only leaks equal blocks, same as legacy. With a stream mode (the comment in Utils.h mentions one), a repeated IV means keystream reuse, which would be worse than today. Any CTR or GCM style mode in this slot needs a nonce that never repeats per key: at least a direction bit in the HKDF info, plus a wider counter or the packet timestamp.

2. The derived key is used for both AES and HMAC (first 16 bytes as AES key, all 32 bytes as HMAC key). HKDF can give separate encryption and MAC keys cheaply, for example with two info labels or a 48-byte output.

3. The comment in Utils.h says 8-byte tag, but the code still truncates to CIPHER_MAC_SIZE (2 bytes), so a forged packet gets through with probability 1/65536 per try. NEW_V1 is a new format anyway, so this is the cheap moment to lengthen the tag. A stream mode also drops the block padding (up to 15 bytes, about 8 on average), which would pay for most of it.

4. The MAC only covers the ciphertext. Session-id and seq-no are bound through the key derivation, but the header and the dest/src hashes are not. Adding them to the MAC input or the HKDF info would cover that.

5. Small and independent of this PR: MACThenDecrypt compares the MAC with memcmp. A constant-time compare costs nothing.

Longer discussion of the crypto side is in #259.

[ripplebiz]

Thanks for the feedback. As it happens, I just had discussions with core team about another issue, and that including the payload-type and dest-hash and src-hash in the derive_key() would give some nice forgery protections. So, that covers some of the points you also made. Changes are in the latest commit [8c5bdc1]

I'll look into the point 2. about HKDF and deriving separate encrypt and MAC keys.

For point 3. yes, I was wondering about increasing the the MAC size, as this is greenfield territory and ideal time to change. 3 bytes would increase the probability by huge factor, but incur smallest increase in overhead I think.

Otherwise, regarding ECB alternatives, as far as I can tell we don't have much choice as there isn't enough IV/nonce uniqueness to work with (?)

[marcelverdult]

Thanks, I saw 5ab9f83 too. The 48-byte HKDF output with separate cipher and HMAC keys covers point 2, and putting type, dest and src into the info covers point 4. It also separates the two directions of a session, except when both hashes share the same byte.

On the ECB question: since 8c5bdc1 the key is derived per packet, so I think a nonce is no longer the problem. If (type, dest, src, session-id, seq-no) never repeats, every derived key encrypts exactly one packet, and AES-CTR with a fixed zero counter block is safe. No IV has to go on the air.

1. The tuple repeats when the 8-bit seq-no wraps. The sender could pick a new random session-id at that point. As far as I can see the receiver only derives the key from the fields in the packet (Mesh.cpp around line 212), so the sender can do this on its own. With 14 random bits, a collision becomes likely after roughly 128 rotations, so around 32k packets per contact pair. That is my estimate, not measured.

   CTR drops the block padding (up to 15 bytes, about 8 on average), which would more than pay for the 3-byte MAC, or even 4 bytes.

2. TRANSPORT_FLOOD has no seq-no, so all flood packets of a session get the same derived key. ECB only leaks equal blocks there, but a stream mode would reuse the keystream. Flood should stay on ECB, or get its own counter, and it may be worth a comment in the code so nobody switches it later.

3. The seq-no gives no replay protection yet, since updateLinkQualityStats() is still a TODO. A captured CRYPTO_NEW_V1 packet derives the same key when it is sent again, so it decrypts and verifies. I haven't checked how much the timestamp inside the plaintext already catches. A small per-contact window on (session-id, seq-no) would close this once the link quality code is there.

Minor: MACThenDecrypt still uses memcmp for the tag. And with 1-byte hashes the receiver tries every contact with a matching hash, so the forgery chance per try grows with the number of candidates.