Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
65ff71d
feat(xtcp): cap a namespace's dump bytes and records, and the namespa…
randomizedcoder Nov 13, 2026
c6469f6
docs: the per-namespace caps
randomizedcoder Nov 13, 2026
630b117
fix(xtcp): at -maxNetns, track the lowest-pid live namespaces
randomizedcoder Nov 13, 2026
0c06570
fix(xtcp): -maxNetns never evicts; waiting namespaces queue first come
randomizedcoder Nov 13, 2026
c1ce90d
feat(config): flags for the socket-count gate, reconcile limits and o…
randomizedcoder Nov 13, 2026
8bf5670
feat(io_uring): EnqueueReadAt, SetIOWQMaxWorkers, RequestID on Result
randomizedcoder Nov 13, 2026
c1bb184
style(io_uring): gofmt
randomizedcoder Nov 13, 2026
d3b87cb
feat(xtcp): socket-count gate before each poll (-maxRecordsPerPoll)
randomizedcoder Nov 13, 2026
df16161
fix(xtcp): sockstat fd lifetime and broken-ring buffers; shard on the…
randomizedcoder Nov 13, 2026
2814132
feat(xtcp): bound reconcile in time and count; per-owner netns quota …
randomizedcoder Nov 13, 2026
bebecfb
fix(xtcp): reconcile limits review: capped scans keep tracked, owner …
randomizedcoder Nov 13, 2026
911f910
fix(xtcp): anchor owners to runtime cgroups, bound the waiting queue,…
randomizedcoder Nov 13, 2026
6a98640
fix(xtcp): timed-out scans delete nothing; over-quota owners get no q…
randomizedcoder Nov 13, 2026
9af699d
perf(xtcp): keep FP-5 fields off the pools' cache lines; lint fixes
randomizedcoder Nov 13, 2026
109ff10
fix(xtcp): rotate shards per gated poll; count origNs in the fd check
randomizedcoder Nov 13, 2026
1b02706
test(xtcp): make the thread-churn test independent of host CPU count
randomizedcoder Nov 14, 2026
7076e27
test(nix): socket-scale microVM for the socket-count gate and floods
randomizedcoder Nov 14, 2026
82a598b
fix(xtcp): shard on max(sport, dport) over the namespace's port range
randomizedcoder Nov 14, 2026
3b1288a
fix(sockfill): turn off TCP keepalive on held connections
randomizedcoder Nov 14, 2026
3741773
Merge origin/main into sec/fp5-dump-netns-caps
randomizedcoder Nov 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

### New

- **Per-namespace caps.** `-maxDumpBytes` (default 256 MiB) and `-maxRecordsPerNetns` (default 262144) bound one namespace's dump per poll: packets past the byte cap are drained undecoded, and records past the record cap are dropped before decode. `-maxNetns` (default 4096) bounds the namespaces tracked. At the cap, tracked namespaces keep their slots, and new ones wait for a freed slot, first come, first served. Alert on `xtcp_netns_dropped_total`. Drops are counted in `xtcp_dump_dropped_total{reason}` and `xtcp_netns_dropped_total`, with one log line per truncated dump or short reconcile. `0` turns a cap off; the matching env vars are `MAX_DUMP_BYTES`, `MAX_RECORDS_PER_NETNS` and `MAX_NETNS`. The caps are startup-only: `ConfigService.Set` keeps the running values. No healthy host should reach the defaults; a host that does now loses that namespace's excess rows, where it used to collect them all ([polling and batching](docs/polling-and-batching.md#per-namespace-caps)).
- **Socket-count gate, reconcile limits, per-owner quota.** Before each poll xtcp2 reads each namespace's sockstat (an fd opened in the namespace; `pread`, or io_uring batches with `-ioUring`) and, when the total is over `-maxRecordsPerPoll` (default 1,048,576), degrades only the namespaces over their fair share: first TIME_WAIT/SYN_RECV are not asked for, then one of K port shards is dumped per poll. `-maxNetnsPerOwner` (default 64) caps namespaces tracked per owner (the container runtime's cgroup), `-maxNetnsAddsPerReconcile` (256) the adds per reconcile, and `-reconcileTimeout` (1s) the namespace scan; a timed-out scan deletes nothing. New metrics `xtcp_poll_sockets`, `xtcp_poll_degraded_total{level}` and `xtcp_reconcile_incomplete_total{reason}`; `xtcp_netns_dropped_total` gains `reason`. `0` turns each off; all are startup-only. Each tracked namespace now holds four fds (was two: sockstat and `ip_local_port_range` are new), and xtcp2 warns at startup when `RLIMIT_NOFILE` can't cover them ([polling and batching](docs/polling-and-batching.md#socket-count-gate), [network namespaces](docs/network-namespaces.md#configuration)). The analysis behind an upstream all-namespace dump is in [docs/kernel](docs/kernel/inet-diag-all-netns.md).
- **Vulnerability check.** `checks.govulncheck` runs govulncheck against a pinned snapshot of the Go vulnerability database, with no tags and with every flavor tag, and fails on a reachable finding not in its accepted list. One is accepted: GO-2026-6859 in `net/http`, fixed in go1.27.4, until the toolchain moves. `nix run .#govulncheck-fresh` checks against the live database and runs `go mod verify`; `nix build .#sbom-<binary>` writes a CycloneDX SBOM for any binary package.

## 0.2.1 (2026-11)
Expand Down
9 changes: 9 additions & 0 deletions cmd/xtcp2/testdata/defineflags-golden.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ hostname hostname stamped on records; defaults to os.Hostname(). Set this in a
ioUring false Opt in to io_uring for netlink reads, one multishot recv per socket (Linux 6.1+). Destination writes stay syscalls
ioUringCqeBatch 128 io_uring max CQEs reaped per PeekBatchCQE call (1-4096)
ioUringRecvBatch 64 io_uring receive buffers per socket, rounded up to a power of two (1-4096). Each holds one packet; with all in use, receiving pauses until they are recycled.
ioUringSockstatBatchSize 64 io_uring: sockstat reads per submit (1-1024). Ignored unless -ioUring. Falls back to IO_URING_SOCKSTAT_BATCH_SIZE env.
ioUringSockstatWorkers 4 io_uring: cap on kernel io-wq workers for sockstat reads (1-64). Ignored unless -ioUring. Falls back to IO_URING_SOCKSTAT_WORKERS env.
ipmetaBootstrapPath /share/xtcp2/ipmeta/bootstrap.lookup.parquet.zst optional zstd-compressed lookup artifact baked into the image and used by -enrichAsn before live refresh. Falls back to IPMETA_BOOTSTRAP_PATH env.
ipmetaCachePath /var/lib/xtcp2/ipmeta/current.lookup.parquet.zst optional writable zstd-compressed lookup artifact used by -enrichAsn before the image bootstrap and refreshed after successful live reloads. Falls back to IPMETA_CACHE_PATH env.
ipv4Ttl 3 outgoing IPv4 TTL for xtcp2's TCP listeners (Prometheus + gRPC); 0 = kernel default. The low default keeps replies from traveling far if the host is internet-exposed; raise it for a distant scraper. Falls back to IPV4_TTL env.
Expand Down Expand Up @@ -79,7 +81,13 @@ localityEvents false with -enrichLocality, keep each namespace's locality snapsh
localityRefreshInterval 1m0s how often -enrichLocality re-dumps every namespace's addresses/routes on the reconcile path (new namespaces are always dumped on the next reconcile; failed or loopback-only namespaces retry on a 30s-5m backoff); 0 = discover each namespace once, never refresh. Falls back to LOCALITY_REFRESH_INTERVAL env.
location deployment grouping/facility this daemon runs in (data center, PoP, region, site, …); stamped on every record's `location`. Falls back to LOCATION env.
marshal protobufList Marshaling of the exported data (protobufList, protoJson, protoText, msgpack, jsonl, csv, tsv)
maxDumpBytes 268435456 cap on the bytes of one namespace's inet_diag dump per poll; packets past it are drained undecoded and counted in xtcp_dump_dropped_total{reason="bytes"}. 0 = unlimited. Falls back to MAX_DUMP_BYTES env.
maxLoops 0 Maximum number of loops, or zero (0) for forever
maxNetns 4096 cap on the network namespaces tracked, xtcp2's own included; at the cap new namespaces wait for a freed slot, first come first served, and are counted in xtcp_netns_dropped_total. 0 = unlimited. Falls back to MAX_NETNS env.
maxNetnsAddsPerReconcile 256 cap on namespaces added per reconcile; the rest wait their turn (xtcp_netns_dropped_total{reason="add_rate"}). 0 = no limit. Falls back to MAX_NETNS_ADDS_PER_RECONCILE env.
maxNetnsPerOwner 64 cap on the namespaces one owner (outermost container id) may have tracked; bind-mounted namespaces with no process are exempt (xtcp_netns_dropped_total{reason="owner_quota"}). 0 = no limit. Falls back to MAX_NETNS_PER_OWNER env.
maxRecordsPerNetns 262144 cap on the records one namespace contributes per poll; the rest are dropped before decode and counted in xtcp_dump_dropped_total{reason="records"}. 0 = unlimited. Falls back to MAX_RECORDS_PER_NETNS env.
maxRecordsPerPoll 1048576 budget on the records all namespaces contribute per poll, checked before the dump from each namespace's sockstat count; over it, namespaces above their fair share stop asking for TIME_WAIT/SYN_RECV, then are sampled by port range across polls (xtcp_poll_degraded_total). 0 = off. Falls back to MAX_RECORDS_PER_POLL env.
maxThreads 2000 cap on Go runtime OS threads (debug.SetMaxThreads); 0 = use Go default 10000
modulus 1 modulus. Report every X inetd messages to output
netlinkers 2 decoders per socket that parse netlink dump packets (one receiver reads each socket). increase this if you have many flows
Expand All @@ -102,6 +110,7 @@ pyroscopeUploadSec 15 Seconds between batched profile uploads to Pyroscope.
pyroscopeUrl Pyroscope server URL (e.g. http://127.0.0.1:4040). Empty disables the agent. Falls back to PYROSCOPE_URL env.
reconcileBeforePoll true Reconcile namespaces immediately before each poll cycle, so a newly-appeared namespace is entered within ~1 poll interval. Ties discovery cadence to poll cadence.
reconcileFrequency 6h0m0s Period of the background namespace-reconcile ticker (Method B /proc scan). With -reconcileBeforePoll carrying discovery this is a rare safety-net expected to find nothing (watch mapReconciler dels/stores in Prometheus); default is long (6h). 0 disables it (startup reconcile still runs once).
reconcileTimeout 1s time budget for one reconcile's namespace scan; a scan cut short adds what it found but deletes nothing (xtcp_reconcile_incomplete_total). 0 = no limit. Falls back to RECONCILE_TIMEOUT env.
resolveContainerId false resolve each socket's owning container id from its cgroup into container_id/container_runtime; needs /sys/fs/cgroup readable (mount it + --cgroupns=host in a container). Falls back to CONTAINER_ID_RESOLVE env.
s3AccessKey s3parquet: S3 access key. Prefer S3_ACCESS_KEY_FILE (path to a file, mode 0400, holding the key): a flag value shows in ps and docker inspect, warns, and is refused by production builds. Never logged.
s3AllowInsecure false s3parquet: allow a plaintext http:// endpoint. The credential and every object cross the network unencrypted; for local test fixtures only. A bare host:port and https:// use TLS. Falls back to S3_ALLOW_INSECURE env.
Expand Down
9 changes: 9 additions & 0 deletions cmd/xtcp2/testdata/printflags-golden.txt
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,15 @@
*s3UploadBackoffCap: 119ms
*reconcileFrequency: 120ms
*reconcileBeforePoll: false
*maxDumpBytes: 1201
*maxRecordsPerNetns: 1202
*maxNetns: 1203
*maxRecordsPerPoll: 1204
*ioUringSockstatBatchSize: 1205
*ioUringSockstatWorkers: 1206
*reconcileTimeout: 1.207s
*maxNetnsPerOwner: 1208
*maxNetnsAddsPerReconcile: 1209
*pyroscopeUrl: pyroscopeUrl-v
*pyroscopeAppName: pyroscopeAppName-v
*pyroscopeSampleHz: 136
Expand Down
Loading
Loading