Skip to content

Add policy-gated ETW logging for sidecar GCS - #2982

Merged
Manish Ranjan Mahanta (marma-dev) merged 5 commits into
microsoft:mainfrom
marma-dev:sidecar-gcs-etw-logging
Nov 12, 2026
Merged

Manish Ranjan Mahanta (marma-dev) merged 5 commits into
microsoft:mainfrom
marma-dev:sidecar-gcs-etw-logging

Conversation

@marma-dev

@marma-dev Manish Ranjan Mahanta (marma-dev) commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Emit sidecar GCS diagnostics through a dedicated Microsoft.Virtualization.GCSSidecar ETW provider using the existing etwlogrus hook.
  • Keep emission disabled until the validated confidential policy permits allow_runtime_logging; reuse existing Log Forward Service provider policy for collection without adding a new permission.
  • Include the provider in default collection at Information level and merge provider-name/GUID case variants without duplicate subscriptions.

Validation performed

  • Changed-package Windows suites, targeted Rego logging-policy tests, Go vet, sidecar/host builds, and Linux shared-policy test cross-compilation passed.
  • Live VBS C-WCOW with Secure Boot enabled confirmed sidecar forwarding and continued host parsing with the timestamp fix in Handle formatting markers in forwarded ETW timestamps #2981. Runtime logging denied: zero sidecar events while other providers continued forwarding.
  • Fixed case-insensitive overrides and added canonical/lowercase/uppercase name and GUID regression coverage; provider and sidecar suites passed, and CI-pinned golangci-lint reported zero issues.
  • Ordinary non-confidential WCOW (NanoServer LTSC 2025, guest 10.0.26100.33674) booted with default options, confirmed SIDECAR_ABSENT, and completed its guest command with exit code 0.
  • Separately tested active Log Forward Service collection on the stock 10.0.28000.2178 VBS guest with the new provider absent and its canonical GUID requested: 216 events from other providers continued forwarding without a UVM startup failure.

Compatibility and coverage notes

  • Handle formatting markers in forwarded ETW timestamps #2981 fixes the host timestamp parsing issue exposed by live testing and is required for correct consumption of the sidecar events.
  • Existing policies using default sources must allow the new provider or enable provider dropping; otherwise existing fail-close behavior may reject UVM startup.
  • Ordinary-WCOW startup and active absent-provider collection were validated separately: the LTSC 2025 guest does not advertise Log Forward Service support. An ordinary forwarding-capable WCOW image has not been tested end-to-end.
  • The existing launcher output-processing teardown hang and forced-shutdown partial-record diagnostic remain outside this PR; all test UVMs are stopped.

Startup event buffering

The sidecar now retains at most 128 pre-policy events / 256 KiB (16 KiB per event). Buffered events flush only after validated allow_runtime_logging approval and ETW collector enablement; policy denial discards them. Oversized or unsupported startup records are dropped. No post-policy events are queued while collection is disabled.

Final VBS C-WCOW test (Secure Boot enabled, private Rego sidecar): 14 pre-policy sidecar startup events were delivered after Log Forward Service enabled; host parsed all without timestamp errors. With runtime logging denied: zero sidecar events, while other ETW providers continued forwarding. Unit coverage includes both policy/collector orderings, denial, event and byte bounds, oversized fields, and concurrent state changes.

Manish Ranjan Mahanta added 2 commits November 5, 2026 21:52
Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>
Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>
Comment thread cmd/gcs-sidecar/main.go Resolved
Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>
@marma-dev

Copy link
Copy Markdown
Contributor Author

Investigated the windows-2022 failure: GitHub reports "The self-hosted runner lost communication with the server." Rego policy, repo, LCOW/WCOW-tagged, non-functional, and parity test steps passed before the runner was lost during the shim-test step. The full job log blob is unavailable, so the underlying runner-disconnect cause could not be established. Pushed provider cleanup in 9f13693; a new CI run should validate it.

@apurv15

apurv15 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Shouldn't early boot/init events of gcs side car be captured but relayed only when enabled?

We could maintain a small, bounded in-memory buffer in the sidecar’s logging hook: record only startup diagnostics; flush them to ETW once the policy is validated and allows runtime logging; discard them if policy denies logging.

Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>
@marma-dev

Manish Ranjan Mahanta (marma-dev) commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

apurv15 Implemented the startup-buffer suggestion in e6a6291. The sidecar buffers at most 128 pre-policy events / 256 KiB (16 KiB per event), and flushes only after both runtime-policy approval and ETW collector enablement. Policy denial discards pending events. Live VBS C-WCOW testing forwarded 14 pre-policy startup events; the denied-policy run forwarded zero sidecar events while other ETW providers continued.

Comment thread internal/gcs-sidecar/logging.go Resolved
Comment thread cmd/gcs-sidecar/main.go Resolved
Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>
@marma-dev
Manish Ranjan Mahanta (marma-dev) merged commit 88c223c into microsoft:main Nov 12, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants