Repository navigation
Conversation
The parser assumed every PCC tree starts at The Linux Foundation and dropped the top level, so projects under a foundation were compared against the wrong group. The group is now the top ancestor of the spine, the project is the level below it, and the leaf is the subproject, for any depth. Leaves with no spine rows are reported as such instead of as unexpected roots. Refs linuxfoundation/lfx-cdp-insights-ops#984 Signed-off-by: Umberto Sgueglia <[email protected]>
PR SummaryMedium Risk Overview Malformed spine rows are handled more accurately: invalid levels below 1 are dropped (e.g. null Adds Reviewed by Cursor Bugbot for commit 86ec6a3. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the PCC-to-CDP hierarchy-mapping update.
No open issues 🎉
What changed in this PR
Corrects PCC-to-CDP hierarchy mapping so the sync uses the top ancestor as the group instead of always discarding it. This addresses false hierarchy mismatches while preserving the sync’s behavior of reporting mismatches without moving projects.
Changes:
- Maps the top ancestor, its immediate child, and the leaf to CDP’s group, project, and subproject levels, including standalone projects.
- Rejects unusable hierarchy levels and removes unused
effectiveDepthandmappingRulefields. - Adds parser tests for hierarchy mapping, row ordering, buffer-encoded levels, metadata, and error cases.
| File | Description |
|---|---|
services/apps/pcc_sync_worker/src/parser/types.ts |
Removes obsolete depth-mapping types and fields; updates error-field comments. |
services/apps/pcc_sync_worker/src/parser/rowParser.ts |
Implements top-ancestor mapping and tightens hierarchy-level validation. |
services/apps/pcc_sync_worker/src/parser/rowParser.test.ts |
Adds 11 tests covering mapping, input handling, metadata, and errors. |
services/apps/pcc_sync_worker/src/parser/index.ts |
Removes the obsolete MappingRule export. |
services/apps/pcc_sync_worker/src/consumer/pccProjectConsumer.ts |
Updates comments to reflect the new hierarchy rules; no functional changes. |
Review effort: Balanced
Summary
The PCC sync compared every project against the wrong CDP group. The parser assumed every PCC tree starts at The Linux Foundation and dropped the top level. In
PROJECT_SPINEthe foundations are at the top, and only Community Projects sit under The Linux Foundation, so most projects were reported asHIERARCHY_MISMATCHwhile CDP had them in the right place.rowParser.ts: one rule for any depth. The group is the top ancestor, the project is the level below it (the top ancestor itself for a project with no parent), and the sub-project is the leaf. Community Projects need no special case: their top ancestor is "The Linux Foundation", which is the name of the CDPtlfgroup.rowParser.ts: a leaf with noPROJECT_SPINErows came through the LEFT JOIN with a null level, parsed as 0, and was reported as "unexpected root". It is now reported as having no valid hierarchy rows.types.ts,index.ts:effectiveDepthandmappingRuleare removed. Nothing read them.pccProjectConsumer.ts: comments only, they described the old depth rules.rowParser.test.ts: first tests of this worker. Trees of 1 to 5 levels, a Community Project, unordered rows, a level serialized as a Parquet buffer, leaf metadata, and the three error cases.Effect
Both parsers were run locally on the Snowflake export and the CDP segments of 2026-11-13 (1,921 leaf projects). Nothing was written. The old parser reproduces last night's production run exactly.
HIERARCHY_MISMATCHSCHEMA_MISMATCHNot in this PR
Verification
pnpm exec vitest run --project server services/apps/pcc_sync_worker/src/parser/rowParser.test.ts: 11 passedpnpm lint --deny-warnings --format=agent,pnpm tsc-check: cleanPCC_DRY_RUN=trueand compare thePCC job completedlog with the table above.Refs linuxfoundation/lfx-cdp-insights-ops#984
🤖 Generated with Claude Code