Repository navigation
Add a multithreaded TPC digitizer device - #16090
Merged
Merged
Conversation
sawenzel
requested review from
davidrohr,
shahor02 and
wiechula
as code owners
November 9, 2026 13:40
This was referenced Nov 9, 2026
Collaborator
Author
|
Added commits after internal review. |
sawenzel
force-pushed
the
tpc-digi-mt-v2
branch
from
November 9, 2026 17:53
cd1b7c7 to
48da7ab
Compare
This moves the per-sector loop of the TPC digitizer spec into a free function, as a preparation for MT digitization. - digitizeSector runs the collision loop and the flushes of one sector and returns the digit groups. - The output of each flush is handed to a callback, so the function makes no DPL calls. - The lane spec now calls it; the chunked writer and the DPL outputs stay in the spec. - The output is unchanged. https://its.cern.ch/jira/browse/O2-7309 Co-Authored-By: Claude Opus 5.5 <[email protected]>
This makes several TPC digitizers usable side by side in one process, as a preparation for MT digitization. - Each DigitContainer keeps its own copy of the dead channel map, because CDBInterface::getDeadChannelMap() rebuilds the shared map on every call. - The one-time settings report moves from the flush path into updateCalibrations(). - Digitizer::setSCDistortionsFrom() shares the distortion maps and their scaling with another digitizer. - digitizeSector() takes an optional mutex for the calls that read calibrations through the CDBInterface. - The output is unchanged. https://its.cern.ch/jira/browse/O2-7309 Co-Authored-By: Claude Opus 5.5 <[email protected]>
This adds an optional single TPC digitizer device that digitizes the sectors with several threads instead of lanes. - --tpc-digi-threads N replaces the TPC lanes by one device; it requires --tpc-chunked-writer. - Each thread owns a digitizer with sector seeding, its own hit chains and its own tpc_driftime_digits_lane file, so the existing merger is unchanged. - The CCDB objects are handled once on the device thread, and the distortion maps are recalculated once and shared. - The digits are identical to the lane spec with --tpc-sector-seeding. https://its.cern.ch/jira/browse/O2-7309 Co-Authored-By: Claude Opus 5.5 <[email protected]>
This addresses the review of the multithreaded TPC digitizer device. - The workers copy the random rings of the first worker instead of filling their own. - The workers no longer flush the process-wide debug streamer; the device flushes it after the join. - The Digitizer serialises its calibration reads itself, so digitizeSector needs no mutex. - The CCDB handling, the calibration files and the common options of both devices are in TPCDigitizerConditions. - The device stops on a second timeframe, which the chunked digit files cannot hold. - Shorter comments in TPCSectorDigitization.h. https://its.cern.ch/jira/browse/O2-7309 Co-Authored-By: Claude Opus 5.5 <[email protected]>
This addresses the second review of the multithreaded TPC digitizer device. - Old lane files are found with o2::utils::listFiles, as in the chunked merger. - Unused includes and the SpaceCharge alias are removed from the device. - setSectorSeedingFrom fills the rings of the source digitizer, so initSectorSeeding is private. - The seeding documentation states that the sectors share the ring contents. https://its.cern.ch/jira/browse/O2-7309 Co-Authored-By: Claude Opus 5.5 <[email protected]>
sawenzel
force-pushed
the
tpc-digi-mt-v2
branch
from
November 10, 2026 21:56
b25a4f3 to
57422d7
Compare
Collaborator
|
Error while checking build/O2/fullCI_slc9 for 57422d7 at 2026-11-11 05:18: Full log here. |
wiechula
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds an optional TPC digitizer device that digitizes all sectors in one process with several threads, instead of one process per lane.
With lanes, every process loads and recalculates its own copy of the space-charge distortion maps, which in PbPb costs about 1 GB of memory and several minutes of CPU per lane. In the new device the maps are prepared once and shared by all threads, while each thread owns a digitizer with its own random rings, hit chains and output file. It is enabled with
--tpc-digi-threads Ntogether with--tpc-chunked-writer; the chunked merger and everything downstream are unchanged.The effect on one anchored PbPb timeframe (run 572106, distortion type 2) is substantial:
The lanes run with
--n-threads-distortions 1, as in O2DPG, and the threaded device with 8.Note that nothing changes by default. The digits of the threaded device are identical to those of the lane spec with
--tpc-sector-seedingfor 1, 4 and 8 lanes and threads, with and without distortions, also when run through the O2DPG workflow up to the TPC clusterization.This builds on #16082, #16083 and #16085, which are merged. It moves the per-sector loop into a shared function, prepares the digitizer for concurrent use and adds the device itself. The last two commits address review comments.
Overall, this halves the memory of the TPC digitization and makes it about three times faster.
https://its.cern.ch/jira/browse/O2-7309
Assisted by Claude Code.