Repository navigation
Conversation
axpy (OpenMathLib#6250) stays on one thread on Apple M4 with OpenMP while the vectors are too short to pay for waking the rest of the OpenMP team, since libomp parks idle threads on Apple Silicon. zaxpy and the arm64 asum, sum, copy, dot, zdot, iamax and izamax kernels make the same one-thread-or-all choice above n = 10000 and pay the same cost: with the default 14 threads on an M4 Pro, sasum at n = 12000 takes 44 us instead of 0.42 us. Move the check from axpy.c into common_thread.h as apple_omp_level1_threads(), which counts the bytes a call reads and writes, and use it in all of these. About 384 KB per thread to be woken is at or just below the measured crossover for every routine; for axpy, which moves three vectors, it is the 128 KB per vector of OpenMathLib#6250. Apply it to unit-stride calls only. Calls with other strides handle one element per iteration, 1.5 to 15 times slower per element than unit stride, so one thread loses to the team well inside the byte budget. OpenMathLib#6250 got this wrong for real axpy: saxpy with n = 300000 and stride 2 takes 151 us in develop and 61 us with this change, because strided calls keep the team again, as before OpenMathLib#6250. Only OpenMP builds for VORTEXM4, and DYNAMIC_ARCH OpenMP builds running on vortexm4, change behaviour; measured on an M4 Pro.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #6250, which it also corrects for non-unit strides (below). With OpenMP on Apple M4, the other level-1 routines that thread above n = 10000 have the same problem axpy had: libomp parks idle threads on Apple Silicon (
KMP_BLOCKTIMEdefaults to 0 there), and above n = 10000 every call wakes the whole team, which costs about 44 us with the default 14 threads on an M4 Pro, whatever the size of the call. That covers zaxpy (interface) and the arm64 asum, sum, copy, dot, zdot, iamax and izamax kernels that VORTEXM4 uses.This moves the check from axpy.c into
common_thread.hasapple_omp_level1_threads(bytes, unit_stride, nthreads), wherebytesis the memory the call reads and writes, and calls it right after each of these routines picks its thread count. A unit-stride call stays on one thread while it moves less than (nthreads - 1) x 384 KB. Measured against one thread, the crossover with the default team is between about 5.2 and 6.5 MB of traffic for all of these routines (asum, sum and iamax read one vector, dot reads two, copy reads one and writes one, axpy reads two and writes one), so one budget per woken thread fits all of them. For axpy, which moves three vectors, it is exactly the 128 KB per vector of #6250, so unit-stride axpy behaves as before.Calls with any other stride keep the whole team. Their kernels handle one element per iteration and are 1.5 to 15 times slower per element than the unit-stride ones (about 15 for sasum), so one thread loses to the team well inside the byte budget. #6250 got this wrong for real axpy: with this change, saxpy with n = 300000 and stride 2 goes from 151 us in develop to 61 us, and daxpy with n = 200000 and stride 4 from 113 us to 57 us, because strided axpy uses the team again, as before #6250.
As before, it applies only to OpenMP builds for VORTEXM4 and to DYNAMIC_ARCH OpenMP builds running on vortexm4; pthreads builds and other targets behave as before. With
OMP_WAIT_POLICY=activelibomp keeps the team spinning and waking it is cheap, so calls near the budget are faster with the team (caxpy at n = 200000: 14 us with the team, 40.5 us on one thread), while smaller ones still gain; #6250 has the same trade-off. In Makefile builds VORTEXM4's scsum uses csum.S, so the csum_thunderx2t99.c change only takes effect in CMake builds.Measured on an M4 Pro (10 performance and 4 efficiency cores), macOS 15.7.9, develop built with Homebrew LLVM 23.1.5 as
DYNAMIC_ARCH=1 USE_OPENMP=1 NUM_THREADS=56(without LAPACK). Each cell is the mean call time over 100 ms of back-to-back calls, starting 150 ms into the process, as the median over 7 rounds that run all cells in a new random order; the ratio is the median before over the median after, with a 95% bootstrap interval over rounds. Default thread count (14); cells marked (same) use the same thread count in both builds:Over the full run (these 14 routines, n = 12000 to 2000000, and the default, 4 and 8 threads), all 122 cells where the thread count changes are faster, by 1.22 to 126 times; the other 130 cells use the same thread count in both builds and range from 0.91 to 1.09 across processes.
Results: copy, iamax, izamax and real axpy are bit for bit unchanged. The reductions (asum, sum, dot) and, with unit stride, complex axpy already depend on how the vector is split between threads: in develop, caxpy at n = 30000 gives slightly different values for the last elements of each thread's part with 14 threads than with one (up to 4.5e-6 relative per real or imaginary part, about 1e-7 relative to |y|). Where this change moves a call to one thread, the result is now exactly the one-thread result of develop; elsewhere it is unchanged. Checked for s, d, c and z variants, sizes on both sides of the threshold for teams 2 to 14, unit, positive, negative and zero strides, and 1, 2, 4, 8, 12 and 14 threads, OMP_ADAPTIVE=1, openblas_set_num_threads and a nested parallel region; the thread counts actually used match the rule in every case.