Repository navigation
Fix quadratic processing time for consecutive abbreviation definitions - #1702
deadstrobe5 wants to merge 4 commits into
Conversation
|
Agreed, fixed in 796d415 (plus c66f873 for line length). The processor now takes every consecutive definition, whatever follows, so that input gives Current master is inconsistent here: two definitions before |
| if m is None or '\n' in m.group(): | ||
| break | ||
| matches.append(m) | ||
| for m in matches: |
There was a problem hiding this comment.
Now that there is no code between these 2 loops, they should be able to be combined into a single loop.
Although, why not just call finditer up on line 160 and then loop through the results?
There was a problem hiding this comment.
Done in b39a7c0: it is one loop over finditer now.
The loop stops at the first line that is not a definition, so the rest of the block still goes back as one block. Taking every match in the block would split the text between definitions into separate blocks, so a definition line in the middle of a list or paragraph would act like a blank line (in a fuzz run a tight list became a loose one).
Against master, 30,000 random documents differ only where a run of definitions is directly followed by a ===/--- line or a : definition list line; those definitions are now consumed, as in your setext example. 200 definitions: 19.3 ms before, 0.8 ms now.
There was a problem hiding this comment.
This looks better. Thank you.
Taking every match in the block would split the text between definitions into separate blocks, so a definition line in the middle of a list or paragraph would act like a blank line (in a fuzz run a tight list became a loose one).
Then let's add some tests for that, unless they already exist. I'm assuming they don't or you wouldn't have mentioned it.
Description
AbbrBlockprocessor.runrecords one definition per call and puts the rest of the block back on the stack, so a block of N definitions is re-tested by every block processor N times and the remainder is copied N times. A glossary of abbreviations appended to every page (the setup Material for MkDocs documents withpymdownx.snippetsauto_append) hits this on every page.This records the following definitions in the same pass, the same way consecutive reference definitions were handled in #1678. A later definition is only taken when the block starts with a definition, the later one fits on one line, and it is followed by another definition or the end of the block. In any other case a processor with higher priority could claim the remainder (a Setext underline, a table separator or a definition list line after it), so that definition is left to the next pass and the output does not change. A test covers the Setext case.
Median of 5 alternating runs in one process, CPython 3.14,
markdown.Markdown(extensions=[...]).convert():abbronly)abbronly, one run)Pages without definitions are unchanged. I compared the HTML and the recorded abbreviations of the old and new code on 50,000 generated documents (definitions inside lists, block quotes and indented blocks, empty and
''titles, multi-line definitions, mixed with tables, headers and definition lists) and found no differences.AI Assistance Disclosure
No AI tools were used in preparing this PR.
AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.
Claude Code (Claude Opus 5.5) was used for everything ticked below. The review attestation in the box above is left for the human submitter.
AI tools were use to assist with the following:
Checklist