[naaz234]

Description

HTMLExtractor.parse_starttag passes < through as text and moves on one character when check_for_whole_start_tag finds no closing >. The locatetagend match behind that check accepts < inside tag and attribute names and carries on across spaces, newlines and blank lines, so in text like a<b a<b a<b with no > after it, every < scans to the end of the document again. Prose with comparisons in it is enough. markdown.markdown(s * n) with no extensions on CPython 3.11, in seconds:

s                           size     master   patched
'a<b '                      100 KB    20.51     0.059
'if a<b and c<d then '      160 KB    29.02     0.048
'<a@a'                      100 KB     2.91     0.047
'a<b ', then '` >' at end   100 KB    20.15     0.053
"a<b x='' "                 100 KB     9.52     0.028

#1676 fixed the same kind of rescan for comments, but its skip does not carry over to start tags. An unclosed tag does not rule out a later one inside it: in <a b<c=" > the first tag is unclosed but <c=" > is a tag. Skipping only when no > follows is not enough either, since a backtick or an unclosed quote where an attribute would start also ends the match (the fourth row). So check_for_whole_start_tag now matches the same text as locatetagend one attribute at a time, and remembers, for the current rawdata, the positions from which the attributes ran out without a > and where the last unclosed tag name ends. The match is greedy and never backtracks past the start of an attribute, so a later < whose tag name ends in the same place, or whose attributes reach one of those positions, has the same result. The second assertion in the new test is the tag name case: <a@a has no space, so each name runs to the end.

For every < followed by a letter, the new check returns the same result as the old locatetagend match on all strings of up to 7 tokens from <a, <, a, space, newline, /, =, ', ", backtick and > (16M strings) and on 300,000 random longer ones, with the positions checked in order and shuffled. Mistakes planted in the remembered positions were caught by the same comparison. Rendered output matches master on 100,000 random documents mixing raw HTML and Markdown, with no extensions, md_in_html and extra. The docs folder with extra and a 400 KB HTML-heavy page take the same time as before. The full suite passes on 3.11 with Pygments 2.7.1 and on 3.13 without it, flake8 is clean, and the new test takes 26 s on master and 0.08 s here.

Python releases from before the 2025 html.parser security fix (3.11.12, for example) found the end of a start tag with locatestarttagend_tolerant, and HTMLExtractor now uses the locatetagend behaviour there too. The suite passes on 3.11.12. Of the same 100,000 documents, 154 render differently from master on 3.11.12, and 133 of those now match what current releases produce.

AI Assistance Disclosure

• No AI tools were used in preparing this PR.

• AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

  AI tools were use to assist with the following:

  • Code
  • Code Comments
  • Documentation
  • Changelog Entry
  • Commit Message
  • The Description Above
  • Other: finding the issue, the regression test, and the timing and output comparison scripts

Checklist

• This PR follows the contribution guidelines.
• The code follows the Code Style Guide.
• The commit message follows the Commit Message Style Guide.
• I have added or updated relevant docs, including release notes if applicable which follow the Documentation Style Guide.
• I have added or updated relevant tests.