Skip to content

2.7.18.16: HTMLParser incremental feed is quadratic (CVE-2026-15308) - #103

Merged
icanhasmath merged 1 commit into
2.7.18.16from
2.7.18.16-htmlparser-incremental
Oct 28, 2026
Merged

icanhasmath merged 1 commit into
2.7.18.16from
2.7.18.16-htmlparser-incremental

Conversation

@icanhasmath

Copy link
Copy Markdown

Backport of python/cpython pythongh-153030 (CVE-2026-15308, PSF-2026-33) to 2.7.18.x.

Problem: HTMLParser.feed() appended each chunk to rawdata and rescanned everything that hadn't been parsed yet. If a construct stays open (a comment, <?…, doctype, CDATA, a start tag, or <script>), feeding small chunks takes quadratic time. On 2.7.18.15, 10,000 64-byte chunks inside <script> take 22s, versus 5.6s for 5,000.

Fix: feed() collects chunks in a list and only re-parses once enough has accumulated. If the last parse made no progress, it waits until the buffer doubles. close() flushes everything. With the fix, 200,000 chunks (12.8 MB) take 0.1–0.3s per construct.

Behavior note (same as upstream): after a feed() that makes no progress, events for the next small feeds can arrive later: when the buffer doubles, or at close(). No data is lost.

Tests: IncrementalFeedTestCase.

  • Upstream's test is gated on the cpu resource, as upstream does.
  • My extra test checks that feeding one character at a time still yields the same events, and that close() delivers held-back data.
  • test_htmlparser passes 43/43 with -ucpu, and test_htmllib, test_sgmllib, test_robotparser and test_pydoc pass.

sgmllib/htmllib have their own feed() and are unchanged.

Part of 2.7.18.16. It targets the 2.7.18.16 integration branch.

…eeding (CVE-2026-15308)

Backport of python/cpython pythongh-153030 (3.14 07efb08) to 2.7.18.x.

HTMLParser.feed() concatenated each chunk onto rawdata and rescanned the
whole unparsed buffer.  While a construct stays unterminated (a comment,
processing instruction, doctype, CDATA section, start tag or RAWTEXT
element such as <script>), every feed() rescans everything buffered so
far, so feeding attacker-controlled data in small chunks takes quadratic
time.  On 2.7.18.15, 10000 64-byte chunks inside <script> take 22s
(5.6s for 5000).  feed() now collects chunks in a list and only joins
and parses once the pending data reaches a threshold: 1 byte after a
call that made progress, otherwise the current buffer size, so the
buffer must double before it is rescanned.  close() flushes pending data.
200000 chunks now take 0.1-0.3s per construct.

As upstream: when a feed() makes no progress, the events for data in
the next small feeds may be reported later (once the buffer doubles, or
at close()); no data is lost.

2.7.18.x: Lib/HTMLParser.py; del list[:] instead of list.clear().  The
cpu-gated upstream test goes in a new IncrementalFeedTestCase, plus a
test that char-by-char feeding still yields the same events and that
close() delivers held-back data.  sgmllib/htmllib keep their own feed()
and are not changed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@ezequielp-activestate ezequielp-activestate left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers. The change is upstream's pythongh-153030 fix (bcf98dd), with del lst[:] for 2.7, and Lib/html/parser.py hasn't changed upstream since. Mixed str/unicode chunks coerce in ''.join() the same way they did in rawdata + data.

  • Repro: an open construct followed by 10,000 64-byte feed() calls, then close():

    construct base this branch
    <script> 39.2 s 0.03 s
    <!-- 5.1 s 0.01 s
    <?x 1.9 s 0.01 s
  • Events: 400 random documents fed in random 1–8 character chunks give the same events as feeding each document whole, with adjacent data merged.

  • Tests: test_htmlparser (with -ucpu), test_htmllib and test_robotparser pass. Against the base HTMLParser.py, the new cpu-gated test did not finish within 300 s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants