Skip to content

Clean untrusted weblog HTML with jsoup's Cleaner - #356

Open
mraible wants to merge 26 commits into
masterfrom
weblog-sanitizer-jsoup
Open

mraible wants to merge 26 commits into
masterfrom
weblog-sanitizer-jsoup

Conversation

@mraible

@mraible mraible commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

This builds on #353, so its diff includes #353's commits until that merges.

When weblogAdminsUntrusted is true (the default), HTMLSanitizer cleans entry text, titles, summaries, user names, bookmark names and weblog taglines. It was a 528-line regular-expression parser from Open Lab (2009), and CodeQL flags it six times for polynomial ReDoS (alerts 89–94). In the new test, a few kilobytes of crafted markup took the old class 43.6 seconds; it takes 28 ms now.

It now uses jsoup's Cleaner with a Safelist that keeps the same rules:

  • the same tags, including image maps and param, without the forbidden script, object, embed, link, style, form and input
  • links to http, https and mailto addresses, with rel and target, and images from http and https or image data that InlineImageData accepts
  • width and height only as pixels or percentages
  • style checked one declaration at a time, keeping only those that can't run script or load anything other than an http(s) url() (no image-set(), expression(), @import, behaviors, escapes or comments)
  • no event handlers
  • quotes in text encoded, so titles and taglines stay safe inside attributes in custom themes
  • a < with no > after it encoded before parsing, so text such as Why x<y matters keeps everything after the <

Names, e-mail addresses, OpenID URLs and blogroll names are plain text, so they go through sanitizeText, which keeps a literal & instead of reading it as the start of an entity.

What changes:

  • Attributes that aren't listed are removed instead of being kept HTML-encoded. The list covers the common ones: class, id, title, lang, dir, align, alt, border, colspan, rowspan, valign, summary, cellpadding, cellspacing, start, type, name.
  • The RSS item titles, the RSS channel descriptions and PaperMod's post link labels turn the sanitizer's &amp; back into & before escaping, so they don't show &amp;. The RSS comment feed also removes markup from the entry title, as the entries feed does.
  • The unused helpers (isSanitized, getText, sanitizer, encode, htmlEncode*, convertLineFeedToBR, removeLineFeed, replaceAllNoRegex) go with the old parser, so none of the Open Lab code is left.

The new HTMLSanitizerTest covers kept markup, scripts, styles, event handlers, javascript: and relative links, iframes, plugins, forms, SVG, unsafe styles and sizes, unclosed <, plain-text ampersands, and the slow inputs. FeedTitleEscapingTest covers the RSS titles. The existing inline image tests pass unchanged. CHANGES.md has an entry under the behaviour changes.

6.1.x keeps its current sanitizer.

UserWrapper now returns screen and full names as HTML-escaped text, encoding them exactly once whether they were stored as typed or already encoded by the sanitizer, and the built-in feed macros turn them back into text before XML-escaping them for Atom author names and RSS dc:creator.

HTML comment notification emails escape the commenter's name and email address. WeblogEntryCommentWrapper escapes HTML comments when the HTMLSubset plugin isn't both configured and listed on the comment, and its Javadoc now describes what getContent does.

Utilities.transformToHTMLSubset and addNofollow find each link's closing bracket with one forward search and stop when none is left, so their time is linear in the input, and addNofollow now adds rel="nofollow" to links that don't already have it.

TagDataServlet writes tag names and the weblog handle with JSON and XML escaping. The planet group pages pass handles and feed URLs to their delete handlers through data attributes, and JspOutputEncodingTest rejects property values inside inline event handlers. CommentDataServlet returns comment content as JSON, which Comments.jsp reads with jQuery's json data type, and UserDataServlet declares its plain text response.
WeblogEntryCommentWrapper.getContent now HTML-escapes any comment that isn't text/html with the HTMLSubset plugin applied, including comments without a content type.
…isting rel values in addNofollow

The comment feed macros and the media file feed now turn author names back into text with unescapeHTML before XML-escaping them, as the entry author macros do, so a name like "A & B" no longer shows as "A &amp; B" in feed readers. FeedAuthorEscapingTest covers the Atom and RSS comment feeds and the media file feed.

Utilities.addNofollow adds nofollow to a link's existing rel value instead of adding a second rel attribute, and a ">" inside a quoted attribute value no longer ends the link early. The linear-time test now allows 2 seconds so it doesn't fail on slow build machines.
addNofollow looked for rel= anywhere in the tag, so a link like <a href="/x" title='foo rel=external' rel="noopener"> had nofollow added to the title text while its real rel stayed followable. It now walks the tag's attributes while tracking quotes, the same way findTagEnd finds the end of the tag, and changes only a real rel attribute. A rel with no value gets rel="nofollow".
addNofollow now parses the HTML with jsoup 1.24.1 and adds nofollow to the rel attribute of each a element with an href, instead of scanning the markup by hand. Only a link's own rel is changed, and links with no href are left alone. Text with no links is returned as is.

The result is jsoup's serialization of the fragment, so open tags are closed, attribute values are double-quoted, and a tag that never ends is dropped, as browsers do. Escaped text stays escaped.

jsoup is credited in the binary LICENSE.
The HTML subset was built with regular expressions: the comment was HTML-encoded, then 21 patterns turned the encoded tags in the subset back into tags and rebuilt links. Utilities.sanitizeToHTMLSubset now uses jsoup's Cleaner with a Safelist of the same tags (a, b, blockquote, br, i, li, ol, p, pre, ul) and the same link schemes (http, https, mailto) on the raw HTML. The HTMLSubset comment plugin and HTML comment notification e-mails call it directly. transformToHTMLSubset stays for templates and decodes its HTML-encoded input before cleaning it.

What changes for commenters:
- A tag outside the subset is removed and its text kept, instead of being shown as encoded markup. Script and style contents are removed.
- A link keeps its href when it has other attributes, such as a title; the other attributes are removed.
- The output is jsoup's: tags are balanced, <br> has no slash, and the link scheme is lowercase.

The new tests cover script, style, event handler, javascript: and data: links, an entity-split javascript: scheme, SVG, inline styles, HTML comments, iframes, and a quote inside an href.
When weblogAdminsUntrusted is true, HTMLSanitizer cleans entry text, titles, summaries, user names, bookmark names and weblog taglines. It was a 528-line regular-expression parser from Open Lab (2009) that CodeQL flags six times for polynomial ReDoS: in the new test, a few kilobytes of crafted markup took the old class 43.6 seconds, and takes 28 ms now.

It now uses jsoup's Cleaner with a Safelist that keeps the same rules:
- the same tags, without the forbidden script, object, embed, link, style, form and input
- links to http, https and mailto addresses, and images from http and https or image data that InlineImageData accepts
- width and height only as pixels or percentages
- style only when it can't load anything or run script (no url(), image-set(), expression(), @import, behaviors, escapes or comments)
- no event handlers

Unlisted attributes are now removed instead of being kept HTML-encoded; the list covers the common ones (class, id, title, lang, dir, align, alt, border, colspan, rowspan, valign, summary, cellpadding, cellspacing, start, type, name). The unused helpers (isSanitized, getText, sanitizer, encode, htmlEncode*, convertLineFeedToBR, removeLineFeed, replaceAllNoRegex) are removed with the old parser, so none of the Open Lab code is left.
A comment saved without a name, or a media file whose creator can't be found, made the comment and media file feeds print the template expression itself as the author, because unescapeHTML passed null through and Velocity prints a null reference as its source text. The template unescapeHTML helper now returns an empty string for null, as removeHTML does, so the author element is empty again.
Quotes in text are encoded again as &quot; and &#39;, so a sanitized title or tagline is still safe when a theme puts it inside an attribute value. Attribute values are left as jsoup writes them, and sanitizing twice gives the same result.

User, bookmark and bookmark folder setters now use a plain-text variant that leaves an ampersand in the text as an ampersand and doesn't read it as the start of an entity, so an e-mail address, OpenID URL or name with an & is stored as typed. Tags are still stripped and quotes still encoded for these fields.

RSS entry and comment titles are plain text, so the feed macros now decode the entities in the entry title before XML-escaping it. A title with an & or a quote no longer shows up as &amp; or &quot; in feed readers.

The allow-list keeps area, map and param again, along with rel and target on links, usemap on images, value and type on list items, and scope and nowrap on table cells. An area's href is still dropped.

Styles are checked one declaration at a time. A declaration is dropped if it has expression(, eval( or javascript( in any case, a url() that isn't http or https, image-set(), a behavior or -moz-binding property, a backslash or a comment. The other declarations are kept, so scroll-behavior and http(s) background images work again.

Width and height must be ASCII digits with no surrounding spaces, and the long-input test uses assertTimeoutPreemptively with a 30 second bound.
Comments run through the HTML subset lost any text an HTML parser reads as a tag outside the subset: generics such as List<String>, comparisons such as a<b and c>d, <[email protected]> addresses, <http://...> URLs, XML snippets, and everything after a link tag that never ends. Before the regular expressions were replaced with jsoup, that text was shown, HTML-encoded.

sanitizeToHTMLSubset now encodes every "<" that doesn't start a complete tag of the subset before jsoup's Cleaner runs, so the parser reads it as text. A formatting tag or end tag is its name and nothing else, and a link's start tag has attributes with values and ends at a ">" outside a quoted value. The scan from each "<" stops at the next one, so it takes linear time. The Cleaner still runs last and decides what the output may contain.

transformToHTMLSubset gets the same behavior for encoded text, so &lt;script&gt;how to&lt;/script&gt; stays text. This also removes the unused replace helper and regex imports from Utilities, and updates the UtilitiesModel Javadoc for transformToHTMLSubset and addNofollow.

@snoopdave snoopdave left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of 0dc851434: The jsoup replacement covers the affected rendering paths, and its tests exercise the previously slow markup. I found no verified issues in the base-to-head change.

No new issues found.

Neither trust setting stores an ampersand encoded: trusted admins' names are stored as typed, and the sanitizer used for untrusted admins only encodes angle brackets and quotes. UserWrapper now decodes just &lt;, &gt;, &quot; and &#39;, and only when weblogAdminsUntrusted is true, before escaping, so a name typed as "Tom &amp; Jerry" displays as typed in both modes. Media file authors in the feeds go through UserWrapper too, with a new $utils.wrapUser(), instead of decoding the stored name.
jsoup's sanitizer writes a non-breaking space in a name as &nbsp;, so UserWrapper now decodes &nbsp; along with the angle bracket and quote entities when weblogAdminsUntrusted is true. The name then shows a non-breaking space, encoded once, instead of the text "&nbsp;".
jsoup reads an entity name such as &copy or &region as a character even without a semicolon, so a URL like ?a=1&copy=2 pasted into a comment came out with a symbol in it. An ampersand that doesn't start a complete character reference is now encoded before the comment is parsed, as the old code showed it.
The test failed whenever five large parses took more than two seconds of wall-clock time, which a busy build machine can cause with no change in the code. It now uses the same 30 second assertTimeoutPreemptively as HTMLSanitizerTest, so only a scan that has stopped being linear fails it.
The wrapper decodes the &#39; the sanitizer stores and escapeHtml4 leaves an apostrophe as it is, so a name reached templates with a raw apostrophe. A custom theme that printed a name inside a single-quoted attribute got markup it didn't expect. The wrapper now encodes apostrophes as &#39;, as the stored value had them.
…nges

A name keeps the encoding it was saved with, but the wrapper decoded the sanitizer's entities only while weblogAdminsUntrusted was true. After an operator turned the setting off, a name such as O'Brien saved earlier showed as O&#39;Brien on pages and in feeds. The wrapper now decodes those few entities whatever the setting is, as it decoded every entity before.
Links with other attributes now keep their href, and a tag a comment leaves open is closed at its end. A test now checks the second.
jsoup drops a tag that runs to the end of its input, so with weblogAdminsUntrusted on, a title such as "Why x<y matters" was cut to "Why x", and a screen name such as "Me<You" was saved as "Me". The sanitizer now encodes each < after the last >, as the old sanitizer did, so the rest is kept as text.
The RSS entries feed strips tags from an entry title before turning its entities into text, but the comments feed didn't, so a title such as <em>Hello</em> World showed its tags as text. The comments feed now does the same as the entries feed.
…k labels

The sanitizer writes an ampersand in a tagline or title as &amp;, and these templates escaped that again, so a tagline such as News & Views reached RSS readers as News &amp; Views and screen readers heard the same in PaperMod's post link labels. They now turn the entities back into text before escaping, as the RSS item titles do.
With weblogAdminsUntrusted on, attributes outside the list are now removed instead of kept as encoded text. The entry also lists the size and style rules the new safelist applies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants