Skip to content

Match weblog banned words as plain words and stop rules that backtrack without end - #357

Open
mraible wants to merge 10 commits into
masterfrom
banned-words-time-limit
Open

mraible wants to merge 10 commits into
masterfrom
banned-words-time-limit

Conversation

@mraible

@mraible mraible commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Banned words rules are regular expressions: rules that start with ( directly, and every other rule as \b(rule)\b. A weblog's own banned words and the site's list are compiled as written, so a rule like ((x+)+)+y held a comment post or a referrer check for as long as it backtracked. CodeQL flags this (alert 95). A comment post checks the commenter's name, e-mail, URL and text.

With weblogAdminsUntrusted=true (the default), a weblog's own banned words are now plain words, not regular expressions:

  • Each line is matched as literal text, case-insensitive in any script, at word edges. A line that starts with ( is no longer a regex.
  • A line that is only letters, digits and spaces in parentheses, such as (casino), matches the text inside anywhere, in any case, so rules like that from earlier versions keep working. Weblog Settings doesn't list these lines.
  • A word in Chinese, Japanese, Thai or another script written without spaces has no edge check, so it matches inside running text.
  • Lines that look blank, such as a lone non-breaking space, are skipped.
  • Weblog Settings labels the field "one word or phrase per line", reports how many rules it accepted, and lists the lines that look like regular expressions, both when the page opens and when the list is saved.
  • The built-in list and the site-wide list from Server Admin are still regexes, and so are weblog rules on sites with weblogAdminsUntrusted=false.

This is a breaking change for weblogs whose banned words use other regex syntax: a rule such as (casino|poker) or viagra|cialis now matches only that literal text. Weblog Settings names those lines so the weblog admin can rewrite them, and CHANGES.md has an item under the 7.0.0 behaviour changes.

Each regex rule is matched through a CharSequence that stops it after 1 second. Every rule gets its own limit, so a long list of plain words against a long comment is slow but never counted as a match. A rule that runs out of time, or out of stack (for example (a|b)+c on long text), is logged. A comment is then treated as spam, while a referrer is let through, so a reader isn't refused the page because a rule was slow. The clock is checked as the match reads the text, so the limit doesn't stop a regex that repeats without reading it. Such rules can only come from the site-wide list or from trusted weblog admins.

A weblog line saved as a plain word that isn't a valid regex, such as (18+, is matched as plain text if the site later sets weblogAdminsUntrusted=false, rather than failing every comment post. Saving the list on a trusted site still refuses it. Blank lines in the site-wide list are now skipped too, rather than matching every value.

The user guide's banned words section and the security model describe the new behavior.

Tests in BannedwordslistTest:

  • The case for ((x+)+)+y timed out before the first commit, both as a regex rule and as a word rule, and passes after it.
  • Weblog rules are matched as words when the setting is on, including words in scripts without spaces, while a site-wide rule is still a regex. With the setting off, weblog rules are regexes again.
  • New cases cover (casino)-style lines, blank-looking lines, an invalid regex line, the lines listed as regex-style, 20,000 plain-word rules against a 10,000-character comment, a referrer that can't be checked in time, and a rule that runs out of stack. Each of them failed before its fix.

The 6.1.x change (#358) keeps the time limit only.

Banned words rules are regular expressions: rules that start with ( directly, and every other rule as \b(rule)\b. A weblog's own banned words and the site's list are compiled as written, so a rule like ((x+)+)+y held a comment post (checked against the commenter's name, e-mail, URL and text) or a referrer check for as long as it backtracked, which CodeQL flags as regex injection.

Each value is now matched through a CharSequence that stops the match once 1 second has passed. The value then counts as a match, so a comment that sets off such a rule is caught as spam, and the rule is logged so it can be simplified. Ordinary checks take a few milliseconds; 40,000 characters against the bundled list take about 24 ms.
@mraible
mraible requested a review from snoopdave November 18, 2026 17:50
@mraible mraible added the 7.0.0 label Nov 19, 2026
…re untrusted

With weblogAdminsUntrusted=true, the default, each line of a weblog's banned words is matched as literal text, case-insensitive, at word boundaries, so a weblog admin can't write a regex rule that holds a comment post or a referrer check. The built-in and site-wide lists are still regexes, as are weblog rules when the setting is off. Weblog Settings labels the field accordingly, and the user guide and security model describe the change.
@mraible mraible changed the title Stop a banned words rule that backtracks without end Match weblog banned words as plain words and stop rules that backtrack without end Nov 19, 2026

@snoopdave snoopdave left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of a8b787415: The matching deadline is shared with #358. The plain-word parsing and UI changes are specific to master, but Unicode words do not yet match as described.

A plain-word rule such as café now ignores case in any script (UNICODE_CASE), so it matches CAFÉ, and checks its word edges with lookarounds over Unicode letters, marks, digits and underscores, so it no longer matches inside caféine. Edges that aren't word characters, as in (foo) or example.com/, still get no check.
…rough when a rule can't check it

All rules used to share one deadline per value, so a long list of plain words against a long comment could run out of time and count the comment as spam. Each rule now has its own limit. A rule that runs out of stack, such as (a|b)+c on long text, is handled like one that runs out of time instead of failing the request. A referrer that can't be checked is no longer refused, since the reader would get a 403 for a slow rule.
…that doesn't compile as plain text

A line holding only a non-breaking or ideographic space became a rule that matched any comment with that character, and an empty line in the site-wide list, or in a weblog's list on a trusted site, matched every comment. Such lines are now skipped.

While weblog admins are untrusted, a line such as (18+ is saved as a plain word. If the site later sets weblogAdminsUntrusted=false, that line is not a valid regex, and every comment post on the weblog failed. At check time such a line is now logged and matched as plain text, and Weblog Settings still refuses it when the list is saved as regexes.
…ipts written without spaces

A plain-word rule checked for a word edge on each end, and in these scripts a word is usually next to other letters, so a rule such as 赌场 never matched running text. There is now no edge check next to a letter of such a script.
…blog Settings

While weblog admins are untrusted, every line is matched as plain text, so a rule from an earlier version such as (casino) or viagra|cialis no longer matches what it used to. Weblog Settings now names those lines when the page opens and when the list is saved, so the weblog admin can rewrite them as words.
…lso match

The site-wide rule (c|d) matched the c in the URL, so the assertion passed even without the example.com/ rule. The test now uses a rule that can't match the URL, and checks that a different domain doesn't match.
With weblogAdminsUntrusted=true, a weblog line that is only letters, combining marks, digits and spaces inside one pair of parentheses is matched as the text inside, anywhere in the value and in any case, so rules written for earlier versions keep matching what they used to. Weblog Settings no longer lists such lines as looking like regular expressions. Any other line, including one with other regex syntax such as (casino|poker), is still matched as literal text at word edges.
if (token.startsWith("(")) {
regexRules.add(Pattern.compile(token));
try {
regexRules.add(Pattern.compile(token));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants