GHSA-fg7f-2386-8897: Natural Language Toolkit (NLTK): ReDoS in NLTK ReviewsCorpusReader FEATURES regex

Summary ReviewsCorpusReader extracts feature annotations of the form *label* followed by a bracketed signed digit (e.g. a label then [+2]) from each review line, using the module-level FEATURES regex. The feature-label sub-pattern is unbounded — an optional greedy run of word-plus-whitespace groups followed by another word, which must then be followed by a literal [. On a long bracket-less line the label can match from every search position to the end of the line, causing quadratic backtracking. A single crafted line in a reviews corpus hangs reviews(), features(), and sents(). Details The label alternative is a greedy, unanchored run of word-plus-whitespace groups followed by a word, which must then be followed by a literal [. On an input that is a long sequence of word-plus-whitespace with no bracket, at each of the *n* starting positions the engine greedily extends the label to the end of the line, only then fails to find the bracket, and backtracks the whole way. re.findall repeats this from every position, giving O(n²) total work. There is no exponential blow-up, but quadratic growth on an attacker-controlled line length is enough to hang the reader: a single line of ~100,000 words consumes CPU for tens of seconds to minutes. PoC import multiprocessing as mp import re import time --- The vulnerable regex, verbatim from nltk/corpus/reader/reviews.py L70-71 --- FEATURES_VULN = re.compile(r"((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]") --- Bounded variant from the fix (PR #3583): cap the per-label word run. A generous bound (real feature labels are short noun phrases) makes the run linear while never affecting legitimate corpora. --- WORD_BOUND = 50 FEATURES_FIXED = re.compile( r"((?:(?:\w+\s){0,%d})?\w+)\[((?:\+|\-)\d)\]" % WORD_BOUND ) TIMEOUT = 20.0 # seconds, per measurement SIZES = [1000, 2000, 4000, 8000, 16000] # words on a single bracket-less line def _bad_line(n_words): """A long line of plain words with NO trailing bracketed annotation.""" return ("word