[cyberspace-cs]

What this fixes

RemoveStopwordsRefiner rebuilt the English stopword set for every row:

def remove_stopwords(self, text):
    words = text.split()
    stopwords_list = set(stopwords.words('english'))   # re-reads the corpus on every call

stopwords.words() opens and parses the corpus file on each call, so a table of n rows performs n file reads and n set constructions before a single token is filtered. The corpus is static for the lifetime of the operator, so it only needs to be read once.

Measured on 20,000 rows (single sentence per row, LazyFileStorage + jsonl, Windows / Python 3.13):

	total	per row
before	17.01 s	0.851 ms
after	0.08 s	0.004 ms

Roughly 210× faster on this workload, and the per-row cost no longer depends on the corpus at all.

Second fix: missing cells abort the step

A null cell (None / NaN) reaches text.split() and raises AttributeError: 'float' object has no attribute 'split', which aborts the whole refine step — one missing value kills the run. Missing cells are now passed through unchanged, which keeps the output row-aligned with the input, the same way WordNumberFilter (#563) and the other general_text refiners behave.

Tests

New file test/cpu_only/test_remove_stopwords_refiner.py, three cases:

• test_stopwords_are_removed — normal text still filters as before.
• test_missing_cells_are_preserved — None and NaN survive and the row count is preserved.
• test_corpus_is_read_once_per_operator — patches stopwords.words and asserts it is called exactly once per operator, so the per-row rebuild cannot silently come back.

On main, two of the three fail (the null cell raises, and the corpus counter records one load per row); with the patch:

$ pytest test/cpu_only/test_remove_stopwords_refiner.py -q
3 passed

No API or output-schema change: the operator still returns [input_key] and writes the refined text back into the same column.

[cyberspace-cs]

Friendly bump for review, @SunnyHaze — CI has been green on all six matrix jobs since 2026-10-20 00:48 and the branch is still mergeable: clean.

Recap of the two things this touches, in case it helps prioritise:

1. RemoveStopwordsRefiner rebuilt the stopword set for every row: 20k rows went from 17.01s to 0.08s by loading the corpus once.
2. A missing text cell reached text.split() and aborted the whole run; missing cells are now treated as having no tokens, matching how WordNumberFilter counts them (fix(filters): count missing text cells as zero words in WordNumberFilter #563).

If the two concerns would be easier to review separately, I am happy to split the NaN handling out into its own PR.