[chivector]

Problem

CiderSampleEvaluator(df_mode="corpus") constructs a separate one-document scorer for every row. Each scorer therefore uses log(1)=0 for the corpus size, making every TF-IDF weight zero. Two distinct five-word references, each matched exactly by its generated text, receive [0, 0] instead of [10, 10].

This makes the corpus mode unable to distinguish matching text from unrelated text.

Change

Build one local CIDEr scorer for all rows of each eval() call, estimate document frequencies from that input corpus, then return its per-row scores in the original order. The scorer is recreated for each call.

The precomputed-IDF path and empty-input behavior remain unchanged. A genuine singleton corpus still has zero IDF; the fix supplies the intended multi-document corpus instead of treating each row as one.

Validation

Added ten CPU cases using the real evaluator and scorer, including:

• Exact matches and unrelated text, row order and custom output keys.
• Gaussian length penalties checked against an independent hand-derived formula for three sigma values.
• Zero weight for n-grams present in every reference.
• Repeated calls, a singleton and an empty frame.
• External IDF remaining independent of the evaluation corpus.

Original source with the new tests: 7 failed, 3 passed. Fixed source plus related evaluation/deduplication regressions: 28 passed. Independent code review and test reruns passed, as did Ruff lint/format on the new tests and git diff --check.

python -m pytest test/cpu_only/test_cider_sample_evaluator.py test/cpu_only/test_ngram_sample_evaluator.py test/cpu_only/test_dedup_single_input_keys.py test/cpu_only/test_minhash_short_texts.py -q

Validated on Windows, Python 3.12.14, NumPy 1.26.4 and pandas 2.2.3. The full model/API test suite was not run.