[zuraiz-anjum]

Summary

In the FinQA reward, when the ground truth has one \boxed{} per value, the percentage points fallback parsed the raw predicted value instead of the one with its year label removed. So 2023: 4.5%, 2024: 6.0% against \boxed{4.5} \boxed{6.0} scored 0, while 4.5%, 6.0% scored 1. This passes the cleaned value to the fallback too.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• New environment
• Refactoring

Alignment Checklist

Before submitting, verify:

• I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
• I have checked .claude/docs/INVARIANTS.md and no invariants are violated
• I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

• Not required (bug fix, docs, minor refactoring)
• RFC exists: #___
• RFC needed (will create before merge)

Details

In compute_reward (envs/finqa_env/server/rewards.py), the multi-value branch does this for each predicted value:

1. Strips a label prefix, so "2023: 4.5%" becomes "4.5%" (pred_val_cleaned).
2. Compares normalize_answer(pred_val_cleaned) with the truth. 4.5% reads as 0.045, so against 4.5 it fails.
3. Falls back to normalize_answer(pred_val, convert_percent=False) to read 4.5% as 4.5 percentage points. This used pred_val, so it parsed "2023: 4.5%". parse_number strips the %, tries float("2023: 4.5"), fails and returns None, and the reward is 0.

The single-value path and the unlabelled multi-value path already get this right (test_percentage_points_basic, and the new test_percentage_points_multiple_boxed). The fix is one argument: use pred_val_cleaned in step 3.

Test Plan

Two tests in TestPercentagePointsNotation in tests/envs/test_finqa_environment.py:

• test_percentage_points_multiple_boxed: 4.5%, 6.0% against \boxed{4.5} \boxed{6.0} scores 1.0 and 4.5%, 7.0% scores 0.0. This passes before and after, as a control.
• test_percentage_points_multiple_boxed_with_labels: the same answers with 2023: and 2024: labels. It fails on main and passes with the change.

Run with PYTHONPATH=src:envs pytest tests/envs/test_finqa_environment.py:

• main with only the new tests: 1 failed (test_percentage_points_multiple_boxed_with_labels), the other reward tests passed, the data-backed tests skipped.
• this branch: no failures.

The three checks from .claude/hooks/lint.sh (usort check, ruff format --check, ruff check) are clean on the files this PR touches; the findings a tree-wide run reports are in other files and are the same on main.

---

Note

Low Risk
Localized scoring-path change in reward logic; improves parsing for labeled percentage answers with new tests, no security or broad behavioral surface.

Overview
Fix multi-valued FinQA reward for labeled percentage-point answers. In the multi-\boxed{} branch of compute_reward, the percentage-points fallback now runs normalize_answer on the label-stripped pred_val_cleaned instead of the raw pred_val, so strings like 2023: 4.5%, 2024: 6.0% correctly match \\boxed{4.5} \\boxed{6.0}.

Adds regression tests for multiple \\boxed{} values with comma-separated percentage answers, both with and without year labels, plus a mismatch case that should score 0.

^{Reviewed by Cursor Bugbot for commit 4b5091e. Bugbot is set up for automated code reviews on this repo. Configure here.}