Skip to content

Fix contracting dim match and batch broadcasting in broadcast_to_matmul - #3103

Open
Hardik K (hardikkaurani) wants to merge 1 commit into
microsoft:mainfrom
hardikkaurani:fix/broadcast-to-matmul-contracting-and-batch-dim
Open

Hardik K (hardikkaurani) wants to merge 1 commit into
microsoft:mainfrom
hardikkaurani:fix/broadcast-to-matmul-contracting-and-batch-dim

Conversation

@hardikkaurani

Copy link
Copy Markdown

Summary

In _broadcast_to_matmul.py, check_if_not_need_reshape inspected reversed input dimensions with:

if dim_from_a not in {1, dim_from_b}:
    return False

This introduced two issues:

  1. Invalid graph rewrite on contracting dimension mismatch (idx == 0):
    At idx == 0, dim_from_a and dim_from_b are the contracting dimensions $K$ of $A$ and $B$. In MatMul, $K$ cannot broadcast and must match exactly. Accepting dim_from_a == 1 when dim_from_b != 1 erroneously permitted rewriting valid reshape graphs into invalid MatMul nodes with mismatched contracting dimensions that fail ONNX shape inference.
  2. Rejected rewrite for valid batch broadcasting (idx > 0):
    At idx > 0, dim_from_a and dim_from_b are batch dimensions. Two dimensions are broadcastable if $d_A = d_B$, $d_A = 1$, or $d_B = 1$. The previous check rejected valid broadcasting when $d_B = 1$ and $d_A > 1$ (e.g. $[2, 3, 4]$ and $[1, 4, 5]$).

Changes

  • Enforce exact equality on contracting dimensions at idx == 0 (dim_from_a == dim_from_b).
  • Enable standard symmetric broadcasting on batch dimensions at idx > 0 (dim_from_a == dim_from_b or dim_from_a == 1 or dim_from_b == 1).
  • Added regression unit tests to _broadcast_to_matmul_test.py:
    • test_reshape_matmul_reshape_replace_when_second_input_batch_dim_is_one: confirms redundant reshapes are removed when input $B$ has batch dimension 1.
    • test_reshape_matmul_reshape_remain_when_contracting_dimension_mismatches: confirms the rewrite is safely declined when contracting dimensions mismatch.

Validation

  • All 18 unit tests in _broadcast_to_matmul_test.py pass.
  • All 322 common rewriter rules tests pass (1 existing skip).
  • ruff check and ruff format --check pass with zero warnings or errors.

In _broadcast_to_matmul.py, check_if_not_need_reshape checked reversed dimensions using if dim_from_a not in {1, dim_from_b}:.

- At idx == 0 (the contracting dimension K), dim_from_a and dim_from_b must match exactly (dim_from_a == dim_from_b). Allowing dim_from_a == 1 when dim_from_b != 1 permitted rewriting graphs into invalid MatMul operations with incompatible contracting dimensions.
- At idx > 0 (the batch dimensions), two dimensions are broadcastable if dim_from_a == dim_from_b, dim_from_a == 1, or dim_from_b == 1. The previous check rejected valid broadcasting when dim_from_b == 1 and dim_from_a > 1.

This change enforces exact equality on contracting dimensions at idx == 0 and symmetric broadcasting on batch dimensions at idx > 0. Adds regression unit tests covering both cases.
@hardikkaurani

Copy link
Copy Markdown
Author

Hello G. Ramalingam (@gramalingam),

I have submitted this pull request to resolve two correctness issues in _broadcast_to_matmul.py:

  1. When contracting dimensions mismatch at idx == 0, exact equality is now required rather than allowing dim_from_a == 1, preventing invalid rewrites into incompatible MatMul operations.
  2. When batch dimensions broadcast symmetrically at idx > 0, the check now correctly accounts for dim_from_b == 1 as well as dim_from_a == 1.

All 18 unit tests in _broadcast_to_matmul_test.py and all 322 common rewrite tests pass, and ruff checks and formatting are clean.

Please review when convenient and let me know if any adjustments are desired.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant