Skip to content

[feat] quantized GEMM support in the cuBLASLt and hipBLASLt backends - #15

Draft
harz05 wants to merge 3 commits into
devfrom
quantized-gemm
Draft

harz05 wants to merge 3 commits into
devfrom
quantized-gemm

Conversation

@harz05

@harz05 harz05 commented Sep 21, 2026

Copy link
Copy Markdown
Member

Summary (WIP):

Exposes through sofieBLAS the GEMM configurations that quantized inference needs, so SOFIE's quantized path can go through sofieBLAS

  • Layouts, descriptors and algorithms are keyed by element, compute and scale type instead of being fixed to float. The float API and its behaviour are unchanged.
  • matmul overload for int8 x int8 -> int32 accumulation (integer alpha/beta, optional strided batch).
  • addOperationConfig overload that reports whether the library has a kernel for a typed configuration instead of ending the process, for callers with a fallback.
  • Epilogue::Relu and DataType { F32, I8, I32 } enum in core.hpp; the matching rows in both Api tables.
  • runGpuInt8Tests: NN, TN and batched int8 matmul with exact int32 comparison, for both tags.

@@ -389,30 +493,26 @@ private:
return ins->second.h;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return ins->second.h;
return &ins->second.h;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant