Support bf16 delegation for fully-connected (#21493) - #21493
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21493
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit a911cec with merge base d632341 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
|
@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D114220918. |
This PR needs a
|
Summary: A reland of D113977357/pytorch#21409 Adds XNNPACK delegation support for bf16 fully-connected, including bf16 dynamic-quant (8da4w). This lets bf16 models (e.g. google/gemma-3-1b-it exported with --dtype bfloat16) lower their linear layers to XNNPACK instead of falling back to portable. Key addition for the reland: an enable_bf16 flag in the xnnpack_config, which is default false. This fixes the internal failures, where the XNNPACK pin is too old. Differential Revision: D114220918
meta-codesync
Bot
changed the title
f60b334 to
a911cec
Compare
digantdesai
left a comment
digantdesai
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
2 participants
Summary:
A reland of D113977357/#21409
Adds XNNPACK delegation support for bf16 fully-connected, including bf16 dynamic-quant (8da4w). This lets bf16 models (e.g. google/gemma-3-1b-it exported with --dtype bfloat16) lower their linear layers to XNNPACK instead of falling back to portable.
Key addition for the reland: an enable_bf16 flag in the xnnpack_config, which is default false. This fixes the internal failures, where the XNNPACK pin is too old.
Differential Revision: D114220918