Enable CPU HBM #2602
jikunshang
started this conversation in
Ideas
Enable CPU HBM
#2602
Replies: 1 comment
-
I have provide a draft on #2603 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Xeon Max series CPU provides 64 GB in-package high bandwidth memory (HBM), which could benefit a lot for memory bound workload like matmul. We can enable hbm feature in ggml/llama.cpp to get better performance.
reference:
Xeon Max Series Product Brief
Intel® Xeon® CPU Max Series Configuration and Tuning Guide
Beta Was this translation helpful? Give feedback.
All reactions