You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary:
We've seen a number of people hit issues with mlock as the default load mode. In particular, mlock will fail above a certain PTE size and may require elevated permissions on some systems. We should not default to using mlock and can leave at as an opt-in optimization.
We could use file or mmap mode. I'm leaning towards mmap, as it can lead to lower peak resident memory size, which can be significant for larger models.
Differential Revision: D74607072
0 commit comments