You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FileDataLoader fails to read the file when size > INT32_MAX (#4435)
Summary:
On macOS, the `read` function will fail with an `EINVAL` error if the size parameter exceeds `INT32_MAX`. This update addresses the issue by adding a check to ensure that the read size does not surpass `INT32_MAX`. On Linux, the maximum permissible read size is 2,147,479,552 bytes ( < `INT32_MAX`), so attempting to read beyond this limit is inconsequential.
Pull Request resolved: #4435
Test Plan:
Exporting llama3 with `python -m examples.models.llama2.export_llama --checkpoint examples/models/llama-2-7B/consolidated.00.pth --params examples/models/llama-2-7B/params.json --coreml --disable_dynamic_shape -kv `
Without fix
Fails with `invalid argument` error.
With fix
Succeeds.
Reviewed By: kirklandsign
Differential Revision: D60321719
Pulled By: shoumikhin
fbshipit-source-id: fca265c6c1edc628b38a5044693ec7bbe0c0b43a
0 commit comments