Skip to content

[BUGFIX] EEXIST: file already exists, symlink when file already downloaded #1253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

frascuchon
Copy link
Contributor

Using the library in Linux environments, I have the following errors when using cached files:

Error: EEXIST: file already exists, symlink '/home/node/.cache/huggingface/hub/datasets--facebook--natural_reasoning/blobs/704a73a635633c697507c74576e30798d42a3455967ef4d5bd0865b5242ee7fe' -> '/home/node/.cache/huggingface/hub/datasets--facebook--natural_reasoning/snapshots/bf8da7626caecdea87df7b5a8d97bf9650d909a3/full.jsonl'

This happens when trying to recreate the symbolic link. This PR checks if the symlink already exists and returns the pointer file.

@frascuchon frascuchon requested a review from coyotte508 as a code owner March 6, 2025 09:19
Copy link
Member

@coyotte508 coyotte508 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @axel7083 if you want to take a look

@@ -100,6 +100,9 @@ export async function downloadFileToCacheDir(
await mkdir(dirname(blobPath), { recursive: true });
await mkdir(dirname(pointerPath), { recursive: true });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to check for exists before mkdir the parent directory then, so we would avoid calling mkdir

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@axel7083 axel7083 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@coyotte508
Copy link
Member

Feel free to merge @frascuchon !

@frascuchon
Copy link
Contributor Author

Feel free to merge @frascuchon !

I would love to do it, but I think I don't have the right permissions 😢

@coyotte508 coyotte508 merged commit 5f8e155 into huggingface:main Mar 6, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants