Skip to content

Add torchtune convertor and README changes #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2024

Conversation

kartikayk
Copy link
Contributor

torchtune supports HF and Meta formatted checkpoints. In this PR I'm add a custom convertor for torchtune. Currently this only support llama2 7B and llama3 8B. I also update the README with the necessary steps.

Test:

Llama3 Model:

python3 build/convert_torchtune_checkpoint.py \
--checkpoint-dir /data/users/kartikayk/cpts/Meta-Llama-3-8B/ \
--checkpoint-files meta_model_0.pt \
--model-name llama3_8B \
--checkpoint-format meta

python3 torchchat.py generate \
--checkpoint-path /data/users/kartikayk/cpts/Meta-Llama-3-8B/model.pth \
--device cuda

Output:
image

HF Format Llama2 7B

python3 build/convert_torchtune_checkpoint.py \
--checkpoint-dir /tmp/Llama-2-7b-hf/ \
--checkpoint-files hf_model_0001_0.pt hf_model_0002_0.pt \
--model-name llama2_7B \
--checkpoint-format hf

python3 torchchat.py generate --checkpoint-path /tmp/Llama-2-7b-hf/model.pth  --device 'cuda'

Output:
image

Kartikay Khandelwal added 2 commits April 23, 2024 19:29
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 24, 2024
@mikekgfb
Copy link
Contributor

Thank you!

@mikekgfb mikekgfb merged commit 9452867 into pytorch:main Apr 24, 2024
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
malfet pushed a commit that referenced this pull request Jul 17, 2024
* Add torchtune convertor

* Update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants