Creating an initial Quantization Directory #863

Jack-Khuu · 2024-06-14T19:01:32Z

For MVP we want to refactor code out of the top level directory. This adds an initial "quantization" folder and moves some of the files.

Specifically:

qops.py
quantize.py

pytorch-bot · 2024-06-14T19:01:35Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/863

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 07ae5c1 with merge base 3ab6cee ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets

llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema

* Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)

* Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)

* Removing all references to HQQ * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Creating an initial Quantization Directory (#863) * Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)

* Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)

* Removing all references to HQQ * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Creating an initial Quantization Directory (#863) * Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)

* Removing GPTQ from all of torchchat * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Rebase + Add back accidental deletion * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Creating an initial Quantization Directory (#863) * Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Removing all references to HQQ (#869) * Removing all references to HQQ * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Creating an initial Quantization Directory (#863) * Initial Creation of a quantization directory * Moving qops * updating import * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867) * Update Quant call using llama.cpp (#868) llama.cpp did a BC breaking refactor: ggml-org/llama.cpp@1c641e6 resulting in some of our CI breaking This updates our CI to match llama.cpp's schema * Updating torch nightly to pick up aoti improvements in 128339 (#862) * Updating torch nightly to pick up aoti improvements in 128339 * Update the torch version to 2.5 * Updating lm_eval version (#865) Fixing CI related to EleutherAI/wikitext_document_level change requirements from using HF Datasets * Pinning numpy to under 2.0 (#867)