Skip to content

update quant docs #425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 24, 2024
Merged

update quant docs #425

merged 1 commit into from
Apr 24, 2024

Conversation

lucylq
Copy link
Contributor

@lucylq lucylq commented Apr 23, 2024 •
edited
Loading

update quant docs, json properties must be in double quotes, groupsize 7 doesn't work (modulo 4096)

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 23, 2024
@lucylq lucylq force-pushed the lfq.download-create-dirs branch 4 times, most recently from 587c8a1 to 303bd9d Compare April 23, 2024 22:11
@lucylq lucylq marked this pull request as ready for review April 23, 2024 22:16
mikekgfb
mikekgfb suggested changes Apr 23, 2024
View reviewed changes
Copy link
Contributor

@mikekgfb mikekgfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

README.md Outdated
@@ -63,7 +63,6 @@ python3 torchchat.py download llama3
* in Chat mode
* in Generate mode
* [Exporting for mobile via ExecuTorch](#export-executorch)
* in Chat mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does that mean? This is a function of the token stream received from the driver. @JacobSzwejbka has been overhauling the chat mode, please work through the application scenario with @JacobSzwejbka to ensure it works. If it does not, it's a bug. Ditto for aoti-generated models.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmn, are we supporting chat mode for exported executorch models as well? I think @JacobSzwejbka's changes are for eager (in generate.py).

Copy link
Contributor

@JacobSzwejbka JacobSzwejbka Apr 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does that mean? This is a function of the token stream received from the driver. @JacobSzwejbka has been overhauling the chat mode, please work through the application scenario with @JacobSzwejbka to ensure it works. If it does not, it's a bug. Ditto for aoti-generated models.

there is a 0% chance I will have time to make chat mode work for anything but eager by the deadline. If that feature is truly required someone else needs to be working on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to dig into this. Please remove this change and I'll approve to land the rest

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry - missed this comment. Updated

mikekgfb
mikekgfb reviewed Apr 23, 2024
View reviewed changes
Copy link
Contributor

@mikekgfb mikekgfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

README.md Outdated
@@ -63,7 +63,6 @@ python3 torchchat.py download llama3
* in Chat mode
* in Generate mode
* [Exporting for mobile via ExecuTorch](#export-executorch)
* in Chat mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to dig into this. Please remove this change and I'll approve to land the rest

@lucylq lucylq force-pushed the lfq.download-create-dirs branch from 303bd9d to f3cfd7d Compare April 24, 2024 03:13
mikekgfb
mikekgfb approved these changes Apr 24, 2024
View reviewed changes
Copy link
Contributor

@mikekgfb mikekgfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@lucylq lucylq force-pushed the lfq.download-create-dirs branch 2 times, most recently from 86b50f6 to b83db35 Compare April 24, 2024 04:17
@lucylq lucylq force-pushed the lfq.download-create-dirs branch from b83db35 to 134f73e Compare April 24, 2024 04:30
@lucylq lucylq changed the title Create dir on download, readme update quant docs Apr 24, 2024
@lucylq lucylq merged commit 25a105f into main Apr 24, 2024
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
b85fe46
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
f4c5191
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
920a622
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
46eb900
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
6b8912e
malfet pushed a commit that referenced this pull request Jul 17, 2024
@lucylq @malfet
create dir on download (#425)
d6da1b4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mikekgfb mikekgfb mikekgfb approved these changes

@JacobSzwejbka JacobSzwejbka Awaiting requested review from JacobSzwejbka

@larryliu0820 larryliu0820 Awaiting requested review from larryliu0820

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lucylq @JacobSzwejbka @mikekgfb @facebook-github-bot