set buffer size to 8192 as default, decode precision as a string, lint #476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

mikekgfb merged 5 commits into main from buffer_size_dtype_setting

Apr 25, 2024

Contributor

mikekgfb commented Apr 25, 2024

set buffer size to 8192 as default (new model arg)
decode precision as a string
lint


          set buffer size to 8192 as default, decode precision as a string, lint

b942e31

facebook-github-bot added the CLA Signed label

mikekgfb requested review from JacobSzwejbka, byjlw and jerryzh168

April 25, 2024 00:35

jerryzh168 reviewed

View reviewed changes

build/builder.py Outdated Show resolved Hide resolved

jerryzh168 reviewed

View reviewed changes

build/builder.py Outdated Show resolved Hide resolved

Michael Gschwind added 4 commits

April 24, 2024 23:40


          typo

6d0dd5b


          merge

b45cec9


          typo

ec80baa


          typo

0bd2c3b

kirklandsign approved these changes

View reviewed changes

malfet approved these changes

View reviewed changes

Contributor

malfet left a comment

Can buffer size change has negative effects on the perf?

mikekgfb merged commit d5bd046 into main

mikekgfb deleted the buffer_size_dtype_setting branch

April 25, 2024 17:43

malfet added a commit that referenced this pull request


          Fix browser init

5dfe3e9

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages
Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: Add CI that repeats above steps

Fixes #785

malfet mentioned this pull request

Fix browser init #797

Merged

malfet added a commit that referenced this pull request


          Fix browser init (#797)

570ff04

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

68018e9

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

2ba99a7

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

2a4a5d7

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

f4cc6dc

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

81be073

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

a6c4298

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

17a0222

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

121176d

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

81e9a9c

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

b21f12d

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

malfet pushed a commit that referenced this pull request


          set buffer size to 8192 as default, decode precision as a string, lint (

c750f24

#476)

* set buffer size to 8192 as default, decode precision as a string, lint

* typo

* typo

* typo

malfet added a commit that referenced this pull request


          Fix browser init (#797)

bc8126e

Update prompt chat is waiting for, which was modified by #476

Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window

Test plan:
```
% python3 torchchat.py browser stories110M &
% curl -L http://127.0.0.1:5000
% curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat
```

TODOs: 
- Add CI that repeats above steps
-Figure out if spawning generator from the browser can be avoided

Fixes #785

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

jerryzh168 jerryzh168 left review comments

malfet malfet approved these changes

kirklandsign kirklandsign approved these changes

JacobSzwejbka Awaiting requested review from JacobSzwejbka

byjlw Awaiting requested review from byjlw

Labels