Skip to content

Commit d59c077

Browse files
authored
simplify the messaging for browser and API. Make sure that skip default doesn't show up to users (#923)
1 parent ddb5820 commit d59c077

File tree

1 file changed

+12
-14
lines changed

1 file changed

+12
-14
lines changed

README.md

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -121,19 +121,22 @@ python3 torchchat.py generate llama3 --prompt "write me a story about a boy and
121121

122122

123123
### Server
124-
Thie mode kicks off a server to send curl requests against.
124+
**note this feature is still in progress and not all endpoints are working ATM**
125+
Server gives you a REST API that matches the OpenAI API spec for interacting with a model
125126

126127
[skip default]: begin
128+
127129
```bash
128130
python3 torchchat.py server llama3
129131
```
130132
[skip default]: end
131133

132-
<details>
133134

134-
> [!NOTE]
135-
> Depending on the model configuration, this query might take a few minutes
136-
to respond
135+
Open another terminal window to interact with the API using curl
136+
137+
138+
**Note: Depending on the model configuration, this query might take a few minutes
139+
to respond**
137140

138141
<summary>Sample Input + Output</summary>
139142

@@ -157,22 +160,17 @@ curl http://127.0.0.1:5000/chat \
157160
{"response":" I'm a software developer with a passion for building innovative and user-friendly applications. I have experience in developing web and mobile applications using various technologies such as Java, Python, and JavaScript. I'm always looking for new challenges and opportunities to learn and grow as a developer.\n\nIn my free time, I enjoy reading books on computer science and programming, as well as experimenting with new technologies and techniques. I'm also interested in machine learning and artificial intelligence, and I'm always looking for ways to apply these concepts to real-world problems.\n\nI'm excited to be a part of the developer community and to have the opportunity to share my knowledge and experience with others. I'm always happy to help with any questions or problems you may have, and I'm looking forward to learning from you as well.\n\nThank you for visiting my profile! I hope you find my information helpful and interesting. If you have any questions or would like to discuss any topics, please feel free to reach out to me. I"}
158161
```
159162

160-
</details>
161163

162164
### Browser
163-
This mode provides access to a localhost browser hosting [Streamlit](https://streamlit.io/).
164-
Running the command automatically open a tab in your browser.
165-
```
166-
streamlit run torchchat.py -- browser <model_name> <model_args>
167-
```
165+
This mode allows you to chat with the model using a UI in your browser
168166

169-
For example, to quantize and chat with LLaMA3:
170167

168+
Running the command automatically open a tab in your browser.
171169
[skip default]: begin
170+
172171
```
173-
streamlit run torchchat.py -- browser llama3 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --max-new-tokens 256 --compile
172+
streamlit run torchchat.py -- browser llama3
174173
```
175-
[skip default]: end
176174

177175

178176
> [!TIP]

0 commit comments

Comments
 (0)