Skip to content

Suppress args in --help that we don't want to advertise yet #912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

Jack-Khuu
Copy link
Contributor

The help menu in torchchat is currently really verbose and has many features that are either (a) not fully vetted (b) wip or (c) ddon't provide enough value

This PR suppressed some of those args to reduce the help menu, making it more "helpful". The groupings aren't complete and the help text could be better, but this moves it in that direction

python3 torchchat.py chat --help

usage: torchchat chat [-h] [--checkpoint-path CHECKPOINT_PATH] [--prompt PROMPT] [--chat] [--gui] [--num-samples NUM_SAMPLES] [--max-new-tokens MAX_NEW_TOKENS] [--top-k TOP_K] [--temperature TEMPERATURE] [--sequential-prefill] [--dso-path DSO_PATH] [--pte-path PTE_PATH] [--output-pte-path OUTPUT_PTE_PATH] [--output-dso-path OUTPUT_DSO_PATH] [--is-chat-model]
                      [--compile] [--compile-prefill] [--dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}] [--quantize QUANTIZE] [--device {fast,cpu,cuda,mps}] [--hf-token HF_TOKEN] [--model-directory MODEL_DIRECTORY] [--profile PROFILE] [--port PORT] [-v] [--seed SEED]
                      [model]

positional arguments:
  model                 Model name for well-known models

options:
  -h, --help            show this help message and exit
  --checkpoint-path CHECKPOINT_PATH
                        Use the specified model checkpoint path
  --is-chat-model       Indicate that the model was trained to support chat functionality
  --compile             Whether to compile the model with torch.compile
  --compile-prefill     Whether to compile the prefill. Improves prefill perf, but has higher compile times.
  --dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}
                        Override the dtype of the model (default is the checkpoint dtype). Options: bf16, fp16, fp32, fast16, fast
  --quantize QUANTIZE   Quantization options. pass in as '{"<mode>" : {"<argname1>" : <argval1>, "<argname2>" : <argval2>,...},}' modes are: embedding, linear:int8, linear:int4, linear:a8w4dq, precision.
  --device {fast,cpu,cuda,mps}
                        Hardware device to use. Options: cpu, cuda, mps
  --hf-token HF_TOKEN   A HuggingFace API token to use when downloading model artifacts
  --model-directory MODEL_DIRECTORY
                        The directory to store downloaded model artifacts. Default: /Users/jackkhuu/.torchchat/model-cache
  --profile PROFILE     Profile path.
  --port PORT           Port for the web server in browser mode
  -v, --verbose         Verbose output
  --seed SEED           Initialize torch seed

Generation Args:
  Configs for generating output based on provided prompt

  --prompt PROMPT       Input prompt for manual output generation
  --chat                Whether to start an interactive chat session
  --gui                 Whether to use a web UI for an interactive chat session
  --num-samples NUM_SAMPLES
                        Number of samples
  --max-new-tokens MAX_NEW_TOKENS
                        Maximum number of new tokens
  --top-k TOP_K         Top-k for sampling
  --temperature TEMPERATURE
                        Temperature for sampling
  --sequential-prefill  Whether to perform prefill sequentially. Only used for model debug.

Exported Model Path Args:
  Specify the path of the exported model files to ingest

  --dso-path DSO_PATH   Use the specified AOT Inductor .dso model file
  --pte-path PTE_PATH   Use the specified ExecuTorch .pte model file

Export Output Path Args:
  Specify the output path for the exported model files

  --output-pte-path OUTPUT_PTE_PATH
                        Output to the specified ExecuTorch .pte model file
  --output-dso-path OUTPUT_DSO_PATH
                        Output to the specified AOT Inductor .dso model file

Copy link

pytorch-bot bot commented Jul 17, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/912

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit ee53cb6 with merge base ec0377c (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 17, 2024
@Jack-Khuu Jack-Khuu merged commit 85c4e71 into main Jul 17, 2024
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants