build: DIS-148 use the tensorrt_llm public wheel from pypi by default in container build #1525
Conversation
richardhuo-nv
requested review from
GuanLuo,
rmccorm4,
tanmayv25,
ptarasiewiczNV,
ishandhanani,
alec-flowers,
nnshah1 and
a team
as code owners
WalkthroughA new command-line flag, Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant BuildScript
participant TensorRTLLM
User->>BuildScript: Run with --use-default-experimental-tensorrtllm-commit
BuildScript->>BuildScript: Validate no conflicting options
BuildScript->>BuildScript: Set TRTLLM_COMMIT to default experimental commit
BuildScript->>TensorRTLLM: Build with experimental commit
Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
container/build.sh (3)
91-93: Variable naming consistency
The variableDEFAULT_EXPERIMENTAL_TRTLLM_COMMIT(missing “ensor”) does not align with the full framework name in the CLI flag (--use-default-experimental-tensorrtllm-commit). Consider renaming toDEFAULT_EXPERIMENTAL_TENSORRTLLM_COMMITandUSE_DEFAULT_EXPERIMENTAL_TENSORRTLLM_COMMITfor clarity.
159-165: Error message typo in flag name
The block for--use-default-experimental-tensorrtllm-commitoutputs errors referring to--use-default-experimental-trtllm-commit(missing “ensor”). Update these echoes to match the full flag.
482-488: Mutual exclusivity error message
The mutual-exclusion check echoes--use-default-experimental-trtllm-commitagain. It should reference--use-default-experimental-tensorrtllm-committo stay consistent.examples/tensorrt_llm/README.md (1)
143-147: Standardize example label formatting
The MTP note uses lowercase "ex:", whereas other sections use "Example:". For consistency, switch to "Example:".
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
container/build.sh(4 hunks)examples/tensorrt_llm/README.md(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Build and Test - vllm
🔇 Additional comments (4)
container/build.sh (2)
94-97: Default pip wheel version verification
You've setDEFAULT_TENSORRTLLM_PIP_WHEEL="tensorrt_llm==0.21.0rc1". Ensure that using the RC as the default is intentional and stable; update to a GA release if required.
490-494: Default pip wheel fallback logic looks correct
The fallback to assignTENSORRTLLM_PIP_WHEEL="$DEFAULT_TENSORRTLLM_PIP_WHEEL"when neither wheel nor commit is set aligns with the PR’s goal to default to PyPI.examples/tensorrt_llm/README.md (2)
65-70: Experimental build example is clear
This snippet accurately demonstrates how to use the new flag for experimental commits.
287-289: MTP note is consistent here
This section correctly reiterates that MTP requires the experimental commit flag.
richardhuo-nv
changed the title
richardhuo-nv
changed the title
Overview:
change the build.sh to use the tensorrt_llm public release version from pypi by default. Since we don't own the CI pipeline of tensorrt llm, guiding our user to use a random commit from Tensorrt-LLM might cause sadness.
Details:
./container/build.sh --framework tensorrtllm, it will install the default tensorrt llm public pip wheel from the pypi--use-default-experimental-tensorrtllm-commitarg to the build.sh to pick up the tensorrt llm build we are currently experimenting.--tensorrtllm-committo specify any commit.build pipeline with the public tensorrt llm pip wheel:
https://gitlab-master.nvidia.com/dl/ai-dynamo/dynamo-ci/-/pipelines/30052494
Unit tests on tensorrt llm examples:
Test with the DS R1 multi node example in ptyche:
Perf benchmark pareto graph:
https://gitlab-master.nvidia.com/dl/ai-dynamo/dynamo-ci/-/jobs/178323320/artifacts/file/benchmark_results/deepseek-ai/DeepSeek-R1-FP4/pareto_plot.png
Where should the reviewer start?
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
Summary by CodeRabbit