[AutoDeploy] Merge Feature Branch Week 3 #5054
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* add an arg to load checkpoint on cpu Signed-off-by: Frida Hou <[email protected]> * fix pre-commit Signed-off-by: Frida Hou <[email protected]> * remove ckpt-device from factory Signed-off-by: Frida Hou <[email protected]> --------- Signed-off-by: Frida Hou <[email protected]>
* example of inductor pattern matcher for RoPE with explicit cos/sin matcher Signed-off-by: Frida Hou <[email protected]> * move to utils Signed-off-by: Frida Hou <[email protected]> * add usage of scalar_workaround, support op_ignore_type Signed-off-by: Ubuntu <[email protected]> * minor Signed-off-by: Ubuntu <[email protected]> * update all 3 types of RoPE matcher to use inductor pattern matcher Signed-off-by: Frida Hou <[email protected]> * address feedback and refine code/doc Signed-off-by: Frida Hou <[email protected]> * minor Signed-off-by: Ubuntu <[email protected]> * fix 2e2 for llama4 and ds rope, remove legalize_graph in canonicalize_graph, update ds rope impl to match with the exported graph Signed-off-by: Frida Hou <[email protected]> * deprecate previous rope matcher Signed-off-by: Ubuntu <[email protected]> --------- Signed-off-by: Frida Hou <[email protected]> Signed-off-by: Ubuntu <[email protected]>
lucaslie
requested review from
Copilot and
suyoggupta
and removed request for
suyoggupta
There was a problem hiding this comment.
Pull Request Overview
This PR merges the Week 3 auto-deploy feature branch, unifying RoPE pattern handling, extending the test harness, and adding a checkpoint_device option across user APIs.
- Replace separate
match_explicit_rope/match_complex_ropecalls with a singlematch_rope_patternthat returns match counts - Extend
run_testto accept and verifycheck_num_matchesand update all tests accordingly - Introduce
checkpoint_devicein CLI args, transformation pipeline, example configs, and build scripts
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_rope_transformation.py | Switch to match_rope_pattern, consolidate test branches, and pass expected match counts |
| tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_quantization.py | Add None placeholders for new dynamic_shapes and check_num_matches parameters |
| tests/unittest/_torch/auto_deploy/_utils_test/_model_test_utils.py | Split shared freqs into freqs_q/freqs_k for clarity in apply_rotary_pos_emb_complex |
| tests/unittest/_torch/auto_deploy/_utils_test/_graph_test_helpers.py | Extend run_test signature with check_num_matches and branch on it |
| tensorrt_llm/llmapi/llm_args.py | Add optional checkpoint_device argument to LLM CLI args |
| tensorrt_llm/_torch/auto_deploy/utils/pattern_matcher.py | New FX‐pattern‐matcher utilities for registering and applying inductive patterns |
| tensorrt_llm/_torch/auto_deploy/transformations/transform.py | Replace legacy RoPE match calls with match_rope_pattern, respect checkpoint_device on load |
| tensorrt_llm/_torch/auto_deploy/transformations/_graph.py | Remove obsolete legalize_graph invocation during GraphModule cleanup |
| examples/auto_deploy/simple_config.py | Introduce checkpoint_device field in example config |
| examples/auto_deploy/build_and_run_ad.py | Forward checkpoint_device from config into the deployment builder |
Comments suppressed due to low confidence (2)
tests/unittest/_torch/auto_deploy/_utils_test/_graph_test_helpers.py:35
- The docstring for
run_testdoesn’t mention the newcheck_num_matchesparameter—please update the header comment to explain this argument.
def run_test(
examples/auto_deploy/simple_config.py:29
Optionalis not imported in this file; addfrom typing import Optionalto avoid aNameError.
checkpoint_device: Optional[str] = None # Device on which to load the model checkpoint
|
/bot run --disable-fail-fast |
suyoggupta
approved these changes
Jun 9, 2025
|
PR_Github #8165 [ run ] triggered by Bot |
|
PR_Github #8165 [ run ] completed with state |
|
/bot skip --comment "all relevant tests passed in pipeline #8165" |
|
PR_Github #8328 [ skip ] triggered by Bot |
|
PR_Github #8328 [ skip ] completed with state |
yunruis
pushed a commit
to yunruis/TensorRT-LLM
that referenced
this pull request
Jun 12, 2025
Signed-off-by: Frida Hou <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merging week feature branch. Take a look at individual PRs and copilot summary for more info