You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary:
Loaded weights were not updating in ET because the returned weights were clones, not the original weights.
Pull Request resolved: #6039
Test Plan:
```
> buck2 run fbcode//executorch/examples/llm_pte_finetuning:runner -- --cfg=fbcode/executorch/examples/llm_pte_finetuning/phi3_config.yaml --model_file=phi3_mini_lora.pte
Evaluating the model before training...
100%|██████████████████████████████████████████████████████████████████████████████████████| 3/3 [31:23<00:00, 627.98s/it]
Eval loss: tensor(2.3778)
grad_start:`1`
param_start:`129`
100%|██████████████████████████████████████████████████████████████████████████████████████| 5/5 [52:29<00:00, 629.84s/it]
Losses: [2.7152762413024902, 0.7890686988830566, 2.249271869659424, 1.4777560234069824, 0.8378427624702454]
100%|██████████████████████████████████████████████████████████████████████████████████████| 3/3 [30:35<00:00, 611.90s/it]
Eval loss: tensor(0.8464)
```
Reviewed By: dpalmasan
Differential Revision: D64084552
Pulled By: dvorjackz
fbshipit-source-id: 9d478dda02f7bcaa5964d83d257d0db5bfe9feab
0 commit comments