Refactor generate.py for modularity (#906)

[dbort]

This is a cherry-pick of @vmpuri's #906

This PR refactors the chat function in generate.py by creating a Generator class, removing unnecessary global variables and simplifying the code structure. Tokens and metrics are now yielded by the Generator rather than being printed directly to stdout, making it easier to re-use this code for non-CLI tools.

Tests:
Generate

python3 torchchat.py generate stories15M --prompt "Once upon a time,"

Using device=mps
Loading model...
Time to load model: 0.46 seconds
-----------------------------------------------------------
Once upon a time, there was a little girl named Lily. One day, she went to the park with her mom. They saw a big tree with lots of pears on it. Lily wanted to eat a pear, but they were too high up. She tried to jump, but she couldn't reach them.
Then, a boy came and took a pear from the tree. Lily was surprised! She thought the boy would be mean, but he was harmless and didn't mean to hurt her. She asked him if she could have a pear too. The boy said yes and gave her a pear. Lily was very happy and thanked the boy for sharing. They sat under the tree and ate the pear together. It was a good day at the park. Once upon a time, there was a little girl named Lily. She loved to play in the garden with her mommy. One day, they were planting some
Time for inference 1: 3.55 sec total, time to first token 0.00 sec with parallel prefill, 199 tokens, 56.01 tokens/sec, 17.85 ms/token
Bandwidth achieved: 2.73 GB/s
*** This first iteration will include cold start effects for dynamic import, hardware caches. ***

========================================

Average tokens/sec: 56.01

========================================

Average tokens/sec: 119.35
Memory used: 0.00 GB

Eval

python3 torchchat.py eval stories15M --tasks wikitext --limit 10

NumExpr defaulting to 10 threads.
PyTorch version 2.5.0.dev20240629 available.
Using device=mps
Loading model...
Time to load model: 0.38 seconds
-----------------------------------------------------------
Using device 'mps'
[Task: wikitext] metric word_perplexity is defined, but aggregation is not. using default aggregation=weighted_perplexity
[Task: wikitext] metric word_perplexity is defined, but higher_is_better is not. using default higher_is_better=False
[Task: wikitext] metric byte_perplexity is defined, but aggregation is not. using default aggregation=weighted_perplexity
[Task: wikitext] metric byte_perplexity is defined, but higher_is_better is not. using default higher_is_better=False
[Task: wikitext] metric bits_per_byte is defined, but aggregation is not. using default aggregation=bits_per_byte
[Task: wikitext] metric bits_per_byte is defined, but higher_is_better is not. using default higher_is_better=False
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Building contexts for wikitext on rank 0...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 541.59it/s]
Running loglikelihood_rolling requests
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:04<00:00,  2.25it/s]
Time to run eval: 22.57s.
Time in model.forward: 1.01s, over 33 model evaluations
forward run time stats - Median: 0.02s Min: 0.01s Max: 0.27s
For model /Users/puri/.torchchat/model-cache/stories15M/stories15M.pt
wikitext:
 word_perplexity,none: 47350.8811
 byte_perplexity,none: 7.7811
 bits_per_byte,none: 2.9600
 alias: wikitext

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/918

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 4a275c5 with merge base e1fb003 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[dbort]

This is a cherry-pick of @vmpuri's #906