Skip to content

Commit d4cc70d

Browse files
committed
Update on "add instructions about getting mmlu score for instruct models"
Differential Revision: [D64254491](https://our.internmc.facebook.com/intern/diff/D64254491) [ghstack-poisoned]
2 parents 555440b + 9ba850a commit d4cc70d

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

examples/models/llama2/eval_llama_lib.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -295,9 +295,9 @@ def eval_llama(
295295
with torch.no_grad():
296296
eval_results = simple_evaluate(
297297
model=eval_wrapper,
298-
tasks=args.tasks,
299-
num_fewshot=args.num_fewshot,
300-
limit=args.limit,
298+
tasks=args.tasks, # pyre-ignore: Undefined attribute [16]: `argparse.ArgumentParser` has no attribute `tasks`
299+
num_fewshot=args.num_fewshot, # pyre-ignore: Undefined attribute [16]: `argparse.ArgumentParser` has no attribute `num_fewshot`
300+
limit=args.limit, # pyre-ignore: Undefined attribute [16]: `argparse.ArgumentParser` has no attribute `limit`
301301
)
302302

303303
for task, res in eval_results["results"].items():

0 commit comments

Comments
 (0)