Search results

[BUG] <code>test</code> split forced to hit <code>Careful</code> warning

bugSomething isn't workingSomething isn't working

Status: Open.

#801 In huggingface/lighteval;

· jamesbraza opened on Jun 7, 2025

[BUG] Forced to hit <code>You cannot select the number of dataset splits</code> with <code>litellm</code>

bugSomething isn't workingSomething isn't working

Status: Open.

#800 In huggingface/lighteval;

· jamesbraza opened on Jun 7, 2025

[FT] integrate typo checker to check typos like <code>refenrence</code>

Status: Open.

#799 In huggingface/lighteval;

· jamesbraza opened on Jun 7, 2025

[FT] supporting train-time vs test-time metrics

Status: Open.

#798 In huggingface/lighteval;

· jamesbraza opened on Jun 7, 2025

[FT] more docstrings and typing in <code>Doc</code>

Status: Open.

#797 In huggingface/lighteval;

· jamesbraza opened on Jun 7, 2025

[FT] <code>StrEnum</code> for <code>suites</code> to intuitively document options

Status: Open.

#796 In huggingface/lighteval;

· jamesbraza opened on Jun 6, 2025

[BUG] Faithfulness metric fails because the SummaCZS model is instantiated but never assigned

bugSomething isn't workingSomething isn't working

Status: Open.

#794 In huggingface/lighteval;

· sahilds1 opened on Jun 5, 2025

[BUG] support direct evaluation of local API and local data sets?

bugSomething isn't workingSomething isn't working

Status: Open.

#793 In huggingface/lighteval;

· HemiFate opened on Jun 5, 2025

[FT] Store system prompt in results

Status: Open.

#788 In huggingface/lighteval;

· lewtun opened on Jun 3, 2025

[BUG] OpenAIClient fails when using newer GPT models

bugSomething isn't workingSomething isn't working

Status: Open.

#785 In huggingface/lighteval;

· ryantzr1 opened on May 27, 2025

[BUG] Python API documentation

bugSomething isn't workingSomething isn't working

Status: Open.

#783 In huggingface/lighteval;

· ryantzr1 opened on May 27, 2025

[BUG] Is AIME24 broken?

bugSomething isn't workingSomething isn't working

Status: Open.

#771 In huggingface/lighteval;

· eldarkurtic opened on May 22, 2025