Skip to content

Commit 672f694

Browse files
authored
Fix: Assign SummaCZS instance to self.summac in Faithfulness metric (#795)
Previously, the SummaCZS model was instantiated but not assigned, causing `self.summac` to remain `None` and leading to runtime errors during scoring. This fix ensures the model is correctly cached in `self.summac`.
1 parent 53df859 commit 672f694

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

src/lighteval/metrics/metrics_sample.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -696,7 +696,9 @@ def compute(self, predictions: list[str], formatted_doc: Doc, **kwargs) -> dict[
696696
dict[str, float]: The faithfulness scores.
697697
"""
698698
if self.summac is None:
699-
SummaCZS(granularity="sentence", model_name="vitc", imager_load_cache=False) # , device=device)
699+
self.summac = SummaCZS(
700+
granularity="sentence", model_name="vitc", imager_load_cache=False
701+
) # , device=device)
700702
inp = formatted_doc.specific[self.input_column]
701703
prediction = predictions[0]
702704
if self.normalize_input:

0 commit comments

Comments
 (0)