Behavior for validation evaluation policies.
Determines which validation examples to evaluate and how to score programs.
Summary
Callbacks
Get the index of the best program given current evaluations.
Get validation batch IDs to evaluate for a program.
Get the validation score for a specific program.
Callbacks
@callback get_best_program(GEPA.State.t()) :: non_neg_integer()
Get the index of the best program given current evaluations.
@callback get_eval_batch(GEPA.DataLoader.t(), GEPA.State.t(), non_neg_integer() | nil) :: [term()]
Get validation batch IDs to evaluate for a program.
@callback get_valset_score(non_neg_integer(), GEPA.State.t()) :: float()
Get the validation score for a specific program.