oumi.evaluation.registry#

Evaluation registry module.

oumi.evaluation.registry.berry_bench(task_params: EvaluationTaskParams, inference_engine: BaseInferenceEngine) → dict[str, Any][source]#

Custom evaluation function registered as berry_bench.

oumi.evaluation.registry.count_letters(task_params: EvaluationTaskParams, inference_engine: BaseInferenceEngine) → dict[str, Any][source]#

Custom evaluation function registered as count_letters.