Skip to main content
Back to top
Ctrl
+
K
Choose version
Search
Ctrl
+
K
Getting Started
Home
Quickstart
Installation
Core Concepts
Tutorials
User Guides
Training
Training Methods
Training Environments
Local Training
VSCode Integration
Notebook Integration
Training Configuration
Monitoring & Debugging
Inference
Inference Engines
Common Workflows
Inference Configuration
Inference CLI
Evaluation
Evaluation Configuration
Standardized Benchmarks
Generative Benchmarks
Leaderboards
Custom Evaluations
LLM Judge
Built-In Judge
Custom Prompts
Custom Model
Running Jobs on Clusters
Deploying a Job
Custom Clusters
Customizing Oumi
Resources
Models
Supported Models
Custom Models
Datasets
Chat Formats
Supervised Fine-Tuning
Pre-training
Preference Tuning
Vision-Language Datasets
Other Datasets
Recipes
Reference
API Reference
oumi.builders
oumi.cli
oumi.core
oumi.core.callbacks
oumi.core.collators
oumi.core.configs
oumi.core.configs.internal
oumi.core.configs.params
oumi.core.datasets
oumi.core.evaluation
oumi.core.evaluation.backends
oumi.core.evaluation.utils
oumi.core.feature_generators
oumi.core.inference
oumi.core.launcher
oumi.core.models
oumi.core.processors
oumi.core.registry
oumi.core.tokenizers
oumi.core.trainers
oumi.core.types
oumi.datasets
oumi.datasets.evaluation
oumi.datasets.grpo
oumi.datasets.grpo.rewards
oumi.datasets.preference_tuning
oumi.datasets.pretraining
oumi.datasets.sft
oumi.datasets.vision_language
oumi.inference
oumi.judges
oumi.launcher
oumi.launcher.clients
oumi.launcher.clouds
oumi.launcher.clusters
oumi.models
oumi.models.layers
oumi.performance
oumi.utils
CLI Reference
FAQ
Troubleshooting
Out of Memory (OOM)
Development
Dev Environment Setup
Contributing
Code of Conduct
Style Guide
Updating the Documentation
About
Changelog
Acknowledgments
License
Citations
Repository
Show source
Suggest edit
Open issue
.rst
.pdf
oumi.evaluation
Contents
Subpackages
oumi.evaluation
#
Subpackages
#
oumi.evaluation.registry
berry_bench()
count_letters()
Contents
Subpackages
