oumi

oumi#

Oumi (Open Universal Machine Intelligence) library.

This library provides tools and utilities for training, evaluating, and inferring with machine learning models, particularly focused on language tasks.

Modules:

  • models: Contains model architectures and related utilities.

  • evaluate: Functions for evaluating models.

  • evaluate_async: Asynchronous evaluation functionality.

  • infer: Functions for model inference, including interactive mode.

  • train: Training utilities for machine learning models.

  • utils: Utility functions, including logging configuration.

  • judges: Functions for judging datasets and conversations.

Functions:

  • train(): Train a machine learning model.

  • evaluate_async(): Asynchronously evaluate a model.

  • evaluate(): Evaluate a model using LM Harness.

  • infer(): Perform inference with a trained model.

  • infer_interactive(): Run interactive inference with a model.

  • quantize(): Quantize a model to reduce size and memory usage.

  • judge_dataset(): Judge a dataset using a model.

Examples

Training a model:

>>> from oumi import train
>>> from oumi.core.configs import TrainingConfig
>>> config = TrainingConfig(...)
>>> train(config)

Evaluating a model:

>>> from oumi import evaluate
>>> from oumi.core.configs import EvaluationConfig
>>> config = EvaluationConfig(...)
>>> results = evaluate(config)

Performing inference:

>>> from oumi import infer
>>> from oumi.core.configs import InferenceConfig
>>> config = InferenceConfig(...)
>>> outputs = infer(config)

Quantizing a model:

>>> from oumi import quantize
>>> from oumi.core.configs import QuantizationConfig
>>> config = QuantizationConfig(...)
>>> result = quantize(config)
Judging a dataset::
>>> from oumi import judge_dataset
>>> from oumi.core.configs import JudgeConfig
>>> config = JudgeConfig(...)
>>> judge_dataset(config, dataset)

See also

oumi.evaluate(config: EvaluationConfig) → list[dict[str, Any]][source]#

Evaluates a model using the provided configuration.

Parameters:

config – The desired configuration for evaluation.

Returns:

A list of evaluation results (one for each task). Each evaluation result is a dictionary of metric names and their corresponding values.

oumi.evaluate_async(config: AsyncEvaluationConfig) → None[source]#

Runs an async evaluation for a model using the provided configuration.

Overview:

This is a utility method for running evaluations iteratively over a series of checkpoints. This method can be run in parallel with a training job to compute metrics per checkpoint without wasting valuable time in the main training loop.

Parameters:

config – The desired configuration for evaluation.

Returns:

None.

oumi.infer(config: InferenceConfig, inputs: list[str] | None = None, inference_engine: BaseInferenceEngine | None = None, *, input_image_bytes: list[bytes] | None = None) → list[Conversation][source]#

Runs batch inference for a model using the provided configuration.

Parameters:

  • config – The configuration to use for inference.

  • inputs – A list of inputs for inference.

  • inference_engine – The engine to use for inference. If unspecified, the engine will be inferred from config.

  • input_image_bytes – A list of input PNG image bytes to be used with image+text VLMs. Only used in interactive mode.

Returns:

A list of model responses.

Return type:

object

oumi.infer_interactive(config: InferenceConfig, *, input_image_bytes: list[bytes] | None = None, system_prompt: str | None = None) → None[source]#

Interactively provide the model response for a user-provided input.

oumi.quantize(config: QuantizationConfig) → QuantizationResult[source]#

Quantizes a model using the provided configuration.

Parameters:

config – Quantization configuration containing model parameters, method, output path, and other settings.

Returns:

  • quantized_size_bytes: Size of the quantized model in bytes

  • output_path: Path to the quantized model

  • quantization_method: Quantization method used

  • format_type: Format type of the quantized model

  • additional_info: Additional method-specific information

Return type:

QuantizationResult containing

Raises:

  • RuntimeError – If quantization fails for any reason

  • ValueError – If configuration is invalid for this quantizer

oumi.synthesize(config: SynthesisConfig) → list[dict[str, Any]][source]#

Synthesize a dataset using the provided configuration.

oumi.train(config: TrainingConfig, additional_model_kwargs: dict[str, Any] | None = None, additional_trainer_kwargs: dict[str, Any] | None = None, verbose: bool = False) → None[source]#

Trains a model using the provided configuration.

Subpackages#

Contents