Can a marketer warp-speed a Claims AI engine in a weekend on Oumi?

I'm a marketer with limited machine learning experience. I built some models in KNIME, a data-focused tool, in the 2010s. That's about it.

A few weeks ago, we had a booth at ITC Insuretech in Vegas, and I wanted something relevant to demo. So, I built Starfleet Insurance, a fictional claims desk on Oumi. I worked on it intermittently over 11 days. My estimate: 3 to 6 hours of hands-on work, plus roughly 5 to 8 hours of computing time. That puts this experiment well within a focused weekend. To be clear, somebody who knows ML could do my part in about 15 minutes.

Oumi is a rare breed of AI startup. Six of our engineers came from Google, where they helped build what became Gemini. They started Oumi because they believe that every company should own the intelligence that runs their business.

Think Claude Code for AI engineering. Oumi automates evaluation, training-data generation, fine-tuning, and deployment. We’re now bringing agent building and evaluation into that same workflow.

Oumi makes it easy enough for a marketer to say "Make it so" and see it done. That’s the opportunity with Oumi. People with business know-how can stay in the captain's chair because the platform makes it simple for them to build useful AI that will live long and prosper.

Welcome to Starfleet Insurance

Starfleet is a fictional claims desk I made up, with claims being beamed in from Planet Nieuw-Vennep. Fender benders (yes, they have those on Nieuw-Vennep) set to stun go straight through. Red Shirt injuries get set on high alert for a specialist. Anything that seems highly illogical goes straight to special Vulcan investigators for auditing.

Why the made-up world? I didn't want to use real people's private information (PII). The claims include names, policy numbers, and medical details. Every claimant, policy, and claim in Starfleet Insurance is fictional.

• A Red Shirt dies on an away mission. An unidentified small craft fires without warning. A lawyer is already involved.
• A crew member dies in a collision while visiting a colony on Starbase 12. The claim needs a handler who deals with complex cases.
• A cracked cooktop in crew quarters. With clear photos and a quote already supplied, this one should sail straight through.

The model reads each claim packet and recommends who should handle it next. That is Claims AI at warp speed. Licensed officers would still decide coverage, how much dilithium (or money) to set aside, settlements, and fraud findings.

Before we set sensors to full scan on the build, stay with me. I'll translate the tech-speak like a Universal Translator as we go. Look for Explain it to me like I’m a marketer boxes.

Here is how the build works. Instead of spending weeks manually engineering data pipelines, writing custom scripts, or configuring training loops, Oumi's Frontier AI Engineer (FAiE) automates the technical heavy lifting across evaluation, schema validation, data synthesis, training, and API deployment while maintaining full strategic control.

What happened inside the build

Here are the core steps and how Oumi and FAiE handle the technical heavy lifting at each stage.

1. Choose one decision (Set course)

Define the specific business task. In this case, the most fundamental service an insurance company does: read an insurance claim and recommend a routing team. When I talked to folks at the booth, I was surprised by how much they struggle with fraudulent claims, which may only get worse with AI. Another key takeaway was how manual processes still are.

Once we define this in the prompt, FAiE automatically translates your goal into a structured task contract and an optimal prompt.

2. Decide what the answer should look like (Calibrate sensors)

Specify required output fields: route, escalation flag, and supporting evidence. FAiE automatically enforces strict schema validation and JSON structure, guaranteeing reliable software integration without custom parsing code.

3. Write the rules for exceptions (Shields up!)

Establish business policies for complex or high-risk claims. Oumi embeds these guardrails seamlessly into the execution flow, ensuring critical exceptions always trigger human review.

4. Set aside the exam (Highly logical, captain)

When generating dataset variations, you explicitly choose the split ratio between training, validation, and evaluation/test datasets. Keep the held-out test set completely separate and never use it in training, so your benchmarks reflect true operational performance without data leakage.

5. Prepare the practice examples (Holodeck simulations)

Prepare training claim-and-route pairs. FAiE handles dataset generation, synthetic data curation, deduplication, and quality checking, saving weeks of manual data gathering by synthesizing thousands of realistic scenario variations in minutes.

6. Measure the starting point (Diagnostic scan)

Benchmark a baseline general model like Claude Sonnet 4.6. Oumi automates evaluation runs and judge scoring to establish an initial reference point before fine-tuning, making it simple to quantify performance gains as edge cases emerge.

7. Train, check, and save versions (Engage engine tuning)

Fine-tune the Qwen3-4B-Instruct-2507 base model on your ~800 synthesized task examples using a LoRA adapter (V6). Oumi fully automates model training, hyperparameter management, compute orchestration, and version tracking, turning complex model training into a background process.

8. Evaluate after training (Diagnostic recalibration)

Evaluate the fine-tuned model on the held-out test set using GPT-5 automated judges (temp 0, seed 1701) to score exact routing matches and multi-layer rubric performance against your pre-training baseline benchmark. GPT-5 measures whether the specialist model passes the target quality bar before clearing the system for warp-speed production deployment.

9. Put the model into a working flow (Energize!)

Publish the trained model into production. Oumi instantly provides production-ready APIs, streamlining critical production deployment considerations like low-latency inference, autoscaling, and fallback handling without requiring backend infrastructure work.

The business owner defines the decision and checks whether the result is useful. Oumi helps run the model-building work. A successful experiment is a starting point for a production system.

Scoreboard: 92% Oumi-Starfleet vs. 78% Claude Sonnet 4.6 (Fascinating, Captain)

On a frozen 197-claim test set, Oumi-Starfleet routed 181 claims correctly. That's 92% accuracy. Claude Sonnet 4.6, given the same prompt, routed 153. That's 78%.

Oumi-Starfleet is the fine-tuned Qwen3-4B-Instruct-2507 model using a LoRA adapter (V6, trained on ~800 cases). Claude Sonnet 4.6 is a general-purpose model built for a wider range of work. Both received the same prompt. Oumi's evaluation judges ran on gpt-5 (temp 0, seed 1701), distinguishing exact match counts from rubric-based layer scores. The final 197-claim test was frozen before scoring. We didn't change the model or the prompt in response to that comparison.

Metric	Oumi-Starfleet	Claude Sonnet 4.6
Exact routing	181/197	153/197
Must-escalate caught	56/59	59/59
Over-routing	12/197	30/197
Under-routing	4/197	10/197
Conflicting answers	0/197	4/197

Even with a lighter, smaller model built by a marketer over a single weekend, Oumi routed more claims correctly and over-routed far fewer cases than Claude Sonnet 4.6. While Sonnet caught all 59 must-escalate cases and Oumi-Starfleet caught 56, missing those few critical escalations is easily eliminated in a production environment by combining the model with simple deterministic rules.

For example, adding a straightforward deterministic rule for recent coverage increases instantly guarantees high-risk claims trigger human review, ensuring full safety without compromising the model's speed or cost benefits.

In this experiment, token costs were roughly 1/40th of the frontier comparison model.

Fascinating. Highly logical.

For regulated enterprises, owning your AI engine provides key advantages:

• Lower Latency: Instant inference responses without relying on external frontier queues.
• Zero PII or IP Leakage: Sensitive customer data and proprietary business logic remain entirely within your secure boundary.
• Complete Auditability: Fully trace, inspect, and explain every decision to compliance and regulatory officers.
• Compounding Accuracy: Performance continuously improves with each production cycle using your own feedback loop.

Owning your AI models and improvement loop ensures long-term strategic control, lower costs, and continuous enterprise value.

Build a model, beam up the mug

I wouldn’t be a marketer if I didn’t offer a free mug to fuel your warp core. Here is how you can claim this limited edition Starfleet Insurance mug:

1. Sign up for Oumi.
2. Paste the starting prompt into the platform assistant. Swap in your own decision workflow. Everything you need to get started is here: https://labs.oumi.ai/starfleet
3. Use Oumi to train and deploy the model. Then send it a test claim through its API and check the answer.
4. Submit the feedback form with your work email and your shipping address.

Qualifying submissions get the Starfleet mug.

It was a great learning experience, and it was empowering to know you can use AI to build your own AI.