Ollaya Runs TypeSafe-Style Decision Models on Your Own Machine

TL;DR: Ollaya is a few-days-old open source runtime that serves small "decision models" (compact models that answer typed yes/no or multiple-choice questions with calibrated probabilities in a single forward pass, instead of generating text). It lets jobs like ticket triage or agent guardrails run on local hardware instead of a hosted API call.

It positions itself against TypeSafe's closed Jev model, ships a TypeSafe-compatible API, and adds an MCP server for agent workflows. Its headline numbers (faster, better calibrated) look real, but they come from three different benchmarks run by different people on different test sets, worth untangling before repeating any of them. It's also genuinely early: every release so far has shipped within about 48 hours of the last.

Most teams that want a yes/no or a routing choice out of an LLM still pay for a full generation call: a prompt, a system message, and a JSON parser hoping the model followed instructions. Ollaya skips that path. It downloads and serves open "decision models" of the kind TypeSafe's closed Jev model popularized: models that take a state (a ticket, an email, a JSON payload) plus a small set of typed questions (yes/no, multiple choice, or a rating) and answer with calibrated probabilities in a single forward pass, on your own hardware.

Ollaya's homepage illustrates this with a guardrail example: prompted with the request "Fix the typo in README.md" but about to run git push --force origin main, decider:2b scores four typed questions (action, on_task, risk, destructive) and answers in 178 milliseconds on an RTX 4090. destructive comes back "yes" at 0.90 probability and on_task comes back "no" at 0.75, giving the agent two numbers to threshold on instead of a free-text judgment call (Ollaya).

Getting a decision in three commands

Install is one script, no separate runtime to configure first. On Linux and macOS:

curl -fsSL https://ollaya.dev/install.sh | sh

That script pulls the latest GitHub release, verifies its sha256, and on Linux with systemd sets up a service on 127.0.0.1:11435 (Ollaya quickstart). Windows gets a PowerShell equivalent, and a Docker image, ghcr.io/ollaya-dev/ollaya, covers servers.

Running a model looks like this:

ollaya run laya --preset triage "I was charged twice for my subscription this month and want a refund."

The --preset triage flag asks a built-in question set, and laya pulls the model on first use if it isn't already local. The response comes back as five typed fields (intent, is_urgent, frustration, refund_requested and churn_risk), each with its own probability, rather than a paragraph left to parse (Ollaya quickstart).

A different shape of inference

The reason this is fast is structural, not just smaller weights. A decision model reads the state and the question set once and reads the answer straight off the option logits, with no autoregressive decoding loop in between:

flowchart LR
    A[State plus typed questions] --> B[Decision model]
    B -->|single forward pass| C[Typed answer, calibrated probability]
    C --> D{Above threshold?}
    D -->|yes| E[Agent or pipeline acts]
    D -->|no| F[Escalate to a person or an LLM]

That single-pass design is also why the answers are typed. A choice question returns a probability per option, a score question returns an expected level, and a noul question returns the probability that a yes/no statement holds, all defined in the question set that request supplies rather than inferred from free text (Ollaya quickstart).

Drop-in for existing TypeSafe code

Ollaya also serves /v1/systemone and /v1/models using TypeSafe's request and response shapes, and the official TypeSafe Python SDK 0.7.1 works against it unchanged once three environment variables point at http://localhost:11435 (Ollaya FAQ). For a team already built on TypeSafe's hosted Jev model, that turns a move to local, open weights into an environment variable change rather than a rewrite.

The latency numbers, and whose benchmark they're from

Three separate benchmarks get cited around Ollaya, and they aren't interchangeable, so it's worth being explicit about which is which:

1. Ollaya's head-to-head latency test (RTX 4090, a five-question request through the HTTP API): 8ms for laya:multilingual, 10ms for laya:en, 15ms for gliclass, and 20ms for nli. decider takes longer, at 155–190ms. All of these are far faster than the 236–276ms median that TypeSafe's hosted Jev posts in third-party benchmarks Ollaya cites by repository, AbdelStark/jev-benchmarks and nibzard/decision-model-benchmark (Ollaya, Ollaya FAQ). Ollaya is upfront that this mixes a local GPU call with a network round-trip and calls it order-of-magnitude rather than apples-to-apples.

2. Ollaya's typed-decisions accuracy benchmark (see "Seven models" below), which scores decider, nli, gliclass and laya:en against each other on Ollaya's own test set.

3. Convai's Laya-vs-Jev calibration benchmark (see "The rivalry" below), run on a separate test set by Laya's creator, not by Ollaya, and not the same evaluation as #2.

The guardrail example on Ollaya's homepage probably reaches for decider rather than laya because of benchmark #2: decider is the most accurate model Ollaya ships and, per the FAQ, also the slowest, a sensible tradeoff when missing a destructive command matters more than shaving off a hundred milliseconds (Ollaya FAQ).

The rivalry behind the numbers

Ollaya's FAQ frames itself as serving "the category TypeSafe created," which oversimplifies a messier story.

TypeSafe AI is barely two weeks old in this form. Founded in 2024 by ex-OpenAI RLHF co-inventor Diogo Almeida with Erik Gafni and Sasha Sheng, it left stealth on September 15, 2026 with a $40 million DCVC-led seed round. Jev's launch video reportedly pulled in tens of millions of views within days (YourStory).

Laya, the model Ollaya defaults to, has an earlier claim staked out. Its creator, Nandakishor Mukkunnoth of Convai Innovations, points to a March 2025 paper on the same non-autoregressive approach, arXiv:2503.23303, and says he built Laya as a direct, openly licensed answer once Jev shipped (Mervin Praison).

That's also where benchmark #3 comes from. The 0.081-for-laya-vs-0.246-for-Jev calibration comparison that Ollaya's FAQ repeats is Mukkunnoth's own head-to-head test, run on a 2,000-question set he assembled. That makes it neither an independent evaluation nor the same test set as Ollaya's typed-decisions benchmark below (Mervin Praison).

The Jev latency figure sits on firmer ground: both Ollaya and that benchmark cite the same two third-party repositories, AbdelStark's and nibzard's, for it.

Seven models, four publishers' worth of tradeoffs

Sizes and descriptions come from the Ollaya homepage. The "most accurate" claim is benchmark #2 from the latency section above: Ollaya's internal typed-decisions test, distinct from Convai's laya-vs-Jev numbers.

decider:2b scores 0.591, the highest of the models tested, against 0.548 for nli, 0.506 for decider:0.8b, 0.477 for gliclass and 0.361 for laya:en (Ollaya v0.3.0 release notes). qwen3guard, kev and von weren't part of that benchmark round.

Worth flagging: the FAQ's "which models can I run" answer and the GitHub repository's one-line description both name only four of these seven: laya, decider, nli and gliclass. qwen3guard, kev and von show up on the homepage and on each model's library page, but neither the FAQ nor the repo description mentions them yet (Ollaya FAQ, Ollaya GitHub).

Kev shipped fast, too: Jared Palmer, Turborepo's creator and now VP of engineering at Cognition, released it on September 20 in three sizes, 0.8b, 4b and 9b, all on Qwen3.5 bases; Ollaya currently packages only the smallest (runtimewire).

Where this connects to agent governance

The MCP server is the more interesting piece for anyone building agent guardrails. Built on the official Rust MCP SDK, ollaya mcp exposes decide, list_models, show_model and pull_model as tools over stdio or HTTP, and claude mcp add ollaya -- ollaya mcp is the entire Claude Code setup (Ollaya agents docs, v0.4.0 release notes).

A decision costs about 10 milliseconds and no per-token fee, so an agent can call one before every routing or destructive-action step rather than reasoning it out in text (Ollaya agents docs). It works more like a policy gate than another model in the loop.

There's also a companion ollaya-decisions skill that agents supporting Agent Skills can load, covering how to pick a model, write choice, score and noul questions, and set a confidence threshold for acting versus escalating (Ollaya agents docs). It installs with npx skills add ollaya-dev/ollaya --skill ollaya-decisions, or ships in the release archive next to the binary for offline use.

Privacy and what "open" covers here

The server binds to 127.0.0.1 by default, on port 11435, one number above Ollama's own default of 11434 so the two can run side by side. States and questions are never logged, and the network gets touched only to pull a model (Ollaya FAQ).

Weights come straight from each model author's Hugging Face repository, pinned to a commit and checked against sha256 on every pull. Ollaya's registry serves only manifests and derived ONNX graphs that reference those weights by URL, rather than rehosting the weights itself (Ollaya FAQ).

The runtime itself is Apache-2.0, and each model carries its own license: Apache-2.0 for laya, decider, gliclass and the ModernBERT-large nli checkpoint, MIT for the DeBERTa-v3-large one (Ollaya FAQ).

Platform coverage

Linux needs glibc 2.38 or newer in practice, which covers Ubuntu 24.04, Debian 13, Fedora 39 and RHEL 10 and newer (Ollaya FAQ).

(From the Ollaya homepage platform table.)

Worth knowing before building on it

Ollaya is also very new. Every one of these releases went out inside about two days:

Dates are 2026, from the Ollaya GitHub releases page. The repository sits at 125 stars and 6 forks as of this writing, with 4 open issues.

The FAQ says plainly that Ollaya isn't affiliated with either Ollama or TypeSafe, only compatible with each one's conventions and API (Ollaya FAQ). That matters most for the TypeSafe compatibility layer: it tracks SDK version 0.7.1 today, and keeping pace with any future change to TypeSafe's hosted API is Ollaya's job to maintain, not something guaranteed upstream.

For a triage or guardrail workload that never needed free-text generation in the first place, it's still a reasonable first machine to point at, with the caveat that a project this young can still change its API shape underneath you.