Cortiq loads a .cmf model onto the device and runs local chat there.
An account and hosted inference are not required. Downloads, the phone API,
Companion, and the optional Oracle are separate network features that you control.
Downloaded models run through the same Rust runtime as the desktop tools, bundled as a native library. Local chat stays on the device; network features are opt-in.
Search the Hub, choose a compatible safetensors architecture and a quantization, and the safetensors → CMF conversion runs multi-threaded on the handset. No laptop in the loop.
The loaded model can be served over an OpenAI-compatible API on a trusted LAN, so a laptop can use the phone as its backend.
iPhone 16 / A18 · TestFlight 1.3.0 (47) · LFM2.5 230M Q4TP. The 126 MiB, 14-layer Ready CMF model is estimated at about 622 MiB RAM. With CPU Auto (5), GPU toggle off, temperature 0.10 and a 128-token cap, three warm completions measured 144.5–166.0 tok/s; the median was 145.7 tok/s.
This is the chat UI metric: completion tokens divided by the full execution interval measured by the engine after generation starts. Prompt processing and the model response are included; queue waiting is not. UI latency is rounded, while the rate uses the engine timer. It is not a synthetic core-only decode number or a tap-to-response latency. A separate interactive completion in the current gallery shows 162.1 tok/s. The compact LFM result belongs to that model and workload; it is not a universal claim for every model in the catalog.
The same phone also ran Hy-MT2 Q4TP for short translation at 14.4–21.8 tok/s (17.75 average native output rate) after warm-up; its end-to-end Wi-Fi round-trip is recorded separately. Translation and compact general chat are intentionally separate model choices, not a single throughput ranking.
Full LFM iPhone record · LFM JSON data · Hy-MT2 translation record · LFM2.5 model card · Privacy boundaries
Core chat and decision paths run against a model file in device storage. Downloads and optional integrations are explicit rather than hidden.
.cmf files from device storagecortiq import-gguf imports GGUF into CMF in the desktop toolchain; the resulting .cmf imports into the phone app.cmf are detected and simply downloadedThe converter offers the same codec ladder the desktop tools produce, so a file made on the handset is a file the rest of the ecosystem reads.
| Codec | What it is | Use it for |
|---|---|---|
Q4TP recommended | 4-bit tiles on a per-row scale ladder | The default for dense models — what the desktop tools emit |
Q2TP | 2/4 mixed MoE profile: gate and up experts at 2-bit, the rest q4tp | Mixture-of-experts checkpoints that would not otherwise fit |
Q8_2F | 8-bit two-field (𝒲 × θ) | Highest fidelity; the reference when you are checking a smaller codec |
Q8_ROW | 8-bit per row | A simpler 8-bit, smaller header, wide compatibility |
Q1T | Training-free ternary {−s, 0, +s}, ≈2.25–3 bit, with a per-row f16 outlier overlay | Below 4-bit on a model that was never trained for it |
Q1 | 1.5-bit for models trained 1-bit (Bonsai, BitNet) | A 27B checkpoint in roughly 5 GB |
F16 | No quantization | Small models, or a baseline to measure against |
The active model determines the contract: chat models expose the OpenAI-compatible chat routes; a CMF Decision model exposes typed decision and skill routes. Use either only on a trusted LAN.
| Endpoint | Purpose |
|---|---|
POST /v1/chat/completions | Chat — JSON and SSE streaming (chat model) |
POST /v1/decide | Convenience endpoint for one skill decision |
POST /v1/decisions | Typed decision request (Decision model) |
GET /v1/skills | Discover skills offered by the loaded Decision model |
POST /v1/completions | Legacy completions |
GET /v1/models | The loaded model |
GET /v1/cortiq/status | Uptime, tokens, tok/s |
GET /v1/cortiq/masks | Task masks baked into the model |
POST /v1/cortiq/switch | Switch the active task |
GET /healthz | Health check |
Pair the handset with a desktop running cortiq worker and the desktop holds the layers, the head and the sampler while the phone keeps the tokenizer and draws. That is what puts a 34.7B MoE on a phone with 2 GB free at 16.3 tok/s. It also works the other way: the phone serves a span of its own layers so a desktop can run a model bigger than its memory.
Measured on a Xiaomi 12 Lite against a MacBook M4 over USB with the same 1.7B model: 12.5 tok/s on the phone alone, 30.9 through the desktop.
The split buys memory, not speed — a token walks the layers in order — and the app says so rather than offering to "speed up" a model that already fits. The transport is labelled too: one round trip per token is 95 ms p99 over Wi-Fi against 2.9 ms on a cable, and the app names which one you are on.
English TestFlight build 47. Start with the LFM2.5 230M speed capture, then browse the current Decision, Ready CMF, API, Oracle, Companion, and settings flows—not the older 1.2 gallery. The device-local bearer field is redacted in the server capture.
scroll
LFM2.5 230M — 162.1 tok/s in the phone chat UI
CMF Decision — accepted card_arrival, confidence and timings
Ready CMF — one file per card; Chat and Decisions are distinct
Decision API — typed routes; bearer value redacted
Oracle — optional, manual and off by default
Companion — split execution and transport boundaries
Settings — English, Oracle boundary and CPU/GPU behaviorEvery .cmf repo published at
huggingface.co/infosave
is pinned in the app as a featured model and downloads in one tap. A compatible
safetensors repository on the Hub goes through the on-device converter. The CMF desktop
toolchain also imports GGUF with cortiq import-gguf; its resulting .cmf
file can be imported from storage.