Models

Cradle runs models locally through node-llama-cpp (llama.cpp). After a model is downloaded, inference happens entirely on the host — no cloud endpoint, no Python runtime in the main process.

Model roles

Every catalog entry has a role that decides how the runner is used:

The runner provider picks the right model automatically:

• acquireClassifier — first classifier, then the smallest non-embedding model if no classifier is configured.
• acquireForAgent(id) — the exact modelId saved on the agent.
• acquireEmbedder — only embedding models.

Built-in catalog

Cradle ships with assets/models-catalog.json covering chat, coding, and embedding models. A few representative entries:

Sources follow HuggingFace GGUF conventions (unsloth/…-GGUF, bartowski/…-GGUF, Qwen/…-GGUF, CompendiumLabs/bge-m3-gguf).

Hardware compatibility

Before pulling a model, Cradle checks whether it fits the host. The verdict is shown in the Models page and in cradle models list:

cradle models list
# ● fits comfortably  qwen3-8b-q4  needs ~10.0 GB; 18 GB total
# ● tight             qwen3-14b-q4 needs ~16.0 GB; 18 GB total
# ● won't run         llama-3.3-70b-q4 needs ~48 GB; 18 GB total

cradle models check <catalogId> prints the full calculation (weights + KV cache

• headroom). The checker lives in src/core/models/hardware-checker.ts.

Download and resume

Model files are downloaded into the user data directory:

• Desktop app: ~/Library/Application Support/Cradle/models/<id>.gguf
• Server: ~/.cradle/models/<id>.gguf

Downloads support resume via .part files and Range: bytes=X- headers, cancellation through AbortController, optional SHA-256 verification, and progress broadcasts every 300 ms.

Runner interface

type RunnerConfig = {
  contextSize?: number
  gpuLayers?: number | 'auto'
  mode?: 'chat' | 'embedding'
}

interface ModelRunner {
  load(modelPath, modelId, config): Promise<void>
  generate(prompt, opts?): AsyncIterable<string>
  generateStructured<T>(prompt, schema, opts?): Promise<T>
  embed(texts: string[]): Promise<number[][]>
  unload(): Promise<void>
  status(): RunnerStatus
}

LlamaCppRunner is the first implementation. It wraps node-llama-cpp v3 with prebuilt binaries for Metal, CUDA, and Vulkan. A model loads in exactly one mode at a time — switching from chat to embedding (or vice versa) unloads and reloads.

Memory management

The model registry keeps loaded runners in a map keyed by the model row ID:

• Budget defaults to 60 % of total system RAM.
• LRU eviction unloads the least recently used runner when the budget is exceeded.
• last_used_at is updated on every acquire().
• shutdownRegistry() disposes every runner cleanly on exit.

Small models (classifier + embedder) are cheap to keep loaded together. Larger agent models are lazy-loaded on first request.

Custom models

You can add a custom model through the Models page or by editing userData/models-catalog-custom.json:

[{
  "id": "my-domain-slug",
  "name": "My domain model",
  "role": "general",
  "sizeMb": 3000,
  "url": "https://huggingface.co/.../model-Q4_K_M.gguf",
  "sha256": null,
  "recommendedRamGb": 6,
  "contextSize": 8192,
  "description": "..."
}]

Custom entries merge with the built-in catalog. A matching id overrides the built-in entry.

Recommended models

cradle models list --recommended shows a weekly-refreshed top-25 list from the OpenLLM Leaderboard, each with the same compatibility verdict. This is an operator-driven feature: the list is fetched and cached, never auto-downloaded.