Unsloth Desktop

Unsloth Desktop is a free, open-source desktop application for running and training models on your own hardware. It is a Tauri app that bundles llama.cpp-based inference for GGUF models, MLX for Apple hardware, plus diffusion image/video and audio model support. It is in beta.

Owners Club currently recommends Unsloth Desktop for local inference and a ChatGPT-like experience.

At a glance

Why we recommend it

• One app for chat, serving, and training. Most local tools stop at chat; Unsloth Desktop also exposes a local API and a no-code fine-tuning path.
• It connects to agents. unsloth start claude points Claude Code (and similar agents) at your local model, which is how you build a fully local agent loop.
• Permission controls on tool use. Tool calls can run inside a restricted environment rather than editing your files directly.
• Permissive licence. Apache-2.0.

Install and first run

1. Download the installer for macOS, Windows, or Linux from the project's download page (linked from https://unsloth.ai/docs/desktop).
2. Launch the app and open the Model hub (or the Select model dropdown).
3. Choose a model and a quantization that fits your device, download it, then chat. No further setup is required for a basic chat.

Configuration worth knowing

• LAN serving. Bind to your network with -H 0.0.0.0 or use the LAN access card. Traffic stays on your network — but so does the risk: anyone on that network can reach the endpoint.
• HTTPS from anywhere. The app offers a free Cloudflare tunnel so you can reach your model from a phone. Treat a public tunnel as a real exposure, not a convenience toggle.
• Serving via CLI. unsloth run --model <repo/quant> starts a model with explicit context, GPU-layer, threading, and sampling settings for reproducible setups.
• Training. LoRA and full fine-tuning run from the app without writing code; modalities include text, diffusion, and audio.

Common pitfalls

• Beta means beta. Five releases in ten days was the launch cadence. Pin a version you trust for a workshop or a demo rather than updating the morning of.
• Quantization sizing. "Download the biggest model" is the single most common failure. Match quantization to VRAM/RAM first.
• Tooling costs latency. Web search, code execution, and tool-call healing all add time. If generation feels slow, turn them off and compare before blaming the engine.
• Existing model folders. Previously downloaded models are usually detected automatically; if not, set a custom models folder explicitly.
• Older GPUs. The docs are explicit that older hardware may not be well supported.

Tips and tricks

• Prove the chat path with a small model, then add web search and tool use. Debugging two new things at once is how people conclude local AI "does not work".
• Use the OpenAI-compatible endpoint as the integration point: point editors, agents, and scripts at it instead of wiring each tool to a model file.
• For training experiments, caption and export a small dataset first; the expensive step should be the training run, not the data wrangling.

Videos

Verified via YouTube oEmbed on 2026-10-28. Most public coverage discusses Unsloth Studio (the web UI) or fine-tuning rather than Desktop specifically — treat them as adjacent, not exact matches for this page.

• Unsloth Studio is insane… fine-tune any AI model locally — David Ondrej.
• How to Fine-Tune any AI Model Locally (FULL Tutorial) — Tech With Tim.
• Fine-Tuning Local LLMs with Unsloth & Ollama — NeuralNine.

Related

• Jan — the lighter, chat-first alternative.
• Agent Zero — what to point your local model at when you want an agent rather than a chat.
• Choosing local AI tools.

Verification note: licence, platforms, and feature claims are from the official docs page https://unsloth.ai/docs/desktop as read on 2026-10-28. Beta status and release cadence are noted because they materially affect whether you should depend on it for a live demo.