Start your AI Journey

How to run an LLM on a laptop: DS4 squeezes DeepSeek V4 Flash, a 284B MoE model, into 76GB at 26 tokens/sec

DeepSeek V4 Flash is a 284-billion-parameter, MIT-licensed Mixture-of-Experts model with only 13B active parameters per token. Two weeks after release, Salvatore Sanfilippo (antirez, the creator of Redis) shipped DS4, also called DwarfStar 4, a pure-C inference engine built to run this frontier-scale model fully locally on a 128GB unified-memory machine, no cloud, no API key, no GPU cluster required.

The trick is asymmetric quantization: the MoE expert layers get crushed to 2-bit (IQ2_XXS/Q2_K), while the routing layer, shared expert, and projection layer stay at full 8-bit precision, because a mistake in the router breaks everything downstream while a mistake in a redundant expert doesn’t. That gets the whole model into roughly 76GB, and on a MacBook Pro M3 Max it hits 26.68 tokens per second, interactive speed for a model this size. DS4 deliberately won’t load arbitrary GGUF files, it ships its own validated weights tested against DeepSeek’s official outputs, plus an HTTP API, tool calling, a coding agent, and a disk-persisted KV cache that survives across sessions. It’s not a llama.cpp or Ollama or LM Studio wrapper, it’s a narrow, single-model engine, and it crossed 13,000 GitHub stars in a month, catching llama.cpp author Georgi Gerganov’s attention.

The video covers what shipped, how the quantization actually works, why antirez chose lock-in over flexibility, and where the hype breaks down, including the real hardware price floor and the much bigger 1.6-trillion-parameter V4 PRO model this doesn’t replace.

For builders paying per-token API bills for flash-tier work who want a serious private AI option running on hardware they own.

Chapters:
0:00 Intro
0:19 Two Drops That Changed Everything
1:48 The 76GB Trick That Shouldn’t Work
3:16 One Model Only, On Purpose
4:41 The Line Antirez Never Says Lightly
5:42 Where This Story Actually Cracks
7:19 So What Do You Actually Do Now

Tools & resources mentioned:
– DS4 (DwarfStar 4), antirez’s inference engine: https://github.com/antirez/ds4
– DeepSeek V4 Flash (model weights & info): https://github.com/antirez/ds4
– antirez blog post on DS4: https://antirez.com/news/165
– llama.cpp (Georgi Gerganov): https://github.com/ggerganov/llama.cpp
– DS4 / DwarfStar 4: https://github.com/antirez/ds4
– llama.cpp: https://github.com/ggerganov/llama.cpp
– DeepSeek V4 Flash: https://build.nvidia.com
– antirez.com: https://antirez.com/news/165
– NVIDIA DGX Spark: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
– LM Studio: https://lmstudio.ai
– Ollama: https://ollama.com

About The Stack
The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs.

We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship.

Subscribe for new breakdowns: https://www.youtube.com/@The-Stack-ai?sub_confirmation=1

#deepseek #llamacpp #localai #privateai #antirez

source