Run Llama in Your Browser Without Internet — WebLLM Guide
Llama 3.2 runs in Chrome and Edge using WebGPU. After one download, it works fully offline. Here is the setup and what it can do.
Llama is Meta's open-weight language model. You can download the weights and run them however you want. Including, now, inside a browser tab without any internet connection after the initial download. Here's how that works.
Why this is possible now
Three things came together to make this work. First, quantization: modern quantization techniques compress model weights significantly, so a 7B-parameter model that would take 14 GB at full precision can be packed into 4-5 GB at 4-bit precision with only modest quality loss. Second, WebGPU: browsers can now access GPU compute directly, making neural network inference fast enough to be usable. Third, the models themselves got better at smaller sizes, with Meta's Llama 3.2 being quite capable at 3 billion parameters.
How to use Llama in your browser
Our Browser AI Chat uses a Llama-based model. When you first open it:
- The model file downloads from a CDN (around 2-4 GB depending on the model size selected)
- The browser caches it using the Cache API, so it persists between sessions
- WebGPU loads the model into your GPU memory
- All subsequent inference runs locally, with no internet connection required
After the initial download, you can disconnect from the internet and the model keeps working.
What to realistically expect
The 3B version is fast (10-20 tokens per second on a modern laptop) and handles writing assistance, Q&A, and summarization competently. It's not going to match GPT-4 on complex reasoning. The 7B version is slower but noticeably more capable. Both are free, private, and work offline.
Storage and memory
The model file lives in your browser's cache, which is separate from your downloads folder and managed by the browser. You can clear it from browser settings if you need the space back. RAM usage during inference is roughly 1.5x the model file size, so a 4 GB model needs about 6 GB of RAM available.
Use cases that make sense offline
Writing and editing where you don't want a cloud service seeing the content. Travel or remote work without reliable connectivity. Situations where your employer prohibits external AI services. Consistent performance regardless of your internet connection speed.