Llamafile 0.10.0: Major Update Brings GPU Support & Multimodal Capabilities

Mozilla’s Llamafile project, designed to run large language models (LLMs) on individual machines without relying on cloud services or container runtimes, has received a substantial update with the release of version 0.10.0. This release marks a significant architectural overhaul, bringing restored GPU support and a streamlined core to the portable LLM runner. The project aims to provide a solution for practitioners working in environments with limited resources or air-gapped security requirements.

Rebuilding for Portability and Performance

The development team deliberately reconstructed Llamafile’s core to create a “cosmopolitan” build of llama.cpp – the underlying engine – prioritizing both portability across diverse operating systems and hardware, and the ability to embed model weights directly within the executable file. This rebuild also incorporates recent advancements in model support that were absent in previous iterations. The goal is to offer a self-contained, easily distributable way to run LLMs locally.

GPU Acceleration Returns to Llamafile

A key feature of the 0.10.0 release is the reintroduction of GPU acceleration. Previous versions of the rebuilt codebase lacked this functionality, but it has now been restored with support for both NVIDIA CUDA on Linux, reintroduced in February 2026, and Apple’s Metal framework for macOS ARM64, implemented via Xcode Command Line Tools in December 2025. Metal support functions in both the terminal interface and server mode. Currently, GPU support for Windows remains unavailable. This addition significantly boosts performance, particularly for larger models, by offloading computationally intensive tasks to the GPU.

New Interface Options and Operational Modes

Version 0.10.0 introduces a terminal user interface (TUI), allowing users to interact with loaded models directly from the command line. A server mode, accessible via the --server flag, is also included. The release expands operational flexibility with support for three distinct modes: chat, command-line interface (CLI), and server. This provides users with a range of options for how they want to interact with and deploy Llamafile.

Expanding Capabilities: Multimodal and Speech Support

Llamafile’s capabilities now extend beyond text-only processing. The mtmd API is accessible through the TUI, enabling multimodal model access directly from the terminal. Models like llava 1.6, Qwen3-VL, and Ministral 3 have been tested with this functionality. Image input is also supported in CLI mode via the --image flag. The integration of Whisper, a speech recognition model, in March 2026 expands the project’s functionality to include audio processing. Mozilla-AI’s release notes detail these additions.

Model Support and Practical Considerations

Mozilla-AI provides a collection of prebuilt example llamafiles with version 0.10.0, ranging in size and performance. A 1.6 GB Qwen3.5 0.8B Q8 quantization reportedly generates approximately 8 tokens per second on a Raspberry Pi 5 without a GPU, while a larger 19 GB Qwen3.5 27B Q5 quantization offers increased capabilities. Other included models are Ministral 3 3B Instruct, llava v1.6 mistral 7b, Apertus 8B Instruct, gpt-oss 20b, and LFM2 24B A2B. The underlying llama.cpp dependency has been updated to commit 7f5ee54, adding support for Qwen3.5 models.

Still, Windows users face a limitation: the operating system’s 4 GB maximum executable file size restricts the employ of many of the larger example llamafiles. The project offers a workaround by supporting external weights, allowing users to load models that exceed the file size limit.

Simplified Build System and Dependencies

The build process for Llamafile has been simplified in the 0.10.0 development cycle. CMake has been replaced with a custom BUILD.mk file, and dependencies are now sourced from the llama.cpp vendor directory. The project now targets cosmocc 4.0.2, and the zipalign utility has been added as a GitHub submodule to ensure alignment with upstream updates. This streamlining aims to make building and maintaining Llamafile easier for developers.

Remaining Work and Future Development

Despite the significant advancements, some features from previous versions have not yet been ported to the new build format. Stable diffusion code remains in the repository but is not yet functional. Pledge() and SECCOMP sandboxing features are currently absent, and Llamafiler for embeddings has been rolled back to utilize llama.cpp’s built-in embeddings endpoint. Certain CLI arguments that functioned in earlier versions are also not yet operational.

Integration tests were added in March 2026, alongside “skill documents” intended for use with AI assistants, suggesting a focus on improving the project’s robustness and integration capabilities. The project’s ongoing development, as documented on the official GitHub repository, indicates a commitment to expanding functionality and addressing remaining limitations.

The continued evolution of Llamafile positions it as a valuable tool for those seeking to leverage the power of LLMs in resource-constrained or security-sensitive environments. The restoration of GPU support and the addition of multimodal and speech capabilities represent significant steps forward, while the ongoing development efforts promise further enhancements in the future.