What is llama.cpp — the engine under Ollama and LM Studio, and what 1.0 promises
llama.cpp is an open-source inference engine that runs AI language models directly on your own computer. Georgi Gerganov started it in March 2023 in plain C/C++, and it loads a model packed into a single GGUF file onto a CPU or GPU. Because those files are usually quantized to 4 or 5 bits per weight, an 8-billion-parameter model needs about 4.9 GB. Ollama and LM Studio both run models on this engine. Version 1.0.0, tagged on November 13, 2026, removed 35 deprecated functions and is the first release to promise that the public C API will break only when the major version number goes up. For serving many users at once, a server engine such as vLLM is a better fit.
The three lines
- Meaning — a C/C++ engine that runs AI models stored as GGUF files on CPUs and GPUs; Ollama and LM Studio run on it
- Size — at Q4_K_M an 8B model is about 4.9 GB and a 70B model about 43 GB; multiply billions of parameters by 0.61
- 1.0.0 — tagged November 13, 2026; 35 deprecated functions removed; public C API breaks only on a major version bump
Key questions
- what is llama.cpp
- **An open-source C/C++ inference engine that runs AI language models on your own computer. It does not build or train models; it reads a finished model file and generates answers.** | Item | Detail | |---|---| | Creator | Georgi Gerganov; repository opened March 10, 2023 | | Language and license | C/C++, MIT | | File it reads | GGUF (weights and settings in one file) | | Where it runs | CPUs, Nvidia, AMD and Intel GPUs, Apple silicon, Snapdragon and more | | How it is used | Terminal chat (`llama cli`), local API server (`llama serve`), C library | | Latest official version | 1.0.0 (November 13, 2026) |
- llama.cpp vs ollama
- **llama.cpp is the engine. Ollama is a program that ships that engine and handles downloading and managing models for you. Ollama carries two engines: llama.cpp and Apple's MLX.** | Tool | What it is | Relation to llama.cpp | |---|---|---| | llama.cpp | Inference engine and basic tools | The engine itself | | Ollama | Model management and running in one program | Bundles llama.cpp; MLX is the default on Apple silicon | | LM Studio | Desktop app with a graphical interface | Runs GGUF models with llama.cpp; also supports MLX on Apple silicon | | vLLM | Server engine for many concurrent requests | Separate project | | MLX | Apple's machine-learning framework for Apple silicon | Separate project |
- what is GGUF
- **The model file format llama.cpp reads. Weights, tokenizer and settings sit in one file, and a label such as Q4_K_M at the end of the file name says how many bits each weight was reduced to.** | Label | Bits per weight | Size of an 8B model | Versus 16-bit | |---|---|---|---| | F16 | 16.00 | 14.96 GiB | 100% | | Q8_0 | 8.50 | 7.95 GiB | 53% | | Q6_K | 6.56 | 6.14 GiB | 41% | | Q5_K_M | 5.70 | 5.33 GiB | 36% | | Q4_K_M | 4.89 | 4.58 GiB | 31% | | Q3_K_M | 4.00 | 3.74 GiB | 25% |
llama.cpp is an open-source inference engine that runs AI language models directly on your own computer. Most of the programs people use to download a model and run it on a laptop or desktop, instead of connecting to a chatbot service, either are this engine or carry it inside. Ollama and LM Studio are the best-known examples.
The project tagged version 1.0.0 at 10:20 UTC on November 13, 2026, three years and eight months after its repository opened on March 10, 2023. It looks like a change of one digit. The substance is a promise: changes that break the public C API will be made only when the major version number goes up. This article explains what llama.cpp is, how to read labels such as Q4_K_M in model file names, how large a model a given computer can run, and how the engine relates to Ollama, LM Studio, vLLM and MLX.
1. What llama.cpp is and why it exists
Using an AI language model involves two stages. Training builds the model. Inference feeds a finished model a prompt and gets an answer back. llama.cpp does inference only. It loads a file holding billions of numbers, called weights, into memory and computes the next token of text, one at a time.
Until early 2023 that work generally meant Python, PyTorch and an Nvidia GPU. Developer Georgi Gerganov began writing a tensor library called ggml in late September 2022 and first used it for whisper.cpp, a port of OpenAI's speech-recognition model. llama.cpp came next. According to Wikipedia it started as a rewrite of inference for Meta's Llama model in dependency-free C/C++, aimed at machines without a GPU. The "llama" in the name comes from that model, and ".cpp" is the file extension for C++ source code. Most models run on it now are not Llama models, but the name stayed.
| Date | What happened | Source |
|---|---|---|
| Late September 2022 | Gerganov begins the ggml library | Wikipedia |
| March 10, 2023 | llama.cpp repository opened | GitHub repository creation date |
| August 2023 | GGUF model file format introduced | Wikipedia, GGUF specification |
| April 10, 2025 | libmtmd introduced, giving image and audio input a single API | Wikipedia |
| February 20, 2026 | The ggml.ai team joins Hugging Face | Wikipedia |
| August 7, 2026 | Semantic versioning plan published in a GitHub discussion; 0.1.x tags follow | ggml discussion #1579 |
| November 4, 2026 | Version 0.9.0 | GitHub releases |
| November 13, 2026 | Version 1.0.0; ggml reaches 1.0.0 the same day | GitHub releases |
Three things explain its spread. First, there is almost nothing to install. The project README lists a plain C/C++ implementation without dependencies as its first feature. Second, it runs on a wide range of hardware. The backend table in the v1.0.0 README has 17 entries, including Nvidia GPUs (CUDA), AMD GPUs (HIP), Apple silicon (Metal), Intel GPUs (SYCL), Snapdragon (Hexagon), generic GPUs (Vulkan) and WebGPU. Third, it uses shrunken models. It supports quantization of weights to integers of 1.5 to 8 bits, and when a model does not fit in GPU memory it can put some layers on the GPU and compute the rest on the CPU.
The GitHub repository had 135,193 stars when checked on November 16, 2026, and the license is MIT. In its 1.0.0 announcement the project team wrote that it supports more than 150 model architecture variants.
2. GGUF files and how to read Q4_K_M
The model files llama.cpp reads are in the GGUF format. The specification describes it as the successor to the earlier GGML, GGMF and GGJT formats, designed around three goals: single-file deployment with no external files, extensibility so new information can be added without breaking existing models, and compatibility with the operating system's mmap call for fast loading. The file holds not only weights but also the tokenizer vocabulary, the context length and settings such as the chat template, so one download contains everything needed to run the model.
GGUF file names on model-sharing sites are long. The specification sets the following convention, with parts joined by hyphens.
| Position | Meaning | Example |
|---|---|---|
| Prefix (optional) | Marks an auxiliary file loaded alongside a base model | mmproj (image and audio input), mtp (speculative decoding) |
| Base name | Model family | Hermes-2-Pro-Llama-3 |
| Size | Parameter count; B is billions, M is millions | 8B, 8x7B (a mixture-of-experts model with 8 experts) |
| Fine-tune | Intended use | instruct, chat |
| Version | Model version | v1.0 |
| Encoding | How the weights were reduced | Q4_K_M, F16 |
| Shard | Sequence number when split across files | 00003-of-00009 |
The specification's example Hermes-2-Pro-Llama-3-8B-v1.0-F16.gguf reads as: the Hermes 2 Pro Llama 3 family, 8 billion parameters, version 1.0, stored as unreduced 16-bit floating point.
The encoding slot causes the most confusion. Take Q4_K_M apart: Q means quantized, 4 is the nominal number of bits per weight, K names a method known as k-quants, and M is a grade within that bit level (S small, M medium, L large). K-quants do not shrink every layer equally. They give more bits to layers that matter more, so the actual average is 4.89 bits, not 4.
| Label | How to read it | Bits per weight | Size (Llama 3.1 8B) | Versus 16-bit |
|---|---|---|---|---|
| F16 | 16-bit floating point, not reduced | 16.00 | 14.96 GiB | 100% |
| Q8_0 | 8-bit, older simple method | 8.50 | 7.95 GiB | 53% |
| Q6_K | 6-bit k-quant | 6.56 | 6.14 GiB | 41% |
| Q5_K_M | 5-bit k-quant, medium | 5.70 | 5.33 GiB | 36% |
| Q4_K_M | 4-bit k-quant, medium | 4.89 | 4.58 GiB | 31% |
| Q4_K_S | 4-bit k-quant, small | 4.67 | 4.36 GiB | 29% |
| Q3_K_M | 3-bit k-quant, medium | 4.00 | 3.74 GiB | 25% |
| IQ2_M | 2-bit i-quant, medium | 2.93 | 2.74 GiB | 18% |
The figures are from the quantization tool documentation in the llama.cpp repository, measured on Meta's Llama 3.1 8B. Labels beginning with IQ are i-quants, which use an importance matrix to limit quality loss at very low bit counts. The documentation says quantization may introduce accuracy loss, usually measured as perplexity or KL divergence. Fewer bits mean a smaller file and more loss.
The documentation also confirms that Q4_K_M is the default. If you give llama.cpp a Hugging Face repository name without specifying a quantization type, it fetches the Q4_K_M file, or the first file in the repository if none exists.
3. How large a model your computer can run
Memory needs can be estimated with arithmetic. File size in gigabytes is the parameter count in billions, times bits per weight, divided by 8. At Q4_K_M's 4.89 bits that is about 0.61 GB per billion parameters. The measured values in the llama.cpp documentation agree: Llama 3.1 8B is 4.9 GB and the 70B model is 43.1 GB, against calculated values of 4.9 GB and 42.8 GB. The same document says the 405B model, 1,625 GB in its original form, comes to 249.1 GB at Q4_K_M.
| Parameters | Q4_K_M file | Q8_0 file | F16 file | Memory to plan for at Q4_K_M (file × 1.2) |
|---|---|---|---|---|
| 4 billion (4B) | 2.4 GB | 4.3 GB | 8.0 GB | About 2.9 GB |
| 8 billion (8B) | 4.9 GB | 8.5 GB | 16.0 GB | About 5.9 GB |
| 14 billion (14B) | 8.6 GB | 14.9 GB | 28.0 GB | About 10.3 GB |
| 27 billion (27B) | 16.5 GB | 28.7 GB | 54.0 GB | About 19.8 GB |
| 32 billion (32B) | 19.6 GB | 34.0 GB | 64.0 GB | About 23.5 GB |
| 70 billion (70B) | 42.8 GB | 74.4 GB | 140.0 GB | About 51.4 GB |
| 120 billion (120B) | 73.4 GB | 127.5 GB | 240.0 GB | About 88.1 GB |
This table rests on three assumptions. The bits-per-weight values measured on Llama 3.1 8B (4.89 for Q4_K_M, 8.50 for Q8_0, 16.0 for F16) are applied to every model. The 1.2 multiplier in the last column adds 20% for the KV cache that stores context and for working buffers, a rough allowance for a short context of around 8,000 tokens. And every layer is assumed to be loaded. Real figures vary with model architecture and settings.
Read the other way, the same assumptions give this.
| Your computer | Comfortable at Q4_K_M | Tight or out of reach | Note |
|---|---|---|---|
| No GPU, 16 GB RAM | 8B | 14B needs other programs closed | CPU-only is slow |
| 8 GB of graphics memory | 8B | 14B only partly on the GPU | Overflow layers go to CPU and RAM |
| 12 GB of graphics memory | 14B | 27B | |
| 16 GB of graphics memory | 14B (Q6_K also fits) | 27B only partly on the GPU | |
| 24 GB of graphics memory | 27B | 32B leaves almost no margin | A long context will overflow |
| Apple silicon, 32 GB unified memory | 27B | 32B | Memory is shared with the operating system |
| 64 GB unified memory | 70B | ||
| 128 GB unified memory | 120B |
A model still runs when graphics memory falls short. llama.cpp can split layers between GPU and CPU. In the v1.0.0 server documentation the number of layers sent to the GPU (-ngl) defaults to automatic, and a --fit option that adjusts unset values to the device's memory is on by default. The more layers that land on the CPU, the slower it gets.
Context length costs memory too. The model stores intermediate results for every token it has read in the KV cache, whose size grows in proportion to context length. System Design Academy worked an example with the Qwen3 0.6B model: 28 layers, 8 key-value heads, 128 numbers per head, 2 bytes per number at 16-bit, with keys and values stored separately, comes to 114,688 bytes per token. A 32,768-token context is 3,584 MiB. That cache is several times larger than the weight file of a 0.6B model. It is why a model can load fine and still run out of memory when you paste in a long document. The FP8 KV cache added in 1.0.0 is an option that cuts each of those numbers from 2 bytes to 1.
Models that compute only part of their weights for each token can do better than this table suggests even when the total parameter count is large. That is the mixture-of-experts design, and 1.0.0 added a feature that caches such a model's expert blocks in main memory.
4. Installing and running it: the minimum from the official docs
Everything below is taken from the README and documentation at the v1.0.0 tag. There are several ways to install.
| Route | Platforms | How |
|---|---|---|
| Install script | Linux, macOS, Windows | The one-line command in the README (below) |
| Prebuilt files | Linux, macOS, Windows, Android, iOS | Download from the GitHub releases page |
| Package managers | By operating system | Homebrew, Winget, MacPorts, conda-forge, Debian |
| Docker | Linux | ghcr.io/ggml-org/llama.cpp:server and others; separate images for CUDA and ROCm |
| Build from source | All | cmake -B build, then cmake --build build --config Release |
The install command the README puts first is curl -LsSf https://llama.app/install.sh | sh on Linux and macOS, and irm https://llama.app/install.ps1 | iex in Windows PowerShell. Both run a script fetched from the internet straight away, so use a package manager or a release file if you would rather not. The package-manager list is the set of distribution channels named in the project team's announcement.
Once installed, there are two basic ways to use it.
| Goal | Command | Explanation |
|---|---|---|
| Chat in the terminal | llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF | Downloads a model from Hugging Face and runs it |
| Start a local API server | llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF | Serves an OpenAI-compatible API and a web interface |
| Run a file you already have | llama cli -m path-to-file.gguf | -m is the model file path |
| Choose a quantization type | -hf user/model:Q4_K_M | Q4_K_M if omitted |
The model in the examples is the small 0.8B one the README uses. A server started this way listens, per the v1.0.0 documentation, on port 9931 at 127.0.0.1. Opening that address in a browser shows the built-in web interface, and programs can send OpenAI-style requests to /v1/chat/completions. The documentation also lists an Anthropic Messages API-compatible route at /v1/messages. In practice, code that calls a cloud API can be pointed at a local model by changing the address.
Common options and their defaults:
| Option | Meaning | Default |
|---|---|---|
-c, --ctx-size | Context length in tokens | 0, meaning the value stored in the model |
-ngl, --gpu-layers | Layers to place on the GPU | Automatic |
--fit | Adjust unset values to fit device memory | On |
-ctk, -ctv | Data type of the KV cache, such as f8_e4m3 or q8_0 | f16 |
--host | Address to listen on | 127.0.0.1 |
--port | Port to listen on | 9931 |
--api-key | Key used for authentication | None |
One note on command names. The v1.0.0 README uses a single llama command with subcommands, as in llama cli and llama serve. The release notes, the Docker documentation and many existing guides still use the older executable names llama-cli and llama-server. They refer to the same tools. Some examples inside the server documentation also still show the older port 8080.
5. How it relates to Ollama, LM Studio, vLLM and MLX
A search for local AI turns up several names that sit at different layers. llama.cpp is the engine at the bottom. Above it are programs that add model management and an interface. Beside it are engines built for other purposes.
| Tool | What it is | Relation to llama.cpp | Good fit when |
|---|---|---|---|
| llama.cpp | Inference engine, command-line tools, local server, C library | The engine itself | You want direct control of options, or to embed an engine in an app |
| Ollama | One program that downloads, lists and runs models | Bundles llama.cpp and updates it alongside its own releases; MLX is the default on Apple silicon | You want to try a model quickly with no setup |
| LM Studio | Desktop app for choosing and chatting with models | Runs GGUF models with llama.cpp; also supports MLX on Apple silicon | You do not use a terminal |
| vLLM | Server engine for many concurrent requests | Separate project | You are serving many users |
| MLX | Apple's machine-learning framework for Apple silicon | Separate project | Speed on a Mac is the priority |
Ollama has two engines. Its release notes list updates to llama.cpp and MLX side by side in almost every version. Version 0.40.0, released on September 25, 2026, changed the default so that on Apple silicon devices, model architectures supported by MLX run on MLX automatically. Version 0.40.2, on October 8, 2026, said models downloaded with earlier versions are upgraded in the background for better performance on llama.cpp. So an Ollama user is running models on llama.cpp or on MLX, depending on the machine and the model.
LM Studio says in its documentation that it runs models on Mac, Windows and Linux using llama.cpp, and that on Apple silicon Macs it also supports Apple's MLX. The app presents GGUF models and MLX models as separate kinds.
vLLM targets a different seat. Its README leads with serving throughput, memory management through PagedAttention, and continuous batching of incoming requests. It is built for one GPU server answering many requests at once more than for one person on one machine. The llama.cpp server supports parallel decoding and continuous batching as well, but the goal the project states is minimal setup across a wide range of hardware.
MLX is an array framework from Apple's machine-learning research group. llama.cpp also supports Apple silicon through Metal, so on a Mac the choice between the two is open. No independent comparison of the two under identical conditions was found.
The llama.cpp team now ships a desktop app of its own. The 1.0.0 announcement introduced Llama apps for macOS and Windows and did not mention one for Linux.
6. What changed in 1.0.0 and what it means for app developers
For most of its life llama.cpp shipped under build numbers such as b12359, one per batch of commits. A number alone did not say which build would break existing code, and developers who embedded the engine had to check what changed each time they updated.
Version numbers did not begin with 1.0.0. Gerganov published a release and versioning plan for ggml-org projects in a GitHub discussion on August 7, 2026, and tags from 0.1.0 through 0.9.0 followed. Version 1.0.0 is the point at which the number becomes a compatibility promise. The project team wrote in its announcement that both llama.cpp and ggml will reflect API and ABI incompatibilities with major-version bumps.
| Kind of change | Part that goes up | Example |
|---|---|---|
A breaking change to the public C API in the include folder | Major | 1.x to 2.0 |
| A backward-compatible feature, backend update, model support or API addition | Minor | 1.0 to 1.1 |
| A bug fix with no API change | Patch | 1.0.0 to 1.0.1 |
Nightly builds continue under build numbers. According to the discussion thread, an official tag is created only when the copy of ggml inside llama.cpp is identical to the released ggml, and each official version corresponds to a specific nightly build. For 1.0.0 that build is b12359. Participants in the discussion wrote that most releases will be minor bumps.
The main changes listed in the 1.0.0 release notes:
| Area | Change | Effect |
|---|---|---|
| API cleanup | Removed 35 functions marked deprecated in llama.h, plus llama_set_warmup, the defrag_thold field and the --defrag-thold option | Code calling these no longer compiles |
| API promotion | Moved 18 functions of the NextN speculative-decoding API from the non-public header llama-ext.h into llama.h; renamed llama_set_embd_state_src to llama_set_ctx_state_src | Speculative decoding now falls under the compatibility promise |
| Memory | FP8 KV caches (f8_e4m3 for -ctk and -ctv) | Half the cache size of 16-bit |
| Mixture-of-experts | A host-memory cache for expert blocks with a configurable CPU share | Running MoE models larger than graphics memory |
| Models | Three text architectures (Kimi-K3.1-Pro, Ripple, xing4_0), the drama-3 text-to-speech model and others | |
| Server security | Fixed CORS origin handling, added protection against DNS rebinding, fixed SSRF in the CORS proxy, restricted the Hugging Face token to huggingface.co hosts | See section 7 |
| Server features | OpenAI-compatible /v1/audio/speech endpoint; redesigned web interface | |
| Multimodal | Removed clip.h and consolidated into mtmd | Apps that used the image-input code directly need changes |
| ggml | Version 1.0.0: deprecated API pruned, FP8 support, performance work across backends |
Speculative decoding speeds up generation by letting a small helper model, or a prediction head attached to the model, propose several tokens ahead that the main model then verifies in one pass. While that API lived in a non-public header it could change without notice. In the August discussion Gerganov wrote that the header was not public, so it could be changed without a version bump.
For app developers this means two things. In the short run there is work: code that calls deprecated functions will fail to build on 1.0.0 and has to be fixed first. After that there is less: within 1.x, code written against the public headers keeps building, and distribution packagers can judge library compatibility from the major number. The promise is not a freeze on change. It is a commitment to signal breaking changes through the version number.
For ordinary users little changes directly. Those on Ollama or LM Studio move up when those apps ship the new engine. Those running llama.cpp themselves should remove --defrag-thold from any scripts that pass it.
7. Limits: speed, large models and security when run as a server
| Limit | Detail | What to do |
|---|---|---|
| Speed | Whatever does not fit on the GPU runs on the CPU and slows things down; CPU-only is rarely practical for large models | Choose a smaller model or a lower-bit file |
| Long inputs | The first pass over a long prompt is slow, a point also raised in comments on the announcement | Set context length no higher than needed |
| Large models | Per section 3, a 70B model needs around 50 GB even at Q4_K_M | Lower-bit files cost more quality |
| Quantization loss | Loss is generally reported as small down to about 4 bits, but it varies by model and task | Compare with your own prompts |
| Concurrent users | Parallel processing is supported, but large-scale serving is not the main aim | At service scale, consider a server engine such as vLLM |
| Security | Exposing the server beyond your machine exposes an unauthenticated API | See below |
Security deserves a closer look. By default llama serve listens on 127.0.0.1, reachable only from the same computer, and no API key is set. For one person on one machine that is fine. The moment --host is changed so another device can connect, anyone on the same network can use the model without a key. There are also attacks through the browser: even with the server open only to your own machine, a script on a web page you visit can be made to send requests to it. DNS rebinding is one such technique, and loose CORS settings open the way. In the v1.0.0 documentation the default for allowed origins (--cors-origins) is to allow all.
Version 1.0.0 worked on this area. According to the release notes it fixed CORS origin handling and protection against DNS rebinding, closed an SSRF hole in which a feature that fetches outside addresses on the user's behalf could be steered to internal network addresses, and blocked direct browser access to child server instances. It also added two options that are closed by default. Using tools or MCP servers on a non-loopback address without an API key requires --allow-unauthenticated-tools, and allowing media URLs such as image addresses to point to local or private addresses requires --media-allow-private-urls. The documentation says the first is only for trusted networks.
The working rule is simple. For solo use, leave the default address alone. If another device has to connect, set --api-key and narrow --cors-origins. Putting the server directly on the internet is not something engine settings alone can make safe.
8. Frequently asked questions
Is llama.cpp free? The engine is open source under the MIT license and costs nothing. What you pay for is your own electricity and hardware. Each model carries its own license, and being able to download a model does not mean every use is permitted.
Does it work without an internet connection? Yes, once the model file has been downloaded. That prompts never leave the machine is the main reason people choose to run models locally.
I use Ollama. Do I need to install llama.cpp separately? No. The engine is inside Ollama. People install llama.cpp directly when they want fine control over options or the newest features first.
Can I run it without a GPU? Yes. The engine was built for machines without one. CPU-only inference is slow, so models of roughly 4 to 8 billion parameters are the realistic range.
Should I download Q4_K_M or Q8_0? With memory to spare, more bits stay closer to the original. A Q8_0 file is about 1.7 times the size of Q4_K_M. For a fixed amount of memory the question becomes a bigger model at fewer bits or a smaller model at more bits, and the answer depends on the task. The llama.cpp default is Q4_K_M.
Do I have to re-download my GGUF files for 1.0.0? The release notes contain no item changing the GGUF format. What breaks in 1.0.0 is the C API and a few command-line options.
Does 1.0 mean the software is finished? No. It is a promise not to break the interface casually. Several nightly builds a day have kept arriving since 1.0.0, and the build number had reached b12425 on November 15, 2026.
Does it run on phones? The project distributes builds for Android and iOS and has backends for Snapdragon and Adreno GPUs. Phone memory limits keep the models small.
9. What remains open or unconfirmed
| Item | Confirmed | Not confirmed |
|---|---|---|
| Supported architectures | The project team says more than 150 | The list was not counted |
| Removed functions | The count of 35, and llama_set_warmup | The name of each of the 35 |
| Age of the project | Three years and eight months from repository creation | Which date the announcement's 3.5 years counts from |
| FP8 KV cache | Option name, the f16 default, and the one-half byte arithmetic | Effect on quality and speed; the report of a failed start on the Apple GPU is one outlet's test |
| Size-by-hardware table | Bits per weight, and measured 8B and 70B sizes | The 20% allowance is an estimate, not a measurement |
| Ollama | That it ships both llama.cpp and MLX | Whether a version carrying the 1.0.0 engine is out |
| Versioning rules | Major, minor and patch defined against the public C API | How server REST API changes will be counted; release.md in the repository still holds older text |
| Speed across engines | The goals each project states | An independent like-for-like comparison |
| Wikipedia | Listed 0.9.0 as the stable release when checked on November 16, 2026 | When it will be updated |
For how weights are shrunk and what 4 bits means, see our explainer on model quantization. The wider trend of running models on the device is in our explainer on on-device AI, the difference between downloadable models and their licenses is in our explainer on open-weight models, and the architecture that lets a large model run with a fraction of the computation is in our explainer on mixture of experts (MoE).
Sources
- GitHub ggml-org/llama.cpp — v1.0.0 release notes (2026-11-13)
- Hugging Face blog (ggml-org) — llama.cpp v1.0.0 announcement (2026-11-13)
- GitHub ggml-org/llama.cpp PR #30565 — llama : prune (merged 2026-11-12)
- GitHub ggml-org/ggml discussion #1579 — releases and versioning of ggml-org projects (2026-08-07)
- llama.cpp README at the v1.0.0 tag — install, usage, supported backends
- llama.cpp docs/release.md at the v1.0.0 tag — versioning rules
- llama.cpp tools/server/README.md at the v1.0.0 tag — server options and defaults
- llama.cpp tools/quantize/README.md at the v1.0.0 tag — sizes by quantization type
- llama.cpp docs/build.md and docs/docker.md at the v1.0.0 tag — building and Docker images
- ggml docs/gguf.md — GGUF format specification and naming convention
- Wikipedia — Llama.cpp
- GitHub ollama/ollama — release notes (v0.40.0, 2026-09-25, and others)
- LM Studio docs — app overview (supported engines and platforms)
- GitHub vllm-project/vllm — README
- GitHub ml-explore/mlx — README
- tbreak — llama.cpp 1.0 stable release (2026-11-13)
- Pasquale Pillitteri — llama.cpp 1.0: stable API and ggml (2026-11-13)
- DEV Community — llama.cpp just hit 1.0 (2026-11-14)
- System Design Academy — llama.cpp v1 explained (2026-11-14)