The record, with its boundaries

Every speed number on this site comes from changing how work is scheduled - how many threads decode uses, which cores they run on. Nothing touches the weights or the sampler, so the obvious assumption is that only the wait gets shorter.

That assumption is not free. ggml splits a row's dot product across threads and sums the partials, so the thread count fixes the order of a floating-point reduction - and floating-point addition is not associative. Different order, different last bits; different last bits in a logit, and a near-tie token flips. This project already depends on the effect being real: the perplexity sweep pins its thread count for exactly this reason. It had never been measured.

Dimensity 7300, Llama-3.2-1B Q4_0, greedy decoding at a fixed seed, perplexity over 12 chunks of wikitext-2. Naive against Auto is the pair that matters - the only comparison where the reduction width actually changes, and also the baseline the decode gain is measured from. Same 96 tokens byte for byte, same twelve per-chunk values, final PPL 19.2128 on all four arms.

Everything on this site optimizes the CPU. That was a design decision, and until it is measured it stays a design decision - "we chose the CPU" and "the CPU is the right choice here" are different claims and only the second is evidence. The phone has a Mali GPU advertising Vulkan compute, so the alternative is real rather than hypothetical.

30.3× slower on prompt, 1.9× slower on decode. Same binary, same Llama-3.2-1B-Instruct-Q4_0, same PP 512 / TG 128, three repetitions, cooldown before each arm. Not a close call that thermal noise could flip.

And the GPU says why itself, in the line it prints at startup:

Mali-G615 MC2 | uma: 1 | fp16: 1 | bf16: 0
            | int dot: 0 | matrix cores: none

No integer dot product and no matrix cores. Q4_0 inference is integer matmul, and that is exactly the instruction class the ISA ladder shows is worth 4.79× on this CPU. The same workload has a hardware path on the CPU and none on the GPU, so it falls back to generic shader arithmetic. The asymmetry is the mechanism: prompt is compute-bound and collapses 30×, decode is bandwidth-bound and both processors wait on the same memory.

Every other number on this site is a constant factor - the same work, done faster. This one is different. Answering turn k of a conversation normally means re-decoding everything said so far, so time-to-first-token grows with the history: O(history). ENTITY saves the conversation's KV cache to a file and restores it instead, which is O(1) in compute and a file read in its place.

A single measurement cannot show a shape, so this sweeps history length. The instrument is llama.cpp's own --prompt-cache, which uses the same llama_state_* family the app wraps, so the result does not depend on ENTITY's UI. Each prompt is a strict prefix extension of the last, the way a conversation grows.

8.2× the history costs 7.6× the prefill - and 0.00 ms either way to restore it. One column has slope, the other is flat, which is the whole claim. The re-prime cost holds at roughly 9.2 ms per token across the range.

ENTITY ships one APK carrying seven CPU backends and picks one at startup by scoring each against the chip. The usual way to argue that is worth something is a cross-device chart - an Armv8.0 phone beside an ARMv9 phone - but that comparison is confounded by everything else that differs between two phones: bandwidth, cluster width, thermal budget, scheduler.

So instead: every variant, forced in turn, on one device. Same model, same four threads, same pinned cores, same cooldown before each arm. The only thing that changes is which instruction set the kernels may use. llama.cpp scores backends in whatever directory it scans, so each variant is isolated in its own folder holding exactly one libggml-cpu-*.so, and the backend that actually loaded is read back out of llama.cpp's own log rather than assumed.

CMF Phone 1, Dimensity 7300, Llama-3.2-1B-Instruct-Q4_0, PP 512 / TG 128, 4 threads pinned to the A78 cluster, 3 repetitions, 45 s cooldown before every arm.

Every optimization here works by reading and steering the hardware - which cores are fast, what they are clocked at, where a thread lands, what the battery is drawing. So: is that about arm64, or about Android exposing arm64?

There is a control, and it is the best one available. Apple silicon is also arm64 - same architecture, same big.LITTLE shape, same class of quantized model, very fast. It simply will not let an application see or steer any of it.

Four cease to exist outright; two survive only degraded. A QoS request is not nothing - it is just not verifiable, where Android reads the applied CPU mask back. And one mechanism gets better: Apple reports the P-core count exactly, where Android has to parse per-core max frequency and count what lands within 10% of the fastest - a heuristic this project has already had to correct once. The asymmetry is about steering and observing, not capability.