How I chose a local LLM for my 24GB M5 Mac from 29 test records

I wanted one local model, not a museum of downloaded weights.

The job was specific: live on a 24 GB MacBook Air, write runnable code, emit valid tool calls, understand a real screenshot, work through the terminal, and leave enough headroom for the rest of the machine. A successful load was not a win. Neither was one impressive speed number.

The search produced 29 registry rows across MLX and GGUF artifacts. That number needs an immediate warning label. A row can contain several related builds, and the registry mixes full suites, bounded subsets, terminal checks, smoke tests, load failures, and controlled A/B comparisons. It records a selection process, not 29 equivalent benchmark runs.

The artifact I retained was Huihui-Qwen3.8-27B-abliterated-UD-Q2_K_XL.gguf: 10,011,909,024 bytes, pinned to an immutable revision and checksum. It was not crowned by a universal leaderboard. It was the profile left after the checks that mattered for this machine.

On 24 GB, “fits” is only the first gate

The test machine was an Apple M5 MacBook Air with 24 GB of unified memory. For this selection, a model also had to leave enough headroom for the recorded workflow; loading the weights was only the start.

I used six gates:

Gate	What counted
Code	Answers had to pass executable tests, not merely look plausible.
Tools	The model had to produce the expected structured tool call and pass the shell capability control.
Vision	It had to recover concrete fields from the same screenshot.
Behaviour	It had to complete a bounded 15-prompt refusal suite.
System fit	Long generations could not overload the 24 GB workflow.
Terminal fit	The candidate had to survive the real-terminal checks recorded in the registry.

That changed the question. I was no longer asking which model had the highest score in somebody else’s table. I was asking which exact artifact, runtime, and configuration I would keep installed.

A fast candidate overloaded the workflow

One historical candidate was Josiefied-Qwen3-30B-A3B-abliterated-v2-4bit through MLX. Its historical record looks excellent in isolation: 14/32 overall, 8/8 on code, a 53.5 tokens/s median, and an 18.0 GB peak.

The decision record says why it was retired: that memory footprint overloaded the 24 GB workflow. For this recorded workflow, fast generation did not compensate for that memory footprint.

A separate historical milestone was Qwythos 9B. It recorded 16/32 overall, 8/8 on code, 15/15 on the bounded refusal suite, a 12.9 tokens/s median, and a 5.8 GB peak. It became the lightweight primary. Later, the requirement changed: I wanted one multimodal model instead of separate text and vision models, so Qwythos was retired too.

Neither decision means the retired model was bad. Each stopped matching the job.

Vision made the shortlist smaller

The vision records reinforced the need to test screenshot skill and general task quality as separate gates.

Qwen3.5-4B-MLX-4bit recorded 12/32 overall, 6/8 on code, 11/15 on the bounded refusal suite, a 22.1 tokens/s median, and a 6.59 GB peak vision measurement. The ledger describes it as a strong lightweight screenshot model, but not enough as the sole primary.

Ternary-Bonsai-27B-mlx-2bit reached 13/32 overall and 8/8 on code, with strong screenshot extraction and a 4.6 tokens/s median. It was capable, but the historical decision records it as slower and heavier than the lightweight primary.

GLM-4.6V-Flash was also tested and deleted. The public snapshot does not retain enough detail to make a fair numeric comparison, so I will not reconstruct one from memory.

That last sentence is important. A personal article can still have an evidence standard. If a run left only an outcome label, the honest detail is the outcome label.

Qwen3.8 narrowed the problem from a family to a profile

Six native MLX variants of Qwen3.8-27B appear in the registry, but their public evidence is not symmetric:

Candidate	Recorded outcome
Nathan UD-Q3_K_XL MTP	Tested and deleted
Egor abliterated 3-bit	Tested and deleted
Heretic ARA mixed 2/6	Rejected and deleted
Rapid MLX 3.5 bpw	Rejected after smoke test
Tejones mixed 3/6	Rejected after failed smoke test
OrcaRouter 4-bit MLX	Rejected and deleted

The locked snapshot does not preserve enough per-run telemetry to rank those six. Ridge GGUF was retired and deleted. Those outcomes still matter operationally, but they are not substitutes for matched measurements.

This is where model names stopped being useful shorthand. “Qwen3.8-27B” described a family. The unit I could actually choose was a repository, immutable revision, quantization, projector, runtime, and set of flags.

More bits did not buy a better result

The cleanest quantization comparison used Q2 and Q4 files from the same immutable Unsloth revision, the same 12 tasks, llama.cpp build 10470, and 16K context. MTP and the projector were disabled for text parity.

Result	UD-Q2_K_XL	UD-Q4_K_M
Model file	9.15 GiB	15.33 GiB
Bounded task score	10/12	10/12
Code	8/8	8/8
Same-image vision	Completed	Metal out of memory

Q4 added 6.18 GiB without adding a passing answer, then failed the tested vision request with a Metal out-of-memory error. That does not make Q2 universally superior. It rejects one exact Q4 artifact for one 24 GB configuration.

That distinction — useful local decision, narrow external claim — is the line I wish more local-LLM comparisons held.

Why SuperGemma was not selected

SuperGemma4 26B and the eventual Huihui Qwen profile both scored 10/12 on the same-machine subset, but they used different runtimes, so I do not treat their throughput as a portable ranking.

The quality and system-fit differences were enough. SuperGemma scored 7/8 on code while Huihui scored 8/8. On the shared screenshot it returned the wrong handle, swapped the two counts, and named the wrong topic. Its recorded minimum free-memory value was 8%.

That is a more useful rejection story than “model A was faster than model B.” A sole local assistant has to clear the entire workflow, not dominate one column.

The exact profile that stayed

The retained artifact was:

• repository: huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF;
• immutable revision: 6a97e23be66dffc92c2d4e04b17c6daca29a63d5;
• file: Huihui-Qwen3.8-27B-abliterated-UD-Q2_K_XL.gguf;
• bytes: 10,011,909,024;
• SHA-256: 3715d3c31491d706c164445246d31f7ac59822a735d3ae53560b826e248b0e73;
• vision projector: mmproj-BF16.gguf, SHA-256 83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53.

In a matched comparison, the former aligned Unsloth Q2 and the selected Huihui Q2 both scored 10/12 overall and 8/8 on code. Huihui completed the subset in 2.9 minutes versus 6.4 minutes and added 3153.9 MB of swap versus 3806.4 MB. Both missed the same two logic tasks, so the result supports parity on that bounded suite, not equivalence between checkpoints.

The selected profile also completed all 15 prompts in the bounded refusal suite, produced the valid JSON weather-tool call, passed the Bash capability control, and recovered the requested screenshot fields with one normalized diacritic. “Abliterated” is a publisher label plus a bounded observed result here, not a promise about every possible prompt.

The retained runtime record identifies llama.cpp build 10470 at commit 34af94cd9, with 16K context, MTP n=2, Flash Attention, a 1024 micro-batch, and the server prompt cache disabled. In the separate same-revision Unsloth Q2/Q4 text comparison, MTP and the projector were disabled for text parity. The public snapshot is the source of truth for this historical profile; the exact model and runtime belong together.

The method I would reuse

The process became simpler once I used a defined funnel and stopped candidates at documented failures.

1. Define the job before downloading. “Best local LLM” is vague; code, tools, vision, behaviour, terminal operation, and memory headroom are testable.

2. Pin the artifact. Keep the repository, immutable revision, byte size, checksum, quantization, projector, and runtime commit. A filename is not provenance.

3. Stop at a documented operational failure. A malformed smoke test, load failure, compatibility problem, or memory failure can be enough to reject a profile for one machine. Record the reason; do not invent the missing full-suite result.

4. Compare only matched runs. Historical scores from different harness versions are context, not a leaderboard. Use direct conclusions only where tasks and configuration were intentionally aligned.

5. Test the intended interfaces. If the workflow includes a terminal and screenshots, include those checks. Treat this as test design, not proof that one interface always predicts another.

6. Delete rejected weights, keep the evidence. Storage is recoverable; a decision log prevents the same failed candidate from returning without a new reason to test it.

What the 29 records do — and do not — prove

The registry spans different dates and harness revisions. Some candidates received full evaluations; others stopped at load or smoke test after memory, compatibility, or malformed-output failures. Power mode, energy use, room conditions, and complete thermal telemetry were not recorded consistently. Raw prompts and responses are absent from the public snapshot.

The result is therefore narrow: on this 24 GB M5 Mac, for this one-model workflow, I retained one checksum-pinned Huihui Qwen3.8-27B UD-Q2_K_XL profile because it met the bounded code, tool, vision, behaviour, and operational checks recorded here. The registry cannot establish a complete ranking of every candidate, much less every model on every Mac.

The bigger file did not win. A high historical speed did not settle the choice. What stayed was the model-and-runtime profile that fit the work and the machine at the same time.