MXFP4, aka Microscaling Format for 4-bit Floating-Point, is essentially a very small floating-point format that borrows one big trick from signal processing and older “block floating point” designs, in that …
This post reports a small, benchmark run on a Raspberry Pi 4B (8GB RAM) device using llama.cpp across five compact GGUF models spanning ~270M to ~1.2B parameters, with aggressive quantization. The aim was to measure how …
Since our earlier “short horizons, fragile state, orchestration first” note 1, there have been more data points published and the picture is becoming a bit sharper, justifying a follow-up post. We now have: A …
On-device LLMs are compact, optimized large language models that run directly on local hardware like smartphones or edge devices, instead of on a remote cloud server. This allows for privacy, since data stays local and …
For pragmatic reasons, Ollama has been the default local backend in our prior benchmark runs. Our recent article on Ollama and Open WebUI practices1 illuminated the need for an alternative. We’ve now added the …
We have seen examples such as Red Hat placing RHEL sources behind customer portals and contracts, and Canonical combining GPL code with contributor license agreements, trademark conditions, and a transition from GNU …
We can run serious long-context inference on commodity Apple silicon, but long context is hard. In this post we’ll touch on what Grouped-Query Attention (GQA) changes, and how to size a context window on ~ 64 GB …
nullbench is our ongoing evaluation run across a rotating set of current large language models1. We score models across practical dimensions—factual depth, reasoning, software engineering, compliance behavior, …
The compliance benchmark within nullbench serves as a fine-grained audit of a model’s capacity to follow instructions under constraint. Unlike accuracy tests, it measures the intersection of obedience, neutrality, …
The nullbench framework continues to evolve toward reproducible, interpretable behavioral analysis of language models. Since version 1.3, which introduced a revised judge panel anchored by glm-4-9b, recent short …