GPT-OSS-20B Sampling & Prompting for Style Control
Best practices for sampling parameters and system prompt design to optimize style compliance and throughput with local LLMs like GPT-OSS-20B.
Execution Is Cheaper, Comprehension Is Not
LLMs have collapsed the cost of execution, but verification, comprehension, and ownership remain the binding constraints in software engineering and adjacent knowledge work.
LLM Fingerprints v1.5: Redistribution with 4chan Data
The latest nullbench run shows no step-function gains in base model capability. Instead, fine-tuning, abliteration, and long-context extensions primarily redistribute behavior rather than raise ceilings. The real …
Agentic AI Raises The Floor More Than The Ceiling
AI agents make easy software tasks cheaper but don’t automate hard engineering. This post tries to explain why autonomy claims overreach and where AI actually helps.
2025 Learnings on LLMs and Software Work
2025 suggests that the gap between expectations and measured outcomes around LLMs in software engineering is no longer subtle. Large investments and confident narratives imply broad productivity gains or partial task …
Block-Floating FP4 for Local Inference in llama.cpp
MXFP4, aka Microscaling Format for 4-bit Floating-Point, is essentially a very small floating-point format that borrows one big trick from signal processing and older “block floating point” designs, in that …
Raspberry Pi Inference: Tiny Quantized Models at the Edge
This post reports a small, benchmark run on a Raspberry Pi 4B (8GB RAM) device using llama.cpp across five compact GGUF models spanning ~270M to ~1.2B parameters, with aggressive quantization. The aim was to measure how …
Short Model Horizons Revisited
Since our earlier “short horizons, fragile state, orchestration first” note 1, there have been more data points published and the picture is becoming a bit sharper, justifying a follow-up post. We now have: A …
Tiny Quantized Models On Device
On-device LLMs are compact, optimized large language models that run directly on local hardware like smartphones or edge devices, instead of on a remote cloud server. This allows for privacy, since data stays local and …
Switching our Inference Backend from Ollama to llama.cpp
For pragmatic reasons, Ollama has been the default local backend in our prior benchmark runs. Our recent article on Ollama and Open WebUI practices1 illuminated the need for an alternative. We’ve now added the …