We revised nullbench with a stability metric that avoids collapse on single spikes and a category set split into more granular writing style and language comprehension tracks. The overall portfolio remains the same1, but …
Large language models capable of acting as agents introduces a new layer of risk. These systems are more than passive text generators but are often equipped with tools and can increasingly get wired into developer …
Control of execution defines ownership. When a large language model file sits on your machine, it runs the same way until you decide to replace it. No silent updates, no routed experiments, no persona patches. When …
Evaluation of large language models (LLMs) is often presented as a leaderboard problem, ranking systems by performance on tasks with clear, objective answers. Prior work showed that such leaderboards hide collapse …
Nullbench is a controlled, reproducible benchmarking framework for large language models (LLMs) designed to isolate inherent response tendencies by evaluating models in a zero-context environment. Unlike ubiquitous …
For a number of years, LLM coding assistants are now marketed as labor multipliers. Investors pitch productivity gains, startups chase valuations, and executives present slides about flatter engineering teams. The claim …
It is well understood that large language models overgeneralize, carrying patterns too far and turning small ambiguities into catastrophic failures. The April 2025 study by Peters and Chin-Yee1 tested large language …
Star Trek showed computer systems breaking in ways that once looked cartoonish: the machine takes a command too literally, or it finds a boundary no one anticipated, and suddenly the ship is compromised. The year is 2025 …