Daily Digest — 2026-08-13

Quoting Florian Herrengt

Simon Willison · 2026-08-12 · 1 min read

Heavy AI-assisted development creates systems where no engineer actually understands the codebase, leading to a compounding problem where even AI tools can't diagnose bugs in the convoluted architecture they helped create. This matters because it reframes AI coding assistance not just as a productivity tool but as a potential source of organizational cognitive debt, where speed of generation outpaces human comprehension and leaves teams unable to reason about their own systems.

AI Agent Cost: Why Price per Token Is the Wrong Metric #AIAgents #AgenticEngineering #shorts

Agentic Engineering · 2026-08-12

This video challenges the common misconception that cheaper AI models are always more economical for agentic systems. It argues that focusing solely on price per token is an isolated and misleading metric. Instead, the true cost should be measured by cost per completed useful task, which integrates factors like model capability, agentic effort, token usage, latency, and task success probability. A more expensive, capable model might ultimately be cheaper per useful task if it completes work efficiently in fewer iterations, whereas a cheaper model could incur higher overall costs due to repeated attempts and user corrections.

read more

The video discusses a critical shift in how we should evaluate the cost-effectiveness of AI models, particularly in the context of agentic systems. Traditional cost evaluation often fixates on the price per token, assuming that a lower token price equates to a more economical choice. However, for agents designed to perform tasks, this metric is fundamentally flawed.

Instead, the presenter advocates for a metric of cost per completed useful task. This broader perspective recognizes that the actual cost isn't just about the raw compute or token usage, but about the efficiency and success rate of the agent in achieving a desired outcome. A cheaper model in terms of token price might actually end up being more expensive overall if it struggles with a task. For example, a cheaper model might: Attempt a task (initial token cost). Explore wrong files or paths, incurring more token usage for unproductive searches. Make a change that fails initial tests, leading to further token consumption for debugging and retry attempts. Require user correction, introducing human time and effort costs that aren't factored into token price. * Ultimately complete the task after several loops of trial, error, and correction, significantly escalating the total tokens used and time spent.

Conversely, a stronger model, despite having a higher token price, could prove to be more economical. Such a model might: Understand the architecture of the system more effectively, leading to more direct and efficient problem-solving. Make the right tool calls on its first attempt, avoiding unnecessary iterations. * Complete the task in one pass, drastically reducing the total number of tokens consumed and the overall time to completion.

This leads to the concept of agent economics as an optimization problem with five key inputs that need to be balanced: 1. Model Capability: The inherent intelligence and understanding of the AI model. 2. Agent Effort: How many steps, iterations, or internal thought processes the agent needs to complete a task. 3. Tokens: The raw token usage, influencing direct API costs. 4. Latency: The time taken for the agent to complete a task, impacting user experience and operational efficiency. 5. Success Probability: The likelihood that the agent successfully completes the task as intended, reducing the need for retries or human intervention.

The objective is to maximize the probability that a useful task succeeds while minimizing the overall cost across these dimensions. The presenter concludes that relying solely on token pricing as an isolated metric is becoming increasingly less useful for evaluating agentic systems. The primary unit that truly matters is cost per useful work completed. Once this comprehensive measurement is adopted, many initial assumptions about which models are 'cheap' or 'expensive' may completely flip, revealing that investing in a more capable (and nominally more expensive per token) model can lead to significant overall savings and efficiency gains for agentic workflows.

There are no lossless transformations of natural-language text

Simon Willison · 2026-08-11 · 2 min read

Every AI rewrite of your text loses information because the model lacks your precise mental model of what you meant, making LLM-polished docs subtly wrong in ways you may not notice. For a senior engineer writing design docs, RFCs, or incident postmortems, this means you're accountable for every sentence an LLM generates in your name — "the AI wrote it" is not a valid response when a colleague questions your reasoning.

How to use AI on a file you can't upload #AI #privacy #productivity #datasecurity #AItools

Nate B Jones · 2026-08-10

This video emphasizes the crucial relationship between AI-driven intelligence and data privacy. It advocates for a user-centric approach where privacy is integrated seamlessly into AI workflows, rather than being an afterthought. The core idea is that AI's utility relies on accessing information, but this access must be managed securely and responsibly. The speaker posits that AI tools should be designed to handle sensitive data appropriately, allowing users to leverage AI without compromising privacy, thus enabling intelligence and safety to coexist effortlessly.

Hello World (xpost from substack)

Charity Majors · 2025-12-19 · 5 min read

TLDR: After pulling back from social media for years, Charity Majors realized she'd been neglecting a core professional responsibility: actively shaping technical discourse around observability and AI in public. She's moving to Substack despite criticism that the platform tolerates extremism, concluding it's "light dove grey" rather than beyond the pale — and that the career obligation to re-engage outweighs the platform ethics concern.