Skill Descriptions Are Tricky

Theo - t3.gg · 2026-08-11

When defining the description for a skill in an AI agent, it should function as a set of trigger keywords rather than an elaborate explanation of what the skill does. The description is always inserted into the model's context, even if the skill isn't used, making concise and relevant keywords crucial for efficient tool selection. Overly verbose descriptions can lead to the model misinterpreting or not utilizing the skill correctly.

Better AI Model or More Agent Effort? Diagnose Coding Agent Failures #claude #agenticengineering

Agentic Engineering · 2026-08-11

This video, by a self-proclaimed "Agentic Engineer," argues that when an AI coding agent fails, engineers often incorrectly assume the problem lies with the model's capability. Instead, the issue might be an effort problem, meaning the agent didn't perform enough steps like reading files, running tests, or checking hypotheses, despite having a capable model. The key takeaway is to distinguish between model capability (what the agent knows) and effort (how much work it's willing to do) when diagnosing failures in AI agents, advocating for configuring both rather than solely focusing on model upgrades.

TDD inside the agent loop - theater or actual value?

Martin Fowler · 2026-08-11 · 22 min read

TLDR: Forcing AI agents to follow TDD inside their own loop offers no meaningful quality advantage — and may slightly hurt outcomes. The core reason: TDD's incremental approach prevents agents from doing upfront design, which is actually where they produce better data models and edge-case coverage. Non-TDD runs naturally sketch the full architecture before writing any code, whereas TDD locks agents into whatever shape the first test happens to define.

Strix launches AI-powered hacker agents to test app security #Strix #AIpentesting #cybersecurity

AI Honeycove · 2026-08-11

Strix is an open-source AI-powered pen-testing tool that emulates real hackers to identify and exploit vulnerabilities in web applications. Unlike traditional scanners that provide vulnerability lists, Strix deploys specialized AI agents to run live targeted attacks, providing working proofs of concept for each found vulnerability. It also suggests fixes, enabling developers to thoroughly secure their code before deployment.

Introducing Muse Glimmer

Simon Willison · 2026-08-10 · 4 min read

TLDR: Meta released Muse Glimmer, a 30B open-weights model under Apache 2.0 — a cleaner license than previous Llama releases — optimized for agentic tasks, tool use, and multi-step reasoning. At ~18GB it runs comfortably on 32GB+ RAM machines alongside other apps, and Willison's hands-on testing confirms solid codebase exploration and detailed vision capabilities.

CatCoder beats Claude and GPT on coding benchmarks #CatCoder #codingAI #artificialintelligence

AI Honeycove · 2026-08-10

The video highlights KwaiKAT's KAT-Coder-Pro V2.5, an agentic AI coding model that has outperformed models from OpenAI and Anthropic on coding benchmarks. Unlike traditional LLMs that only read code, KAT-Coder-Pro V2.5 trains by executing code in real-world projects and iteratively fixing errors, mimicking a human programmer's workflow. This novel training approach allows it to build complex projects like a playable Minecraft version and fix production bugs efficiently.

read more

The video introduces KAT-Coder-Pro V2.5 by KwaiKAT, a groundbreaking AI coding model that has demonstrated superior performance over established models like OpenAI's GPT and Anthropic's Claude on various coding benchmarks, including SWE-Bench Pro and PinchBench. This is particularly noteworthy because KwaiKAT is a lesser-known entity in the AI space.

What sets KAT-Coder-Pro V2.5 apart is its unique training methodology. Unlike most coding LLMs that primarily learn by passively consuming vast amounts of finished code from the internet (effectively memorizing code patterns without understanding execution), KAT-Coder-Pro V2.5 is designed as an agentic coding model. This means it is trained within verifiable environments and engages in a process-aware trajectory with harness randomization.

The model's training process involves being placed inside 100,000 real-world software projects. In these environments, KAT-Coder-Pro V2.5 doesn't just generate code; it executes the code, observes the outcomes (including compilation errors, runtime bugs, and test failures), and then iteratively debugs and refines its code. This trial-and-error, feedback-driven learning closely mirrors how a human software engineer develops and debugs code. It learns not just what code looks like but also how it behaves and interacts within a live system.

Key achievements highlighted in the video include its ability to build a playable version of Minecraft in 1300 lines of code and fix a real-world production bug in just 1 minute and 20 seconds. These accomplishments underscore its practical utility and advanced problem-solving capabilities in software development.

The technical architecture likely involves a Rollout Engine for generating code, an Environment Module (Sandbox/Container) for execution, and an Experience Buffer to feed back results into a Train Engine for continuous improvement. This closed-loop learning system is crucial for its agentic capabilities. The model's success suggests a shift in AI coding model development from purely generative approaches to more interactive, execution-based learning, which could have significant implications for developer tools and automated software engineering in the future.

The real #AI bottleneck is your attention #agenticengineering #openai #anthropic #vibecoding

Agentic Engineering · 2026-08-07

This video argues that as AI automates more execution tasks, the true bottleneck shifts from intelligence to human attention. While AI handles routine work, humans become responsible for understanding, prioritizing, and making critical decisions for multiple AI-driven projects. Therefore, agentic engineering will value focused human attention more than ever, as it's the scarce resource needed to steer complex, automated workflows.

Why the Frontrunners Say Coding Is Solved BUT Engineering is Not

Beyond Coding · 2026-06-17 · 46 min read

Two senior engineers (Jeroen Gordijn and Jeroen Dee) argue that software development — the act of writing code — is already solved by AI agents, while software engineering (architecture decisions, system design, cloud infrastructure, integrations) remains a human domain. Both stopped writing code entirely in late 2024 and now operate at a higher abstraction layer, using spec-driven development to guide agents rather than typing syntax. The conversation covers practical challenges of AI-generated code at team scale: PR review bottlenecks, code quality conventions, and the emerging concept of the 'Dark Factory' — fully automated software production pipelines with no human in the loop.

read more

The development vs. engineering distinction is the central thesis. The guests argue that "software development" — the mechanical act of typing code — has been commoditised since roughly Claude Opus 3/4 (one guest cites November 24th of the prior year as his personal inflection point). "Software engineering," however, remains relevant: deciding where to store artifacts, which cloud provider to use (Azure vs. AWS), how to structure integrations, what the system should do and how it should behave. This is a deliberate conceptual separation worth internalising — the craft is shifting from implementation to specification and architecture.

Both guests stopped writing code entirely, not overnight but through a gradual learning process. One describes the early transition as awkward — reverting to directing the agent line-by-line ("you see that file, on line 12, change that") before learning to operate at a higher level of abstraction. The implication is that engineers who resist this shift are not just slower — they are actively misallocating cognitive effort on something the model can do. The guests treat language-specific syntax knowledge (their example: Go) as no longer a meaningful differentiator.

Pull request review as the new bottleneck is a key operational finding. When the team embraced coding agents — including product owners generating code — PR volume spiked rapidly and human reviewers became the constraint. This is a concrete, reproducible problem any team will hit as agent adoption scales. Their proposed solution is risk-tiered review: high-risk PRs still get human eyes, low-risk changes are routed toward automated agent review. The longer-term ambition is to remove humans from the review loop entirely for a significant fraction of changes.

The Dark Factory concept is borrowed from manufacturing. In a conventional factory, lights are on because humans need to see to work. A dark factory runs entirely on robots — no lights needed. The software analogy: a pipeline where a specification goes in one end and working software comes out the other, with no human intervention in between. The guests are actively experimenting with what this looks like inside their engineering department. It is not fully realised yet, but it is the directional target.

Spec-driven development is the methodology they use to get closer to that target. The contrast they draw: you can open an agent harness and say "build me this feature" and get something, but it is essentially throwing a dart — you might hit the bullseye, probably not. Investing time upfront in a detailed specification — what the feature should do, how it should behave, what constraints apply — dramatically improves output quality and alignment with the existing codebase. One guest notes that as he has gotten better at speccing, he finds himself looking at the generated code less and less, because it is increasingly correct by the time it arrives.

Code quality and conventions surface as a tension point. One guest challenges the premise that code quality matters if you can throw away and regenerate a codebase from specs: if the spec is the source of truth, why care about the quality of the generated artefact? The counter-argument is pragmatic — until full regeneration from spec is reliably possible (acknowledged as not yet achievable), you still need code that is simple and easy to change, because you will be iterating on it. Conventions and quality standards are therefore still worth capturing, likely in the spec or as part of the harness configuration rather than enforced through manual code review.

The pull request comprehension question — should engineers understand agent-generated code before merging it — is raised but not fully resolved. The guests acknowledge the theoretical appeal but point to a scaling problem: as teams generate increasing volumes of code, full comprehension per PR becomes practically impossible. Their answer is to shift the verification question from "do I understand this code" to "does this code do what the spec said it should do" — i.e., move from code review to behaviour/outcome verification. This is a significant cognitive and process shift for teams used to treating PR review as the primary quality gate.

The broader career implication is stated bluntly: engineers who do not learn to work with these tools risk being left behind — the train-departing-the-station metaphor is used explicitly. The guests frame this not as a distant concern but as something already in motion, with the transition from developer-as-coder to developer-as-architect/specifier happening now.

I spent 3 days at MIT... the robot hype is worse than you think

Fireship · 2026-08-11

This video, titled "Robots are not what you think they are" from Fireship, critiques the hype surrounding humanoid robots. While recent demonstrations from DeepMind and 1X showcase impressive capabilities like multi-robot collaboration and object manipulation, the reality of deploying general-purpose humanoid robots for complex tasks like household chores is still decades away. The primary challenges are achieving human-level dexterity consistently and the lack of real-world training data, forcing a reliance on simulations, and a fundamental difference in how large language models (LLMs) and robotic policies operate regarding discrete vs. continuous outputs. The video concludes that despite rapid AI advancements, the "easy problems" for humans (like dexterity) remain "hard problems" for robots, a concept known as Moravec's Paradox, creating significant opportunities for software developers in robotics.

read more

The video opens by highlighting recent impressive demonstrations in humanoid robotics: Google DeepMind's Gemini Robotics 2 and Silicon Valley startup 1X's Neo robot. Gemini Robotics 2 showcases a humanoid robot performing tasks like walking, crouching, tying knots, screwing in lightbulbs, and multi-robot collaboration for cleanup. 1X's Neo robot is shown playing Xbox and opening snack bags. These demos suggest a future where robot slaves are imminent.

However, the video immediately counters this perception by revealing that robotics researchers at MIT believe a maid robot is still 10+ years away, and that's being optimistic. The core problem is not complex locomotion (walking, backflips), which is largely a solved problem for humanoid robots, but multi-finger dexterity and consistent reliability. When examining the fine print of these robot demos, multi-finger dexterity success rates range from 0% to 90%, which is unacceptable for real-world deployment (e.g., a maid dropping dishes 10% of the time).

This discrepancy between perceived and actual capabilities is explained by Moravec's Paradox: what is easy for humans is hard for computers, and vice versa. While AI can easily beat grandmasters at chess, it struggles with tasks a two-year-old can do, like stacking blocks. Evolution has spent 500 million years optimizing human sensory-motor skills, while reasoning is a relatively recent development.

From a technical standpoint, a robotics policy is fundamentally different from a large language model (LLM). LLMs generate discrete tokens and can take as much time as needed to produce text, and inaccuracies don't typically cause physical damage. A robot, however, needs to emit a continuous stream of precise values (joint angles, torques) hundreds of times per second to dozens of motors, all in unison. Even a slight error can lead to the robot failing or damaging itself, highlighting the unforgiving nature of the physical world.

Another major challenge is data. LLMs learn by ingesting vast amounts of existing text data (the entire internet, books). For robots, a comparable real-world data source for physical interaction simply doesn't exist. Researchers are attempting to overcome this by creating simulations or synthetic data, akin to a robot learning to fly a real plane after using a flight simulator. However, researchers are still unsure of the optimal training methods.

The current debate in robotics training revolves around imitation learning vs. reinforcement learning. Imitation learning involves a human tele-operating a robot to teach it behaviors, which the robot then attempts to clone. This method is simple but difficult to scale. Reinforcement learning (RL) involves letting the robot learn through trial and error, providing reward signals for desired actions. While effective for some tasks (like Unitree robots learning Kung Fu), RL is currently not robust enough for safe, general-purpose robots.

The video also touches upon the commercial landscape, mentioning companies like 1X, Figure, and Tesla producing highly publicized demo videos. However, these robots are not generally available for purchase, and their real-world capabilities outside of controlled demos are unclear. Other commercially available robots like the Boston Dynamics Atlas (acquired by Hyundai and Google), Chinese Unitree G1 ($13,500), and AGIBOT A2 Ultra exist, but the overall robotics field for human replacements is smaller than the hype suggests.

In conclusion, the video emphasizes that human-like dexterity and reliable operation in unstructured environments are profound challenges. The prediction is that we might see GTA 6 before humanoid robots are commonplace in kitchens. This creates a significant opportunity for software developers to write the code that brings these robots to functional reality. The video concludes with a plug for Omnigent, an open-source meta-harness for AI agents, allowing orchestration of Claude Code, Codex, and custom agents, highlighting its sandboxed environment, shareable sessions, and Apache 2.0 license.

GitHub Models is now retired

Simon Willison · 2026-08-09 · 2 min read

GitHub Models, which offered a unified LLM API usable inside GitHub Actions via the built-in GitHub token (no separate API key needed), has been permanently shut down — likely because free/subsidized token costs became unsustainable as agentic AI usage scaled up. If you have any GitHub Actions workflows making LLM calls through GitHub Models, they are broken right now and need to be migrated to a direct provider API key like OpenAI or Anthropic with an explicit spending limit.