The speaker, Mat Gibiett from Dynatrace, highlights a critical challenge in AI-first engineering: AI agents lack a "nervous system" to understand the context of their operations, especially when replacing tasks previously performed by humans. This leads to issues such as agents getting stuck in repetitive loops, operational problems, and an inability to measure the real value they bring to users.
He emphasizes that while AI agents operate at speed, they often do so "without sight." This means they lack real-time feedback from running software in development and production, preventing them from making decisions with full context and understanding the consequences of their actions. This missing context leads to issues similar to hallucinations in large language models, where agents generate outputs that seem plausible but are factually incorrect or inefficient in the given environment.
Bluebox by Dynatrace aims to solve this by providing AI agents with "eyes into production." The platform achieves this through several key capabilities:
Scans repositories: It automatically discovers services and their dependencies by analyzing code repositories.
Instruments with OpenTelemetry: It uses OpenTelemetry, an open-source standard for observability, to instrument applications and collect real-time telemetry data.
Builds a live services topology: It constructs a dynamic, real-time map of all services and their interdependencies within an environment.
Provides production runtime context: This is the most crucial aspect. Bluebox integrates this runtime context directly into the AI agent's decision-making process.
These capabilities address specific challenges for different roles within an engineering team:
Developers: Bluebox helps answer questions like "What did my agent actually build/change?" and "Why am I stuck in a loop?" By integrating production context into local testing, developers can understand the blast radius of their changes before merging, ensuring that new features or fixes don't negatively impact other services. This shifts testing from local validation to understanding systemic impact.
Team Leads: They can gain confidence in deploying AI-generated code. Bluebox helps handle operational issues by continuously monitoring production signals and surfacing incidents the moment they appear. It allows for investigation with full runtime context, enabling proactive problem-solving rather than reactive troubleshooting. This ensures developers remain focused on building rather than endless debugging.
* Heads of Engineering: Bluebox addresses concerns about whether coding agents are building the right things and if they bring value to users. By providing comprehensive context, it enables measurement of AI agent value. Furthermore, it fosters full trust in the code being shipped by the team, as it can detect root-cause issues in code changes, powered by causal AI, before they escalate.
The speaker introduces an "Autonomy Ladder" concept, illustrating how Bluebox facilitates controlled automation:
Level 1 (Observe only): Agents only monitor and collect data, providing insights without taking action.
Level 2 (Suggest Fix): Agents detect issues and suggest potential fixes, but a human engineer reviews and approves the changes.
Level 3 (Open PR): Agents can automatically open a Pull Request (PR) with a suggested fix, but human intervention is still required for review and merge.
Level 4 (Autonomous PR): The highest level of autonomy, where agents can automatically create, test, and merge fixes without human input, given they have sufficient context and trust.
Bluebox ensures that human engineers remain in control throughout this process. It provides evidence-backed plans attached to every PR, shares the blast radius of changes before merging, and ultimately requires human review and merge decisions. It also confirms that fixes have held post-deployment.
Critically, Bluebox is built on OpenTelemetry standards, meaning it does not introduce new tools to learn for developers, easing adoption. By planning before prompting, building with non-frontier models (when appropriate), and reducing iteration and token consumption, Bluebox aims to make AI agent development more efficient and effective. This approach transforms AI development from a "prompt and pray" method to a data-driven, context-aware engineering discipline, ultimately helping agents ship code that engineers can truly trust to production.