Soheil Feizi introduces Verifiable Continual Learning (VCL) for AI agents, drawing an analogy to how humans learn from experience by acting, getting feedback, and improving without forgetting. The goal of VCL is to enable AI agents to continuously improve from their experiences without forgetting past learnings.
The architecture of an AI agent is typically broken down into three main layers: the Model (e.g., LLMs, weights, model selection), the Harness (e.g., prompts, skills, tools, code, workflow), and Memory (e.g., in-session state, persistent knowledge). The agent interacts with the World, which includes users, tools, and data policies, generating logs and outputs.
Feizi identifies two fundamental challenges in continual learning:
1. Getting Feedback (P1): How do we know if the agent performed well, and if not, what should it have done instead? In development, this is often handled with curated benchmarks and evaluators. In production, however, raw logs are not explicit feedback. Feedback can be generated automatically by an LLM or code analyzing the log, which scales to every session. Alternatively, human experts can provide critical feedback on a lower volume of sessions, catching subtle correctness, policy, and taste issues. Even with logs plus feedback, a significant gap remains: it's not directly testable.
2. Agent Optimization (P2): Given feedback, which layer or components of the agent should be changed, and how? Simply having log+feedback isn't enough; what's needed is a replayable learning environment. This environment is an inferred distribution that replays what happened and defines what success means. It comprises: Observed trace + feedback (what happened), Mocked/real tools (what the agent can call), a Synthetic user (what interaction repeats), and Evaluators (what success means). The output of this environment is executable, allowing candidate agents to be run against it, and fixes kept only if they pass.
Feizi outlines three layers for improving the agent:
Model: Updating the weights of LLMs or other models, or selecting different models. Techniques like Supervised Fine-Tuning (SFT) (imitating correct trajectories with labeled examples) and Reinforcement Learning (RL) post-training (sampling, scoring against a reward/preference signal, reinforcing what wins, e.g., DPO, GRPO, RLVVR) are used. LoRA (Low-Rank Adaptation) can limit the set of parameters that change, leading to cheaper and safer updates. Model updates are generally the most expensive option and require benchmarks + evaluators.
Harness: Editing prompts, skills, and code around the model. Trace-to-harness involves a coding agent rewriting prompts, adding tools, or patching workflows based on log+feedback. This is mostly vibe-based as there's no test to confirm the change helped. GEPA (Generalized Evolutionary Programming for Agents) & prompt search mutates prompts, scores candidates, and keeps winners through evolutionary optimization. This is testable but requires a benchmark to score against, which means creating a replayable learning environment.
* Memory: Storing facts and learned skills so the agent doesn't rediscover them. Information memory (e.g., Letta, mem0) stores facts or corrections (e.g., "always confirm the date before booking"). Skill distillation (e.g., skills.md) compresses successful trajectories into reusable "how-to" packets. Memory updates are the cheapest and fastest and work directly on log+feedback, but are usually unverified.
A good learning engine asks for the smallest durable change at the right layer.
Feizi introduces four principles for Verifiable Continual Learning (VCL):
1. Replayable: Turn a log + feedback into a testable learning environment. This fixes the fundamental problem of getting actionable feedback.
2. Holistic: Route each fix to the right layer—model, harness, or memory. A single failure can have multiple causes and repairs, so the fix must be applied where it most effectively addresses the root cause with the smallest durable change.
3. Lifelong: Improve without regressing on what already worked. This means integrating regression-aware learning within the agent optimization loop, not as a post-hoc check. The goal is to maximize performance on new failures (E_k+1) subject to no regression on past successes (E_1...E_k).
4. Efficient: Pick the smallest fix that works, so the loop runs continuously. Updates should be prioritized based on cost: memory writes (cheap) -> skill/prompt edits (low-mid) -> search over harness (mid) -> model updates (expensive).
RELAY's learning loop incorporates these principles: Signals (logs, feedback, prompts) are transformed into Replayable learning environments. Root-cause analysis routes the fix to the correct layer (Holistic). Regression-aware optimization ensures Lifelong improvements. The process generates a Reviewable, versioned update, closing the loop.
Feizi demonstrates VCL in practice with a Meridian Support Agent, a reproducible testbed for continual learning in a tool-using support agent. This benchmark features a single source of truth (fixed company policy and database define correct actions) and interacting policies (refunds, escalation, etc.). It uses deterministic evaluators, treats decisions as tool calls, and is regression-sensitive by design.
An example illustrates creating a learning environment from a prompt (e.g., an adversarial customer demanding an unauthorized refund). RELAY CLI creates a simulator (persona, intent, mocked/real tools) and evaluators (pass/fail metrics with feedback), all from one interactive command. Simulating the current agent shows struggles (e.g., failing to route escalation, latency budget issues). Running `relai optimize` improves the average score significantly (e.g., +10% average improvement, 97% average score from 87%). The change, such as canonicalizing refund-escalation arguments, is applied on the live tool and subject to online regression control. This demonstrates compounding lifelong agent improvements where each update is tested, every gain is measured, and nothing that already worked is broken.
In summary, agent continual learning is not just model fine-tuning; useful updates can occur in the harness or memory. Production logs are not learning environments and must be transformed into replayable tasks. The frontier is regression-aware continual improvement, fixing new failures while verifying old ones are preserved. Verifiable Continual Learning = replayable + holistic + lifelong + efficient.