Zero Cold Starts: Serverless AI Agents on Akamai Functions — Thorsten Hans
This video introduces the Edge-Native AI Stack, a powerful combination of CNCF Spin and Akamai Functions designed for building ultra-fast AI Agents and MCP servers. The stack addresses critical infrastructure frictions like latency, cold starts, and vendor lock-in by leveraging WebAssembly (Wasm) on the server-side. Key takeaways include portability, speed, and a NoOps development experience, enabling developers to focus solely on business logic while Akamai handles global distribution and scaling.
read more
Thorsten Hans, a Senior Developer Advocate at Akamai, presents the Edge-Native AI Stack, a solution aimed at overcoming common challenges in building and deploying MCPs (Model Context Protocol) and AI Agents. These applications demand fast response times, but traditional region-bound architectures often fall short, leading to latency issues, long cold starts (from seconds to minutes), and overprovisioning which is financially and environmentally unsustainable. Furthermore, many existing solutions suffer from vendor lock-in due to proprietary runtimes and SDKs, making workload migration complex and costly.
The core of the Edge-Native AI Stack is the adoption of WebAssembly (Wasm) for server-side applications. Wasm binaries are super tiny, easy to distribute, fast to run, and portable, offering a significant advantage over containers and VMs.
The stack comprises two main components:
CNCF Spin: This is a developer tool for building server-side Wasm apps. It is vendor-agnostic and consists of three pillars: CLI: Addresses inner-loop concerns for developers. WASI Runtime: Based on Wasmtime, allowing applications to run locally or on constrained environments. SDKs: Language-specific SDKs (e.g., Rust, TypeScript, Python) to boost developer productivity for tasks like interacting with HTTP services or databases. Spin allows coding applications in any language that can be compiled to WebAssembly (WASI).
Akamai Functions: This is a truly serverless Edge Computing Platform built on Wasm. Key features include: Globally-Distributed: Deploy once, available across the globe. Fully-Managed: No knobs to turn, no cloud-infrastructure to manage. Security: Zero-trust & shared-nothing execution within the WebAssembly sandbox, preventing unauthorized external resource interaction. Built-in Key-Value Store: Also globally distributed and fully managed, automatically associated with the app, eliminating credential management. Zero Cold Start: Applications cold start in an average of 0.4ms, addressing a major pain point of traditional serverless and containerized solutions.
Demo: MCPs with wasmcp The demo showcases how to build and deploy an MCP server using `wasmcp`, a polyglot framework for composing Wasm-based MCP servers. Instead of hand-rolling an MCP server, `wasmcp` simplifies the process by allowing developers to focus on business logic (implementing tools, resources, and prompts) while handling non-functional requirements like authentication (e.g., OAuth2, OIDC). Developers can use off-the-shelf components, compose them together, and deploy them.
The workflow includes: 1. `wasmcp new --language typescript hello`: Creates a new TypeScript-based tools component (e.g., `hello`). 2. `make`: Compiles the TypeScript code down to WebAssembly, generating a Wasm component. 3. `wasmcp compose server dist/hello.wasm -o server.wasm`: Composes the Wasm component into a full-fledged MCP server. For demonstration, authentication was omitted, but `wasmcp` can integrate it. 4. `spin up -f server.wasm`: Runs the MCP server locally using the Spin CLI, making it accessible on `localhost:3000`. 5. MCP Inspector (Browser UI): Connects to the local MCP server, lists tools (e.g., `Example Tool`), and allows running them, demonstrating local functionality.
Demo: Serverless AI Agents with Vercel AI SDK For AI Agents, the process is similar, but leverages agentic SDKs like the Vercel AI SDK. The demo creates a simple game agent: 1. `spin aka deploy --no-confirm --build`: Compiles and deploys a Rust-based `decision-log-mcp` application (a slight variation, also defining tools and using a local SQLite key-value store during development). This process generates OCI layers, uploads them to Akamai Functions, and handles global distribution and health checks. 2. Deployment Time: The application becomes globally distributed and ready to use in roughly 60 seconds. Akamai Functions then provides a public URL for the deployed agent. 3. Client Interaction: Using a client like Zed, the user can add the deployed Akamai Functions URL as an MCP server. The client then discovers the defined CRUD tools (e.g., `insert_decision`, `list_decisions`). 4. AI Agent Interaction: The user can then interact with their IDE's AI agent (e.g., Claude Sonnet 4.6) and instruct it to use the deployed MCP tools. For example, by typing "Track a decision to use OAuth2.0 for authn", the AI agent identifies the intent, asks for permission to run the `insert_decision` tool, and the user approves, triggering the tool on Akamai Functions. Subsequently, the user can request "Give me all the decisions", and the AI agent runs `list_decisions` to retrieve the tracked decision. 5. Performance: The video highlights a request processing time of around 34-824 milliseconds for a full loop through the LLM, demonstrating the ultra-fast nature of Edge-Native AI with Akamai Functions. The underlying JavaScript runtime (StarlingMonkey) is roughly 10MB, and a Wasm application, even when written in JavaScript, performs faster than a native JavaScript application in terms of cold starts and runtime performance, due to the memory snapshot approach.
Final Takeaways Portability & Openness: WebAssembly enables compiling code once and running it everywhere, including the Edge. The stack is built on open standards (CNCF Spin, WASI, Bytecode Alliance). Speed: Akamai Functions eliminates cold starts, making AI tool calling instantaneous, with average cold start times of 0.4ms. * NoOps: Developers manage logic, not infrastructure. Akamai Functions handles containers, VMs, and global distribution, simplifying the deployment experience to a single `spin aka deploy` command.