About
During my PhD I researched language design and implementation at University of Milan. I have worked at UniCredit Bank’s R&D department, and at Red Hat on Drools and Kogito; I joined Tetrate to work on the Wazero WebAssembly runtime and I continued work on Wazero, Chicory, Extism, XTP, and other Wasm-related technology at Dylibso. Today I am back at Red Hat where I joined the AI Engineering organization to work on llm-d, the distributed LLM inference framework!
Session
Token My Breathe Away: Distributed LLM Serving with llm-d
TalkChasing LLM providers is a game of cat and mouse: they often change billing strategies and rate limits, they sunset models and rename tiers. And you have to put trust into handing over your customers' data. But what is the alternative? Serving LLMs in production is a real engineering challenge: hardware is expensive, resource utilization is unpredictable, latency is highly variable. Becoming a "token producer" demands a radically different approach to infrastructure. In this technical session, we explore llm-d: an open-source, Kubernetes-native, distributed inference framework. llm-d's workload-aware router schedules requests by factoring in KV-cache state, load and latency; moreover, llm-d supports advanced optimization techniques, such as offloading KV cache to the CPU, and disaggregating the prefill and decode inference phases to improve throughput. Join us to learn how to serve LLMs efficiently and reliably on your very own infrastructure!
Speaking at
- View event →
Cloud Native AI Summit — Paris
December 2–3, 2026