Hrittik Roy

Inference on GPUs fails in repetitive ways: wrong image or artifact, mismatched CUDA or runtime, undersized GPU memory, bad resource requests, or a model that passes offline checks but regresses under real traffic. On a shared k8s GPU platform, those mistakes become multi-tenant incidents - noisy neighbors, OOMKills, SLO breaches, and rollbacks that waste accelerator time. This talk describes how one team built conformance for inference workloads, checks applied before production traffic, covering container and model artifacts, GPU capacity and visibility contracts, health and readiness semantics, and minimum observability (metrics/traces where used). Attendees leave with a practical checklist they can reuse: how to separate "builds" from "serving" conformance, how to catch regressions early, and how to align GPU scheduling and quotas with inference SLOs. We will share what worked, what did not, what teams pushed back on, and a short checklist for platform and app owners.