Networking in the Age of Inference: How the AI Network Maximizes Tokens per Watt – Hedgehog Triple-T
Inference now accounts for most AI compute being deployed, and it uses the network very differently from training. Marc Austin, Founder and CEO of Hedgehog, explains why every LLM request runs in two phases, prefill and decode, that stress hardware in opposite ways, and why disaggregating them onto separate pools turns the KV cache handoff into a networking problem measured in gigabytes per request and terabits per cluster. Drawing on SambaNova’s live Computex demonstration of heterogeneous disaggregation across NVIDIA and SambaNova accelerators on a Hedgehog fabric, Marc shows what the AI network must deliver to make mixed-accelerator inference work.
Marc Austin is CEO and co-founder of Hedgehog, the open source AI Network fabric. Marc has
over three decades of experience across cloud, networking, and enterprise software, Marc has
spent his career turning complex infrastructure into scalable, accessible platforms.Marc
previously held leadership roles at Cisco and Amazon, driving innovation in network automation,
IoT, and digital platforms.
Marc’s mission at Hedgehog is democratizing hyperscale infrastructure—bringing the power of
AI-ready networking to every enterprise through open, automated, and radically simpler
software.
