Networking in the Age of Inference: How the AI Network Maximizes Tokens per Watt – Hedgehog Triple-T
Inference now accounts for most AI compute being deployed, and it uses the network very differently from training. Marc Austin, Founder and CEO of Hedgehog, explains why every LLM request runs in two phases, prefill and decode, that stress hardware in opposite ways, and why disaggregating them onto separate pools turns the KV cache handoff into a networking problem measured in gigabytes per request and terabits per cluster. Drawing on SambaNova’s live Computex demonstration of heterogeneous disaggregation across NVIDIA and SambaNova accelerators on a Hedgehog fabric, Marc shows what the AI network must deliver to make mixed-accelerator inference work.
Marc Austin is the Chief Executive Officer and founder of Hedgehog. Marc is a fox who knows many things and a hedgehog who knows one big thing. As a Hedgehog he knows that millions of cloud operations teams will use Hedgehog AI networks to train and fine tune AI models for inference at the data edge. As a fox he knows many things from his experience leading mass-scale automation strategy at Cisco, Internet of Things networking at Jasper, digital media delivery at Amazon, mobile application development founding Canvas, the birth of smartphones at AT&T, early mobile ride sharing founding Mobiquity, internet search at Infoseek, e-commerce at Internet Shopping Network and leading people through adversity in the United States Army.
