What an amazing amount of people coming out for the #OCPGlobalSummit executive sessions! Just off the stage, Hojun Shim delivered an incredible session: “Scaling Innovation for the AI Era: Keep Your Compute Running at Full Throttle with Highly Scalable Memory.” The numbers shared on stage really put the hardware challenge of the agentic era into perspective: 💠 Context dominates: In the agentic era, it is the context—not the model—driving memory traffic. 💠 Up to 6x Growth: Gigabytes read per generated token have surged roughly 6x in just two years. 💠 Up to 60x Growth: The KV cache created every month has grown 60x, completely outrunning current compression techniques. While capacity has grown across every tier over the same period, it still hasn't kept up with demand. To close this gap, Hojun shared how Samsung is pioneering solutions across three core architectural pillars – Speed, Memory Expansion, and Persistence. An impactful session on what it takes to scale open AI infrastructure for the road ahead. https://lnkd.in/eiyB_FpV #SamsungSemiconductor #SamsungMemory #KVCache #Tokenization #SamsungMAXLab #SamsungGOST
Tackling the hardware bottlenecks of the agentic era head-on! Fantastic insights from Samsung Semiconductor at the OCP Global Summit on scaling memory bandwidth, capacity, and persistence for AI infrastructure.