Is memory the moat?
Wafer 2 months ago 25
Open source models like Kimi K3 are reaching frontier capability levels but require massive parameter counts (2.8 trillion), making AMD's MI355X GPU a cost-competitive alternative to NVIDIA's B300 for serving them. On a benchmark with 1,024-token input and 400-token output, the MI355X achieved 952 tokens per second per node at 48 tokens per second per dollar, compared to the B300's 33 tokens per second per dollar, despite some software framework gaps. As AMD ships better day-zero support for these large models and engineers close optimization gaps, NVIDIA's traditional advantage in software maturity becomes less defensible for inference workloads.
