It's mind blowing seeing these multi exaflop single rack systems.
The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022.
It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors.
Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.
Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a radical optimization for AI workloads, is sometimes considered wasteful now.
AI workloads have some truly peculiar and counterintuitive properties - the kind of things you might expect to see in biology instead of conventional computing. Intrinsic error tolerance, for one. It did necessitate some rethinking and reprioritization, and I'm not quite sure if we converged to the general shape of an "optimal" AI accelerator as of yet.
> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.
It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 workloads. I don't see anyone referring to these AI clusters as supercomputers, for the same reason they aren't referring to the racks as mainframes.
I wonder if there's a term for this kind of semantic narrowing?
NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a supercomputer but don't yet have the same preconceptions about AI factories.
I don't think that Nvidia single handedly caused the move away from the term supercomputer. But I think they had a hand helping push firmly in that direction.
It's definitely older than that. Originally, way back in the mechanical days when super-computing was hyphenated, it was about how many millions of instructions per second could be handled. That changed in the 1960s with a series of CDC mainframes that put all of their focus on floating point pipelines, moving the category to refer to floating-point crunchers used for scientific simulation (which is what the frontier of computing was used for). Eventually parallelism would be added in at some point, and the term has fossilized into a "monolithic cluster with massive FLOPS intended specifically for scientific simulation". Because for the longest time, that's who needed all the compute.
The PIM/m of the FGCS defied this a little, sometimes being called a "symbolic supercomputer", but never without the "symbolic" qualifier. It was the first time in history where the frontier of computational throughput wasn't about scientific simulations. It wasn't even about arithmetic!
Right now is also interesting, because the world's most powerful FLOPS monster monoliths aren't being used for scientific simulation. They fail at being supercomputers for that reason, but a computer scientist from the 60s would probably raise his eyebrow at that.
This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing" a specific precision, is that they are usually properly analyzed to determine the needed precision, and then run at that precision.
One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.
ancient intel did 80 bit fp until your intermediate result was written from registers to memory. Since you typically have no control over when that happens, even in very low level languages like C, the 80 bit precision mostly had the effect of making your code produce different results on different compilers. If you needed 80 bit precision and deterministic results you had to write the algorithm in assembly.
There are plenty of laymen that think "AI bubble pop" means that "AI" usage will go to 0 and then all of a sudden all these hardware and datacenters related companies are buying will become totally unused and flood the market.
but you have to rewrite all your software to ROCm. And ROCm, to this day, still sucks.
ZLUDA basically crash on high memory demand, and only accounted for ~70% of CUDA API coverage (and it is still buggy).
But hey, at least it does run on Rust-CUDA. I'm one of the few who ported it and fixed a few bugs on ZLUDA. I used it to run a simple SHA256 kernel and it ran sure, but I gave it up because of those fundamental problems on AMD GPUs. You can't believe how messy ROCm is. I wonder if Vulkan compute kernel using SPIR-V would be a better choice.
> I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process.
> Our actual experience was we had one engineer who start doing it. They spun up Claude, asked it, hey, bring up this machine, left it going over the weekend. And we ended up with a graph of the actual performance of our leading model on it, just going up and up and up over the weekend.
That only works if you have all the unit tests and making sure the behaviors between different GPUs are expected, and can tolerate difference in implementation details such as numerical stability and complexity for some functions. AMD and Nvidia are doing vastly different computation for different implementation to the same algorithm for their APIs (can't tell due to NDA, but one algorithm Nvidia did it using broadcast, while AMD is using tile), and that's one of the reason I gave up. And perhaps only for light kernels doing basic BLAS operations and GEMMs. With a standard to follow, that's one of places where it indeed is giving it a good chance, for example on Tensorflow with ROCm or PyTorch. You basically have an insurance of the behavior and a translation layer for ROCm and CUDA by Python.
But anything other than that, nah.
Edit: Maybe I remember what algorithm that is. It is a trick to do coding theory stuff by using NTT (Number Theoretic Transform, the friend of DFT, the Discrete Fourier Transform, but on a ring and hence applicable to finite field) on GPU to do computation over a GF(2^8) field, for my former employment but I went back to using GFNI. Still I have a NDA to adhere to and can't really explain what happens.
That, or, you know. Ask Claude Code to write all of those unit tests.
Variance in numerical instability across hardware is basically a given, because all the different accelerators implement their "fast paths" in slightly different ways. Modern AI just takes it on the chin.
Anthropic, for example, already has to support both Nvidia CUDA and Google/Broadcom TPUs - AMD aside. And I expect those two to have more architectural and software differences than AMD ROCm and Nvidia CUDA accelerators would.
And yes. If you're wondering how they solved the problem of variance between those two: they didn't! Claude instances that run in the TPU land produce slightly different outputs than those in Nvidia racks! Just not different enough for, you know. Anyone to give a shit. No statistically significant effect on model performance.
Using AI slop in software that runs ML models is like buying a Ferrari with a motorbike engine. Might be good for demos or as a proof of concept, but that can’t be used for serving LLMs at scale
Read the "Chipping Away at the CUDA Moat: Using Agents to Enable Day 0 Support" section of [1] where they list a number of agent-created PRs that have been integrated into production vLLM/SGLang code and as well as AMD kernel fixes.
At the risk of noting that this is a PR statement from someone who needs something from AMD - you do realize that you just shat upon the CTO of the largest possible consumer of a technology talking about that technology? It's not 2022 anymore.
Just pointing out, there is a reason the large LLMs keep working to try to make it so their own tools can't attack their own moats, by preventing the exact behavior that you dismiss above.
That would be a very defensible take back in year 2023. Now though?
Anthropic has Mythos. That thing's low level code "AI slop" is better than the "meatbag slop" most software developers write, and it can keep cracking at a given problem with persistence.
OpenAI has GPT-5.6, and also that rabid dog of an AI model that was last seen out in the wild tearing HuggingFace open.
Modern LLMs are very, very capable - not just of writing raw code, but also of persistent, methodical problem solving. Which is what you want to tackle things like "port from an exotic system A to an exotic system B and smoke test the port". Persistently hunting for testable optimizations is a good fit too.
That kicked off my PTSD to setup ROCm on my 6600XT and 9070XT, not even with different containers. I have to build an image for EACH architecture. That's duping 15GB for each arch on my already terribly small SSD.
That depends on the workload. When buying for on-prem inference, MI355x is quite capable.
It still requires a ton more ops time, not every dc handles it cooling-wise, and you have to be careful with the model selection, but when it fits, it fits well
> The basic GCN microarchitecture underpinned every single one of AMD’s compute accelerators for nearly 15 years [...]. With CDNA5 AMD has moved over to a microarchitecture that is based on the RDNA series putting a bookend to the long-lived line that was the GCN microarchitecture.
So everything is new again. Will this be another "buy now, have ROCM work in 3 maybe years if you are lucky"?
At least going forward, with both AMD's AI accelerators and Consumer GPUs using the same RDNA based IP, there will be less reason for AMD to drop RDNA GPUs from the ROCm/HIP support matrix! And so with both MI series accelerators and consumer GPUs/Graphics all using the same basic RDNA Micro-Architecture the ROCm/HIP code base developed for "CDNA5"/later will work for consumer RDNA as well with minor changes required! Polaris and Vega GPUs/Graphics both have been dropped from the ROCm/HIP support matrix. But with AMD's "The Rock" software stack making use of SPIR-V in the same manner as Nvidia/PTX there made be little issues getting that to work with older Vega/Polaris GPUs. even if AMD's not validated/verified that software stack with Consumer Vega/Earlier Consumer GPUs and Integrated Graphics. AMD's biggest issue has been dropping its older GPU micro-architectures from the ROCm/HIP support matrix too soon, and just look at Blender 3D's HIP Back End that requires RDNA2/Later GPUs and Graphics for any Radeon iGPU/dGPU Accelerated Blender 3D Cycles rendering support!
An apt title. To get to the sun, you need to slow down from orbital speed to zero. And then you fall in. Do they run CUDA yet or is this yet more impressive but unusable hardware?
It's mind blowing seeing these multi exaflop single rack systems.
The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022.
It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors.
All the same just wow. Much compute.
Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.
Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a radical optimization for AI workloads, is sometimes considered wasteful now.
AI workloads have some truly peculiar and counterintuitive properties - the kind of things you might expect to see in biology instead of conventional computing. Intrinsic error tolerance, for one. It did necessitate some rethinking and reprioritization, and I'm not quite sure if we converged to the general shape of an "optimal" AI accelerator as of yet.
> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.
It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 workloads. I don't see anyone referring to these AI clusters as supercomputers, for the same reason they aren't referring to the racks as mainframes.
I wonder if there's a term for this kind of semantic narrowing?
I think the term in this case is marketing.
NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a supercomputer but don't yet have the same preconceptions about AI factories.
I don't think that Nvidia single handedly caused the move away from the term supercomputer. But I think they had a hand helping push firmly in that direction.
It's definitely older than that. Originally, way back in the mechanical days when super-computing was hyphenated, it was about how many millions of instructions per second could be handled. That changed in the 1960s with a series of CDC mainframes that put all of their focus on floating point pipelines, moving the category to refer to floating-point crunchers used for scientific simulation (which is what the frontier of computing was used for). Eventually parallelism would be added in at some point, and the term has fossilized into a "monolithic cluster with massive FLOPS intended specifically for scientific simulation". Because for the longest time, that's who needed all the compute.
The PIM/m of the FGCS defied this a little, sometimes being called a "symbolic supercomputer", but never without the "symbolic" qualifier. It was the first time in history where the frontier of computational throughput wasn't about scientific simulations. It wasn't even about arithmetic!
Right now is also interesting, because the world's most powerful FLOPS monster monoliths aren't being used for scientific simulation. They fail at being supercomputers for that reason, but a computer scientist from the 60s would probably raise his eyebrow at that.
FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.
This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing" a specific precision, is that they are usually properly analyzed to determine the needed precision, and then run at that precision.
One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.
Datatype requirements vary in time and space for a given application. We're just beginning to optimize for this, so there is yet much to discover.
ancient intel did 80 bit fp until your intermediate result was written from registers to memory. Since you typically have no control over when that happens, even in very low level languages like C, the 80 bit precision mostly had the effect of making your code produce different results on different compilers. If you needed 80 bit precision and deterministic results you had to write the algorithm in assembly.
432 GB of RAM per chip. Tens of terabytes per rack. Clients queuing up to buy them.
RAM prices are not coming down any time soon.
Did anyone think RAM prices were coming down soon?
Of course. Wishful thinking is a true staple of human intelligence.
There are plenty of laymen that think "AI bubble pop" means that "AI" usage will go to 0 and then all of a sudden all these hardware and datacenters related companies are buying will become totally unused and flood the market.
Hey, they are only projected to cost about $5.5million a rack. Pocket change, really
but you have to rewrite all your software to ROCm. And ROCm, to this day, still sucks.
ZLUDA basically crash on high memory demand, and only accounted for ~70% of CUDA API coverage (and it is still buggy).
But hey, at least it does run on Rust-CUDA. I'm one of the few who ported it and fixed a few bugs on ZLUDA. I used it to run a simple SHA256 kernel and it ran sure, but I gave it up because of those fundamental problems on AMD GPUs. You can't believe how messy ROCm is. I wonder if Vulkan compute kernel using SPIR-V would be a better choice.
Co-founder and Chief Compute Officer Anthropic:
> I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process.
> Our actual experience was we had one engineer who start doing it. They spun up Claude, asked it, hey, bring up this machine, left it going over the weekend. And we ended up with a graph of the actual performance of our leading model on it, just going up and up and up over the weekend.
https://x.com/austinsemis/status/2080336781782753635
That only works if you have all the unit tests and making sure the behaviors between different GPUs are expected, and can tolerate difference in implementation details such as numerical stability and complexity for some functions. AMD and Nvidia are doing vastly different computation for different implementation to the same algorithm for their APIs (can't tell due to NDA, but one algorithm Nvidia did it using broadcast, while AMD is using tile), and that's one of the reason I gave up. And perhaps only for light kernels doing basic BLAS operations and GEMMs. With a standard to follow, that's one of places where it indeed is giving it a good chance, for example on Tensorflow with ROCm or PyTorch. You basically have an insurance of the behavior and a translation layer for ROCm and CUDA by Python.
But anything other than that, nah.
Edit: Maybe I remember what algorithm that is. It is a trick to do coding theory stuff by using NTT (Number Theoretic Transform, the friend of DFT, the Discrete Fourier Transform, but on a ring and hence applicable to finite field) on GPU to do computation over a GF(2^8) field, for my former employment but I went back to using GFNI. Still I have a NDA to adhere to and can't really explain what happens.
That, or, you know. Ask Claude Code to write all of those unit tests.
Variance in numerical instability across hardware is basically a given, because all the different accelerators implement their "fast paths" in slightly different ways. Modern AI just takes it on the chin.
Anthropic, for example, already has to support both Nvidia CUDA and Google/Broadcom TPUs - AMD aside. And I expect those two to have more architectural and software differences than AMD ROCm and Nvidia CUDA accelerators would.
And yes. If you're wondering how they solved the problem of variance between those two: they didn't! Claude instances that run in the TPU land produce slightly different outputs than those in Nvidia racks! Just not different enough for, you know. Anyone to give a shit. No statistically significant effect on model performance.
Using AI slop in software that runs ML models is like buying a Ferrari with a motorbike engine. Might be good for demos or as a proof of concept, but that can’t be used for serving LLMs at scale
This is untrue.
Read the "Chipping Away at the CUDA Moat: Using Agents to Enable Day 0 Support" section of [1] where they list a number of agent-created PRs that have been integrated into production vLLM/SGLang code and as well as AMD kernel fixes.
[1] https://newsletter.semianalysis.com/i/208362548/chipping-awa...
At the risk of noting that this is a PR statement from someone who needs something from AMD - you do realize that you just shat upon the CTO of the largest possible consumer of a technology talking about that technology? It's not 2022 anymore.
Just pointing out, there is a reason the large LLMs keep working to try to make it so their own tools can't attack their own moats, by preventing the exact behavior that you dismiss above.
He has engaged in a ton of bullshit marketing for years.
That would be a very defensible take back in year 2023. Now though?
Anthropic has Mythos. That thing's low level code "AI slop" is better than the "meatbag slop" most software developers write, and it can keep cracking at a given problem with persistence.
OpenAI has GPT-5.6, and also that rabid dog of an AI model that was last seen out in the wild tearing HuggingFace open.
Modern LLMs are very, very capable - not just of writing raw code, but also of persistent, methodical problem solving. Which is what you want to tackle things like "port from an exotic system A to an exotic system B and smoke test the port". Persistently hunting for testable optimizations is a good fit too.
Spir-v is already happening: https://rocm.blogs.amd.com/software-tools-optimization/spir-...
> PYTORCH_ROCM_ARCH="gfx900;gfx906;gfx908;gfx90a;gfx942;gfx950;\ gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;\ gfx1150;gfx1151;gfx1200;gfx1201"
That kicked off my PTSD to setup ROCm on my 6600XT and 9070XT, not even with different containers. I have to build an image for EACH architecture. That's duping 15GB for each arch on my already terribly small SSD.
Well, that appears to be the exact problem that they're trying to solve with spir-v.
> The shift SPIR-V enables is from ahead-of-time compilation per target to compile once, just-in-time specialize on device.
That depends on the workload. When buying for on-prem inference, MI355x is quite capable.
It still requires a ton more ops time, not every dc handles it cooling-wise, and you have to be careful with the model selection, but when it fits, it fits well
> The basic GCN microarchitecture underpinned every single one of AMD’s compute accelerators for nearly 15 years [...]. With CDNA5 AMD has moved over to a microarchitecture that is based on the RDNA series putting a bookend to the long-lived line that was the GCN microarchitecture.
So everything is new again. Will this be another "buy now, have ROCM work in 3 maybe years if you are lucky"?
At least going forward, with both AMD's AI accelerators and Consumer GPUs using the same RDNA based IP, there will be less reason for AMD to drop RDNA GPUs from the ROCm/HIP support matrix! And so with both MI series accelerators and consumer GPUs/Graphics all using the same basic RDNA Micro-Architecture the ROCm/HIP code base developed for "CDNA5"/later will work for consumer RDNA as well with minor changes required! Polaris and Vega GPUs/Graphics both have been dropped from the ROCm/HIP support matrix. But with AMD's "The Rock" software stack making use of SPIR-V in the same manner as Nvidia/PTX there made be little issues getting that to work with older Vega/Polaris GPUs. even if AMD's not validated/verified that software stack with Consumer Vega/Earlier Consumer GPUs and Integrated Graphics. AMD's biggest issue has been dropping its older GPU micro-architectures from the ROCm/HIP support matrix too soon, and just look at Blender 3D's HIP Back End that requires RDNA2/Later GPUs and Graphics for any Radeon iGPU/dGPU Accelerated Blender 3D Cycles rendering support!
Arguably this might reduce it, because now ROCm won't be prioritised for the GCN-based systems?
Not anymore: https://rocm.blogs.amd.com/software-tools-optimization/spir-...
Duplicate post:
https://news.ycombinator.com/item?id=49025398 (13 hours earlier)
An apt title. To get to the sun, you need to slow down from orbital speed to zero. And then you fall in. Do they run CUDA yet or is this yet more impressive but unusable hardware?
AMD never misses a chance to miss a chance.
I guess I submitted the link at the wrong time or something.
I don't have that many kidneys
[flagged]