Exascale represents the next frontier in supercomputing, performing calculations a billion times faster than 40 years ago—reaching quintillions (1018) of operations per second. Exascale computing will drive transformative breakthroughs across physical sciences and engineering, enabling the development of cutting-edge technologies to address major global challenges—particularly in energy, medicine, and advanced materials. In this post, we explore key features of supercomputing, including foundational concepts and a glimpse into its evolution over time.
Theoretical Peak Performance – FLOPS
A starting point for understanding computer performance is to look at FLOPS or FLOP/s (Floating-Point Operations Per Second), which measures a computer’s speed in performing arithmetic operations. This quantity, also referred to as Theoretical Peak Performance, is approximated by the following formula:
Where
- Clock speed (Hz) – How many cycles a processor (e.g. CPU) executes per second.
- Cores – Number of independent processing units working in parallel (e.g. CPU-core, GPU-core).
- FLOP is a way of encoding real numbers (i.e. FP64 (64 bits) or FP32 (32 bit), FP16 (16 bit)…)
- FLOP/cycle – How many floating-point operations (e.g., additions/multiplications) a core can perform per clock cycle.
The FLOPS formula reveals a key insight: for a given clock speed, the computational performance is primarily determined by the number of cores. To illustrate its meaning, let’s examine the legacy Cray-2— the fastest machine of its time released ~ 40 years ago (1985). Here’s how its performance breaks down using the FLOPS formula:
- Clock speed: 244 MHz (4.1 ns cycle time)
- Cores: 4 vector processors (effectively acting as cores)
- FLOP/cycle: 2 floating-point ops/cycle via pipelining
To put this into perspective: The Cray-2’s performance—roughly 2 gigaFLOPS— was equivalent to performing one calculation per second for 62 years. Today, the world’s fastest supercomputers, like El Capitan, operate at exaFLOPS scales (1 billion gigaFLOPS)—making them a billion times faster than machines from the 1985 era. This billion-fold acceleration in just four decades—highlights the exponential demand for computational power to tackle major challenges in the world.
History of Supercomputing
The timeline below captures over six decades of supercomputing evolution—from MegaFLOPS to Exascale. It showcases the remarkable journey of supercomputers, from the 1960s’ MegaFLOPS pioneers like the IBM 7030 (Stretch) to today’s Exascale giants like El Capitan. Each era—GigaFLOPS (Cray-2), TeraFLOPS (ASCI Red), and PetaFLOPS (Roadrunner)—marked a 1,000-fold leap in performance, driven by innovations in architecture, parallelism, and energy efficiency.
The evolution of supercomputing architecture falls into three distinct phases: the Vector Processing Era (1960-1990), characterized by single-system vector machines; the Cluster Era (1990-2000), marked by distributed parallel computing; and the ongoing Heterogeneous Era (2000-present), defined by hybrid CPU-GPU accelerator systems. Now, systems like Frontier and El Capitan deliver ExaFLOPS-scale power (>10¹⁸ calculations/sec), enabling breakthroughs across scientific domains. Key applications include:
- Advancing nuclear weapon science and scientific discovery.
- Pushing the boundaries of quantum molecular simulations.
- Predicting the structure of proteins through AI-driven simulations.
- Understanding the functionality of virus & potential cures.
- Designing new molecules with unique functionality for modern technology.
- Delivering reliable weather and climate predictions.
The table below complements the supercomputing timeline with verified source references.
| Era (Year) | System | Performance | Architecture | Source |
|---|---|---|---|---|
| MegaFLOPS (1961) | IBM 7030 “Stretch” | ~2 MFLOPS | 1 CPU | IBM Archives |
| GigaFLOPS (1985) | Cray-2 | 1.9 GFLOPS | 4 vector processors | Cray-2 Manual |
| TeraFLOPS (1997) | ASCI Red (Intel) | 1.3 TFLOPS | 9,472 CPUs | U.S. DOE |
| PetaFLOPS (2008) | IBM Roadrunner | 1.7 PFLOPS | 122,400 cores | IBM History |
| ExaFLOPS (2022) | Frontier (HPE/AMD) | 2.055 EFLOPS | ~9M cores | Top500 |
| ExaFLOPS+ (2024) | El Capitan (HPE/AMD) | 2.746 EFLOPS | 11M+ cores | Top500 |
What’s High-Performance Computing ?
High-Performance Computing (HPC) refers to a computing paradigm that emerged in the early 2000s. This approach of computing employs parallel processing architectures to solve complex computational problems by distributing workloads across multiple nodes interconnected with high-speed networks (see also here). At its core, HPC systems leverage:
- Massive parallelism: Coordinating thousands of processors working concurrently
- Specialized infrastructure: High-bandwidth, low-latency interconnects and parallel filesystem
- Scalable architectures: Capable of supporting thousands of interconnected servers, scaling up to over ten thousand nodes, i.e. from clustered servers to modern supercomputers
Unlike conventional computing, HPC enables simultaneous data processing across distributed resources, making it essential for scientific simulations, big data analytics, and advanced modeling applications.
What’s a Supercomputer ?
According to the Oxford English Dictionary, a supercomputer refers to “The design and use of computers with exceptionally high processing power or speed“. This definition is inherently dynamic, as processing power or speed ultimately relies on the state of the technological era. Consider the Cray-2 from the 1980s – with just four processors, it represented the state-of-the-art in supercomputing. Today, as we enter the era of exascale computing, a supercomputer refers to a more powerful HPC system that operates seamlessly in the exaflops range. Here one can distinguish between:
- Traditional computing clusters refer to a collection of servers (or compute nodes) connected via slower interconnects, where data transfer relies on the traditional TCP/IP network protocol. The use of TCP can place a heavy load on the operating system, which in turn increases latency (the time it takes to transfer data). These clusters typically used for shared computing within a single node, and are often deployed in on-premise data centers or cloud platforms where applications do not require intensive computational power.
- HPC clusters consist of a group of compute nodes connected via high-speed interconnects such as InfiniBand, HPE Slingshot, and OmniPath. These interconnects are designed to offer:
- (i) High bandwidth: Fast data transfer capacity.
- (ii) Low latency: Minimal data transfer delay.
- (iii) High scalability: Ability to handle increased load efficiently. These interconnects enable high-performance communication between compute nodes, with the help of libraries like Libfabric and UCX. These libraries are designed to facilitate fast data transfer with minimal delay and can leverage RDMA (Remote Direct Memory Access) capabilities for even faster communication.
- Modern supercomputers can be viewed as advanced HPC clusters that operate at the exaflops scale.
Conclusion
From the early megaflop era to today’s exaflop-scale systems, the evolution of supercomputers reflects our ongoing need for more computing power. Theoretical peak performance and FLOPS offer key metrics for tracking this progress, while the shift from traditional clusters to HPC clusters and modern supercomputers highlights advances in architecture and scale.
Supercomputers now serve as platforms not only for HPC, but also for AI and data analytics, thus enabling solutions to some of the world’s most critical challenges. Yet, despite their transformative impact across science, engineering, and technology, they come with significant downsides, particularly high energy consumption. As we push the boundaries of computing, achieving a balance between performance and energy efficiency will be essential for sustainable progress.
References
- IBM. (1961). IBM 7030 Stretch. IBM Archives. https://www.ibm.com/history/stretch
- Cray Research. (1989). CRAY-2 Computer Systems Functional Description. https://www.mirrorservice.org/sites/www.bitsavers.org/pdf/cray/CRAY-2/HR-0200-0D_CRAY-2_Computer_Systems_Functional_Description_Jun89.pdf
- U.S. Department of Energy. (n.d.). NNSA’s High Performance Computing Achievements. https://www.energy.gov/nnsa/nnsas-high-performance-computing-achievements
- IBM. (2008). IBM Roadrunner Breaks Petaflop Barrier. https://www.ibm.com/history/petaflop-barrier
- Top500. (2022). Frontier – HPE Cray EX235a, AMD Optimized 3rd Gen EPYC 64C 2GHz, AMD Instinct MI250X. https://www.top500.org/system/180047/
- Top500. (2024). El Capitan – HPE Cray EX, AMD Zen 4 + AMD Instinct. https://www.top500.org/system/180307/
- HPC. (2020). History of high performance computing. https://hpc.netl.doe.gov/about/history-of-hpc/