Have Supercomputers Truly Been Replaced by Integrated Clusters of Smaller Computers?

Twenty years ago, supercomputers were specialized machines designed to tackle big scientific and engineering problems. Manufacturers such as CDC, Cray, Alliant, NEC, and others were all part of this niche. Problems ranging from astrophysics simulations to global weather predictions required complex algebraic operations on gigantic arrays of numbers. These specialized architectures have progressively been replaced by clusters of powerful computers, high-end PCs connected via very fast networks. Today, we refer to this as High Performance Computing (HPC).

The Evolution of Supercomputing

Typical problems included simulations of astrophysics, supersonic airflow, military jets, space shuttle landings, weather prediction at a global or continental scale, and car crash simulations. These tasks often involved millions, and now sometimes billions, of linear matrix operations. As problems grew larger, specialized architectures were superseded by clusters of powerful computers, high-end PCs connected by high-speed networks.

Modern HPC Clusters

Today, HPC clusters can range from a single rack with 8 or more processors to entirely new data centers. Each computer in a cluster typically features a vast amount of memory, multiple CPUs, and potentially dozens or hundreds of cores built on architectures like x86, Intel, AMD, or GPU-based systems with even more cores. The connectivity between these units is highly elaborate, with networking components often being more expensive than the actual computing units.

The Rise of Supercomputers

Supercomputers are now seen as installations with thousands of small computers. These are primarily found in big universities and research centers. Each computer in a supercomputing cluster often has a significant amount of memory, multiple CPUs, and even hundreds of cores, usually built on standard x86, Intel, AMD, or GPU architectures.

The Next Generation: Exacomputers

Future advancements in computing include the creation of exacomputers, which are at least 100 times more powerful than today's largest supercomputers. The challenges in developing exacomputers go beyond simply purchasing and installing large numbers of smaller computers. Designing even bigger, faster, and more complex connectivity mechanisms is also a significant hurdle.

Reliability and Crash Prevention

Modern computers can guarantee several thousand hours between failures. However, when integrating millions of such computers, statistically, there could be a crash every few minutes. This poses a significant challenge, especially for long-running tasks. Starting a computation job may require several hours just to load the program and distribute initial data across all computers. Random crashes in such installations could halt the job even before it begins. Therefore, new methods to run computer programs are necessary to ensure that desired results are achieved despite the random crashes of hundreds of computers.

Optimized vs. Specialist

It's worth noting that existing computer farms dedicated to cloud computing or Bitcoin calculations are not necessarily classified as HPC/supercomputers. These computer farms are optimized for a large number of smaller tasks, while supercomputers are capable of solving a few tasks of enormous size. However, whether they're part of these farms or dedicated HPC setups, they typically require large buildings, consume electricity faster than medium-sized cities, and necessitate large cooling subsystems.

Conclusion

The transition from traditional supercomputers to integrated clusters of smaller computers has brought about significant advancements in computing power and flexibility. However, the challenges in creating even more powerful systems, such as exacomputers, are complex and multifaceted, requiring innovation in both hardware and software.

Key Takeaway: Despite the shift towards integrated clusters, the concept of supercomputers is still relevant, especially for tasks that require immense computational power. New methods and technologies are continually evolving to overcome the challenges posed by such large-scale computing environments.