Why Don't We Use a Single Processor Instead of a Thousand in a Supercomputer?

Supercomputers are designed to solve complex problems that require immense computational power. While it might seem logical to use a single high-speed processor for these tasks, the reality is far more nuanced. The current architecture of supercomputers, which relies on numerous processors working in parallel, is not only feasible but also efficient. This article explores the advantages and limitations of using a singular processor, the current state of technology, and why most supercomputers leverage multiple processors.

Why Multiple Processors?

Supercomputers use a large number of processors, each capable of high-speed operations, to improve data handling and processing efficiency. This approach is rooted in the economics and practical limitations of high-speed CPUs. The fastest high-speed CPU is often more expensive than two CPUs at half the speed, making it cost-prohibitive to use a single massive processor.

The Cost Factor

The cost of semiconductor technology plays a significant role in the design of supercomputers. While it's tempting to opt for a single high-speed processor, the economics of producing such a component often make it unfeasible. The fabrication technology to create a single, massive processor is not commercially available, and the economics do not justify the cost. It's simply more cost-effective to use multiple, smaller, and less expensive processors working in tandem.

Supercomputers with a Single Processor

It's important to note that supercomputers can indeed use a single processor in certain scenarios. Companies like Cerebras Systems have developed supercomputers with a single massive processor that comprises hundreds of thousands to millions of cores. These specialized systems excel at solving specific, data-intensive problems more efficiently than traditional multi-processor setups. However, such systems are not the norm and are typically limited to specific applications.

Supercomputers and Data-Parallel Processing

The majority of supercomputers are built for highly data-parallel and data-local applications. This means they can break down problems into smaller, manageable pieces that can be processed independently by different processors. Each processor communicates only with its neighboring processors, making the system design more efficient and scalable.

Case Study: Cerebras Systems

Cerebras Systems is a leading company that has developed a supercomputer with a single massive processor inside, featuring hundreds of thousands to millions of cores. This system excels at solving specific types of problems more efficiently than traditional multi-processor setups. For instance, in areas like weather prediction, where vast amounts of data need to be processed in a short time, the parallel processing capabilities of such systems are invaluable.

Technical Limitations and Practical Considerations

There are several technical and practical limitations to building a single, massive processor. First, the physical size and fabrication technology of semiconductor chips are limited. Current technology cannot produce a single chip larger than 12 inches (300mm) in diameter due to manufacturing constraints.

Wafers and Wafer Size

The economics of chip manufacturing also play a role. For most computer chips, using a larger wafer does not result in significant cost savings. In fact, it often becomes more expensive to produce larger wafers. This is because the cost of the fabrication process does not scale linearly with the size of the wafer, and the yield of usable chips per wafer decreases as the wafer size increases.

Power and Cooling Considerations

Power and cooling are also critical challenges in building supercomputers. A single Cerebras processor box can consume 20 kilowatts of power, equivalent to the power usage of an entire block of single-family homes. The electrical characteristics required to power such a system are more commonly found in industrial processes like smelting aluminum. The team behind Cerebras had to push existing technology to the limits to manage these power and cooling requirements.

Conclusion

The quest for more computing power continues to drive technological innovation. While a single, high-speed processor might seem like the solution to achieving supercomputing speeds, the practical limitations of cost, fabrication technology, and power management make it impractical in most scenarios. The current architecture of supercomputers, which relies on a large number of processors working in parallel, remains the most efficient and scalable solution for tackling complex data-intensive problems.