At the recent RSA Conference, Intel showed off its latest innovation, the Heracles chip, which will help make efficient encrypted computing possible at scale. Having just been unveiled at the International Solid-State Circuits Conference (ISSCC), Heracles represents Intel’s leading-edge 3-nanometer FinFET technology integrated with high-bandwidth memory. This FHE is both secure and efficient. This powerful combination, along with other features that boost security, makes the chip a major advancement in fully homomorphic encryption (FHE). It’s a big development for secure data processing!
The Heracles chip’s most remarkable aspect is its size and design. Heracles is roughly 20 times the scale of normal FHE research chips, which tend to be 10 square millimeters or smaller. It has 64 compute cores, which are called tile-pairs and laid out in a grid of eight pairs by eight pairs. With 64 megabytes of cache memory and a robust on-chip 2D mesh network, the chip facilitates rapid data processing across its tiles.
Impressive Performance Metrics
Heracles, meanwhile, realizes incredible processing speeds, breaking down data at a geologic scale of 9.6 terabytes per second. This efficiency is made more possible through the chip’s hopping from tile-pair to tile-pair. The chip’s crazy fast 39 microsecond execution of FHE’s key math transformation shows that it’s a gamechanger for any computing task done on encrypted data.
In live demonstrations at ISSCC, Heracles smacked traditional computing systems like CPUs and GPUs all over the court. It performed FHE’s essential math transformation 2,355 times faster than an Intel Xeon CPU running at 3.5 GHz. Heracles beat all other systems in at least seven critical operations. Upon completion, it was declared to be 1,074 – 5,547 times quicker than the competitor.
“When Intel starts talking about scale, that usually carries quite a bit of weight.” – Kurt Rohloff
This on stage testimony is illustrative of how Heracles exceeds the level of a prototype. Most importantly, it emerges as a credible answer for practical use cases requiring trusted data computation. According to Ro Cammarota, “We have proven and delivered everything that we promised,” affirming the reliability of this new technology.
Design Innovations
Sustainable architectural design and Heracles’ clear architectural section are both key to the building’s remarkable performance. Each tile-pair is connected to their neighbors with fat 512-byte tiles, making fast-to-dram access a reality. The chip can run three synchronized streams of instructions simultaneously: one for transferring data onto and off the processor, another for internal data movement, and the last for executing mathematical operations.
Sanu Mathew emphasized the importance of this architectural balance, stating, “It’s all about balancing the movement of data with the crunching of numbers.” This equilibrium allows Heracles to vastly increase its data processing through parallelism, as well as advanced complex computation and logics.
Its counterpart is Heracles, which sports 48 gigabytes of high-bandwidth memory. This memory connects directly to the processor, which aids in delivering speeds up to 819 gigabytes per second. This integration allows the chip to handle extensive datasets reliably, all while providing lightning-fast communication.
“This is like the first microprocessor… the start of a whole journey.” – Sanu Mathew
Future Applications
The influence of Heracles reaches far beyond what is possible today. We just know, as the experts do, that it will have a central part to play in new applications that will demand fast encrypted computing solutions. “Where you start to need hardware is emerging applications around deeper machine-learning oriented operations like neural net, LLMs, or semantic search,” noted Kurt Rohloff.
John Barrus added that many smaller models can benefit from accelerated hardware, stating, “There are a lot of smaller models that, even with FHE’s data expansion, will run just fine on accelerated hardware.” This flexibility demonstrates Heracles’ capability to be customized for myriad applications in diverse verticals.
Nick New, Vice President of Performance Technologies, told us he was most excited about the chip’s ability to drive technological limits even farther. “We’re looking at pushing way past that digital limit,” he remarked, indicating that the development of Heracles may lead to unforeseen advancements in encrypted computing.