Intel recently announced its very first Heracles chip – developed exclusively for Fully Homomorphic Encryption (FHE) computing. This groundbreaking technology was showcased at the IEEE International Solid-State Circuits Conference (ISSCC) held in San Francisco, where it demonstrated its ability to dramatically accelerate encrypted data processing tasks. According to Intel, Heracles can increase FHE computing speeds by up to 100 billion times. Its performance is 5,000 times faster than that of a leading Intel server CPU.
Heracles takes advantage of advanced 3-nanometer FinFET technology combined with high-bandwidth memory to maximize acceleration of encrypted computing at scale. The chip features 64 compute cores laid out in an eight-by-eight array. It operates at a clock frequency of 1.2 GHz and has a 64 MB cache memory. This unique combination makes Heracles a player to watch in the world of highly secure, confidential computing.
Performance Breakthroughs
On the precision side, Heracles demonstrated strong performance metrics that underscore its ability to bring a paradigm shift to encrypted computing. The chip performed FHE’s key mathematical transformation in just 39 microseconds. This accomplishment is equal to a jaw-dropping 2,355-fold increase over an Intel Xeon CPU running at 3.5 GHz. This amazing feat exemplifies Heracles’s ability to tackle high dimensional, computationally complex analyses in less time than ever.
By exploiting a on-chip 2D mesh network, Heracles can achieve unprecedented speeds. This network interconnects the tiles using long, 512 bit wide buses. This unique architecture not only improves compute density, it further accelerates and streamlines data movement. As the chip skips from tile-pair to tile-pair, it can relay information at a breathtaking speed of 9.6 teraflops per second.
Intel’s Sanu Mathew stressed that we’re at a point where the movement of data must be balanced with compute tasks. He stated, “It’s all about balancing the movement of data with the crunching of numbers.”
Innovative Design Features
Heracles is remarkable in design, not just due to its sheer size, but for its cutting-edge cooling technology. Heracles is loaded with two liquid-cooled 24-gigabyte high-bandwidth memory chips, an improvement that sets the machine apart. Measuring roughly 10 square millimeters, it is roughly 20 times the size of current FHE research chips.
To fully exploit the workloads, Heracles executes three parallel instruction streams in lock step. This architecture allows for more efficient data transfers to and from the processor. It does so while controlling the movement of data internally and externally and applying mathematical operations. Bringing these features together truly makes Heracles a pioneer of the next generation encrypted computing.
Ro Cammarota from Intel expressed confidence in the chip’s capabilities, asserting, “We have proven and delivered everything that we promised.”
Applications and Future Prospects
Heracles opens new doors in creating secure, high-performance computing environments. It’s particularly powerful when it’s used for deep learning tasks, which require complex machine learning activities such as neural networks and semantic searches. Kurt Rohloff highlighted the significance of this advancement, stating, “Where you start to need hardware is emerging applications around deeper machine-learning oriented operations like neural net, LLMs, or semantic search.”
Intel’s ongoing investment in and prioritization of scaling this technology speaks well to its eagerness to see FHE adopted for security applications. John Barrus noted that most of the smaller models can reasonably run on this accelerated hardware, even with FHE’s data growth.
During our webinar, Anupam Golder from Microsoft Research highlighted some of the unique challenges that FHE presents. He clarified that while the size of the ciphertext is roughly the same as the plaintext, it often increases greatly when performing FHE computations. This reality has created a growing need for efficient processing solutions such as Heracles.
Sanu Mathew encapsulated the transformative potential of Heracles by likening it to a pivotal moment in computing history: “This is like the first microprocessor… the start of a whole journey.”