Duality Technologies has taken a major leap forward in the realm of secure data processing. They announced several new capabilities in fully homomorphic encryption (FHE), which they say will accelerate the performance of large language models (LLMs). The company’s multi-party computation framework enables users to engage with AI systems without exposing their data, keeping it encrypted end-to-end. As Kurt Rohloff, Rashmi Agrawal, and Yuriy Polyakov demonstrate, hardware acceleration plays a central role. They argue that clever algorithmic improvements can make inference more efficient without requiring access to sensitive data.
The technology is based on the revolutionary concept of Fully Homomorphic Encryption (FHE). It supports arithmetic operations to be performed on encrypted data without needing to decrypt it, preserving data confidentiality. This is especially important in today’s world where data privacy reigns supreme. Duality’s framework is underpinned by lattice-based cryptography, based on difficult mathematical problems for vectors in a grid. This innovation supports confidential computing and provides a complete solution to the unique challenges presented by memory-bound algorithms.
Hardware Acceleration Boosts Efficiency
As President Kurt Rohloff of Duality Technologies explains, hardware acceleration can play a critical role in bringing up the speed of FHE operations. As he points out, specialized hardware can help speed FHE processes by two to three orders of magnitude. This advancement comes at a time when more widely used, larger AI models are being deployed and the processing requirements can be massive.
“They can decrypt the results and get the benefit of running the LLM without actually revealing what was asked or what was responded.” – Kurt Rohloff
Duality has brought a hardware abstraction layer to its OpenFHE platform. This change helps simplify moving away from traditional CPU backends. Today, it’s much easier to move to higher performance solutions such as GPU and application specific integrated circuits (ASICs) or FPGAs. Together, these advances collectively bring us closer to realizing FHE for use in the real-world applications.
The Power of CKKS Scheme
As one of Duality’s underlying core technologies, the Cheon-Kim-Kim-Song (CKKS) scheme is seen as an instrumental addition to the Duality framework. As Yuriy Polyakov pointed out, this scheme does a great job of vectorizing large blobs of floats and pulling high throughput rates. The CKKS scheme enables efficient homomorphic comparison operations, which in the past have been too slow due to costly processing needs.
“That allows us to do a very efficient homomorphic comparison operation of large vectors.” – Yuriy Polyakov
Polyakov elaborates by saying that the process of training models isn’t changing, but what’s changing is optimizing inference. By taking either of these approaches, users can take advantage of improved model performance without having to go through lengthy retraining cycles themselves.
Community Efforts and Future Prospects
As Duality continues to develop its FHE technologies, the company understands that collaboration can facilitate this within the tech community. Rashmi Agrawal of Microsoft underscores the need to create fully homomorphic encryption solutions is a cutting-edge technical challenge that requires collaboration. The technology that Duality is now making broadly available are tools to do just that, to expand the horizons of what is possible in secure computing.
“This is a complicated and sophisticated problem that requires community effort. We’re making those tools available so that, together with the community, we can push the state of the art and enable inference for large language models.” – Yuriy Polyakov
Rohloff echoes similar sentiments regarding privacy technologies. He claims that with this solution, users no longer have to sacrifice exposing their data for analysis and insights gained from their data. Fully encrypted models, using Fully Homomorphic Encryption (FHE), present dynamic new possibilities. They feature noninteractive end-to-end confidentiality, which guarantees that sensitive data remains confidential during the entire computation, from input to output.
“FHE is strongest when you need noninteractive end-to-end confidentiality because nobody is able to see your data anywhere in the whole process of computing.” – Rashmi Agrawal