A team at Duality Technologies is revolutionizing data privacy for artificial intelligence applications by employing fully homomorphic encryption (FHE) to enable secure computations on encrypted data. This innovative approach allows users to interact with large language models (LLMs) without exposing sensitive information, marking a significant advancement in the field of confidential computing.
Kurt Rohloff, a key figure at Duality, emphasizes the importance of hardware acceleration in speeding up FHE for LLM inference, particularly with larger AI models. This innovation addresses the inherent limitations on memory-bound algorithms in FHE. It guarantees that the data remains private for the duration of the computation.
The Science Behind Fully Homomorphic Encryption
Fully homomorphic encryption, or FHE, is an advanced cryptographic technique that enables computations on encrypted data without ever requiring decryption. This approach is based on lattice-based cryptography, which is based on hard mathematical problems related to vectors in a lattice. The promise of FHE is inarguably its ability to protect full confidentiality through the entire data processing lifecycle.
As Rashmi Agrawal, a privacy and security researcher specializing in this field explains, fully homomorphic encryption algorithms are extremely memory-bound. “Because of that lattice-based encryption scheme, you blow up the data size,” she explains. This growth in data magnitude creates huge challenges for faster processing and analysis time. The aggregated challenge makes low-cost adoption of FHE across large-scale projects difficult.
The duality team is not only committed to making Fully Homomorphic Encryption (FHE) much more efficient. They are particularly focused on advancing the CKKS scheme, short for Cheon-Kim-Kim-Song. This specific scheme works particularly well for machine learning applications since it is natural to work with large vectors of real numbers. As noted by Yuriy Polyakov, a member of the Duality team, the CKKS scheme provides dramatic throughput. This powerful capability makes homomorphic comparison operations on large vectors both efficient and possible.
“That allows us to do a very efficient homomorphic comparison operation of large vectors.” – Yuriy Polyakov
Optimizing Inference Without Retraining
Another innovatory feature of Duality’s framework is the way it processes user queries. The system additionally requires that each user prompt be encrypted using FHE before communicating it with any LLMs. The model can reason about the query without ever needing to decrypt it and then returning an encrypted response. This guarantees that no sensitive information is revealed throughout the entire transaction.
“As Yuriy Polyakov explains though, their approach preserves classic training objectives but improves speed and efficiency during inference. “Whatever we do on the inference does not require retraining,” he states. “In our approach, we still want to make sure that training happens the usual way, and it’s the inference that we essentially try to make more efficient.”
This cutting-edge approach balances privacy protection for users with improved efficiency in AI models. It’s an important consideration for all organizations looking to take advantage of AI capabilities without sacrificing individual privacy.
The Future of Confidential Computing
The release of Duality’s framework is another key step in the evolution of confidential computing. FHE technology is still in its infancy. As Rohloff explains, confidential computing isn’t the only game in town though it has certainly been around longer and makes for a good alternative to encrypting data at rest.
Rohloff emphasizes the necessity for robust privacy technologies in today’s data-driven landscape: “We’re entering into a renaissance of the applicability and usability of privacy technologies to enable secure data collaboration.” He discusses how recent breakthroughs in FHE enable organizations to unlock the value of their data. All this happens without leaking a single byte of sensitive user information.
“They can decrypt the results and get the benefit of running the LLM without actually revealing what was asked or what was responded.” – Kurt Rohloff
There are enormous implications of fully encrypted LLMs using FHE. Most importantly, they expand on the exciting new potential for safer data exchanges across sectors. Organizations are grappling with other data privacy considerations. Duality’s innovations promise to create AI applications that are more secure, intelligent, and effective.