Pruna AI is set to make waves in the artificial intelligence industry by open sourcing its cutting-edge AI model optimization framework on Thursday. The company's framework is designed to optimize various AI models, from large language models to image and video generation, speech-to-text, and computer vision models. By employing techniques such as caching, pruning, quantization, and distillation, Pruna AI aims to enhance model efficiency without compromising quality.
The enterprise offering from Pruna AI boasts advanced optimization features, including an optimization agent that significantly streamlines the process. Recently, Pruna AI achieved a remarkable feat by making a Llama model eight times smaller while maintaining performance levels. This achievement underscores the potential of their compression framework in delivering substantial efficiency gains.
Pruna AI's framework not only supports a wide range of models but also meticulously evaluates any quality loss that may occur post-compression. According to Pruna AI co-founder and CTO John Rachwan, the framework aggregates various efficiency methods, making them accessible and easy to combine for users.
"But you cannot find a tool that aggregates all of them, makes them all easy to use and combine together. And this is the big value that Pruna is bringing right now." – Rachwan
In addition to open sourcing its framework, Pruna AI is on the verge of releasing a compression agent, which is touted as one of their most exciting features yet. The company envisions that its customers will perceive its compression framework as a worthwhile investment, much like renting a GPU on cloud services.
"It’s similar to how you would think of a GPU when you rent a GPU on AWS or any cloud service," – John Rachwan
Earlier this year, Pruna AI secured $6.5 million in seed funding from notable investors including EQT Ventures, Daphni, Motier Ventures, and Kima Ventures. This investment has fueled the company's development efforts and expansion plans.
Pruna AI charges by the hour for its pro version, providing a flexible pricing model for businesses seeking to leverage their optimization tools. Some of its existing users include Scenario and PhotoRoom, who benefit from the framework's ability to standardize the saving and loading of compressed models and evaluate their performance post-compression.
“We also standardize saving and loading the compressed models, applying combinations of these compression methods, and also evaluating your compressed model after you compress it,” – John Rachwan
As Pruna AI shifts its focus more towards image and video generation models, the open sourcing of its framework represents a pivotal moment for the company. With this move, they aim to foster innovation and collaboration within the AI community by providing a robust toolset for optimizing AI models.