PrismML releases Bonsai 2, a compact AI model for PCs and smartphones
UKPulse News Desk
AI lab PrismML has released Bonsai 2 27B, a new compressed large language model that can run on personal computers and potentially high-end smartphones. The model achieves a significant reduction in memory size while maintaining 98% of the original model's benchmark performance.
- PrismML's Bonsai 2 27B model compresses Alibaba's Qwen3.8 27B to 5.9 GB, a 9x to 10x memory reduction.
- Bonsai 2 matches 98% of Qwen's aggregate benchmark scores, an improvement from its predecessor's 95%.
- The company aims to apply its compression technique to models in the several-hundred-billion-parameter range in the coming months.
AI lab PrismML has released Bonsai 2 27B, its latest compressed large language model (LLM), which is designed to run on personal computers and potentially high-end smartphones. The model reduces Alibaba's open-source Qwen3.8 27B model to 5.9 GB, representing a 9x to 10x reduction in memory compared to the original.
PrismML, founded by Caltech researchers and led by Professor Babak Hassibi, specialises in compression technologies for LLMs. The company states that Bonsai 2 matches 98% of Qwen's aggregate benchmark scores, an improvement from the first Bonsai model released in March, which matched 95%.
The compression technique, which PrismML calls "ternary" weights, simplifies the information stored for each weight in a model. Hassibi indicated that the startup's next objective is to apply this method to even larger models, with releases in the several-hundred-billion-parameter range anticipated in the next couple of months.
Why this matters: The development of smaller, high-performing LLMs could enable advanced AI capabilities to run directly on personal devices, potentially enhancing privacy and accessibility by reducing reliance on cloud-based processing.
What this means for you: You may be able to use advanced AI models directly on your PC or smartphone, potentially offering more private and free access to AI capabilities without sending data to the cloud.