Microsoft researchers claim to have developed an AI that runs on CPUs

Microsoft researchers have announced the development of the largest 1-bit AI model to date, marking a significant leap in AI efficiency and accessibility. The model, called BitNet b1.58 2B4T, is openly available under an MIT license and is designed to run effectively on CPUs, including Apple’s M2 chip, without the need for heavy-duty GPUs.

Bitnets, short for “bit networks,” compress the internal weights of a model into just three possible values: -1, 0, and 1. This quantization drastically reduces the computational and memory requirements needed to operate the models, making them ideal for resource-constrained environments.

According to Microsoft’s research team, BitNet b1.58 2B4T contains 2 billion parameters and was trained on a dataset comprising 4 trillion tokens — roughly equivalent to the text of 33 million books. Despite its compressed structure, the model demonstrated strong performance across several standard AI benchmarks. In testing, BitNet b1.58 2B4T outperformed other major models of comparable size, including Meta’s Llama 3.2 1B, Google’s Gemma 3 1B, and Alibaba’s Qwen 2.5 1.5B, particularly in areas such as math problem-solving (GSM8K) and commonsense reasoning (PIQA).

Perhaps even more noteworthy is the model’s speed and efficiency. Microsoft’s researchers report that BitNet b1.58 2B4T can operate at up to twice the speed of traditional 2 billion-parameter models, all while using a fraction of the memory typically required. This opens up possibilities for running powerful AI tools on devices previously considered unsuitable for such workloads.

“This is an exciting step forward,” the Microsoft team wrote in their official announcement. “By compressing model weights down to 1 bit without dramatically sacrificing performance, we can start thinking about bringing large-scale AI capabilities to far more kinds of hardware.”

However, the breakthrough does not come without challenges. The BitNet b1.58 2B4T model currently requires Microsoft’s custom-built framework, bitnet.cpp, to achieve its advertised performance levels. The framework, at this stage, supports only specific CPU hardware configurations and does not work with GPUs, which continue to dominate the AI infrastructure landscape.

The lack of GPU support could prove a significant limitation. Many current AI workflows — particularly in cloud computing and large-scale model deployment — are deeply reliant on GPU acceleration. Without broader hardware compatibility, bitnets could remain confined to niche applications for the time being.