{"id":822,"date":"2025-04-18T17:30:03","date_gmt":"2025-04-18T17:30:03","guid":{"rendered":"https:\/\/www.techrepublic.com\/?p=4302447"},"modified":"2025-04-18T17:30:03","modified_gmt":"2025-04-18T17:30:03","slug":"microsoft-releases-largest-1-bit-llm-letting-powerful-ai-run-on-some-older-hardware","status":"publish","type":"post","link":"https:\/\/blog.tecnoartesanos.com\/index.php\/2025\/04\/18\/microsoft-releases-largest-1-bit-llm-letting-powerful-ai-run-on-some-older-hardware\/","title":{"rendered":"Microsoft Releases Largest 1-Bit LLM, Letting Powerful AI Run on Some Older Hardware"},"content":{"rendered":"
<\/div>\n

Microsoft researchers claim to have developed the first 1-bit large language model with 2 billion parameters. The model, BitNet b1.58 2B4T, can run on commercial CPUs such as Apple\u2019s M2.<\/p>\n

\u201cTrained on a corpus of 4 trillion tokens, this model demonstrates how native 1-bit LLMs can achieve performance comparable to leading open-weight, full-precision models of similar size, while offering substantial advantages in computational efficiency (memory, energy, latency),\u201d Microsoft wrote in the project\u2019s Hugging Face depository<\/a>.<\/p>\n

<\/span>What makes a bitnet model different?<\/span><\/h2>\n

Bitnets, or 1-bit LLMs, are compressed versions of large language models. The original 2-billion parameter scale model trained on a corpus of 4 billion tokens was shrunken down into a version with drastically reduced memory requirements. All weights are expressed as one of three values: -1, 0, and 1. Other LLMs might use 32-bit or 16-bit floating-point formats.<\/p>\n

SEE: Threat actors can inject malicious packages into AI models<\/a> that resurface during \u201cvibe coding.\u201d<\/strong><\/p>\n

In the research paper<\/a>, which was posted on Arxiv as a work in progress, the researchers detail how they created the bitnet. Other groups have created bitnets before, but, the researchers say, most of their efforts are either post-training quantization (PTQ) methods applied to pre-trained full-precision models or native 1-bit models trained from scratch that were developed at a smaller scale in the first place. BitNet b1.58 2B4T is a native 1-bit LLM trained at scale; it only takes up 400MB, compared to other \u201csmall models\u201d that can reach up to 4.8 GB.<\/p>\n

<\/span>BitNet b1.58 2B4T model performance, purpose, and limitations<\/span><\/h2>\n

Performance compared to other AI models<\/h3>\n

BitNet b1.58 2B4T outperforms other 1-bit models, according to Microsoft. BitNet b1.58 2B4T has a maximum sequence length of 4096 tokens; Microsoft claims it outperforms small models like Meta\u2019s Llama 3.2 1B or Google\u2019s Gemma 3 1B.<\/p>\n

Researchers\u2019 goal for this bitnet<\/h3>\n

Microsoft\u2019s goal is to make LLMs accessible to more people by creating versions that run on edge devices, in resource-constrained environments, or in real-time applications.<\/p>\n

However, BitNet b1.58 2B4T still isn\u2019t simple to run; it requires hardware compatible with Microsoft\u2019s bitnet.cpp framework. Running it on a standard transformers library won\u2019t produce any of the benefits in terms of speed, latency, or energy consumption. BitNet b1.58 2B4T doesn\u2019t run on GPUs, as the majority of AI models do.<\/p>\n