Reflection releases Beam, an open-weight 501B parameter model for coding

What happened

Reflection, a new laboratory based in the United States, has released Beam, an open-weight artificial intelligence model [2]. The model is designed for coding tasks and agentic use cases, which refers to AI that can act as an agent to complete complex workflows [2].

Beam is a mixture-of-experts (MoE) model, a type of architecture where only a small part of the total network is used to process each piece of information (token) [2]. This allows the model to have a large capacity while running at the speed of a much smaller model [2]. The model has 501 billion parameters in total, but only 23 billion are active per token [2]. It is a text-only model that supports a context window of 1 million tokens, meaning it can process very large amounts of information at once [2].

According to Hugging Face, the model is released under the Apache 2.0 license, which allows users to own, customize, and scale it [2]. It is available with day-0 support for several software libraries, including transformers, SGLang, and vLLM [2]. It also includes a dedicated open-source harness written in the Rust programming language to help achieve high performance [2].

For users who want to customize the model, several methods are available:

• Supervised fine-tuning (SFT) can be done using TRL’s SFTTrainer [2].
• To manage memory during training, users can use LoRA adapters with PEFT, which trains a small set of weights while keeping the main model frozen [2].
• A recipe developed by Reflection and RadixArk allows for training the model with GRPO in Miles using BF16 [2]. This process can update the full model or use LoRA [2].

In performance tests, the model achieved a score of 78.6% on the SWE Bench Pro v2-Hard benchmark and 81.6% on the SWEBench Verified benchmark when using the BF16 version [2].

Who is affected

Beam is intended for teams that want to run a large-scale model on their own hardware, cloud, or infrastructure [2]. It is also aimed at developers who need to customize a model for specific use cases [2].