What is Step-3.5-Flash-REAP-149B-A11B?
Step-3.5-Flash-REAP-149B-A11B is an open-weight causal language model published by Cerebras. It is designed to generate and analyze text, write code, solve mathematical and reasoning tasks, and participate in tool-enabled workflows. The model is not a hosted subscription product in the supplied research; it is a downloadable checkpoint intended for local, self-hosted, or research deployment.
The name describes both its model family and its compressed form. The checkpoint is derived from Step-3.5-Flash, which has 196 billion total parameters. Cerebras applies Router-weighted Expert Activation Pruning, abbreviated as REAP, to remove a portion of the model's experts without retraining the resulting checkpoint. The REAP version contains 149 billion total parameters and activates approximately 11 billion parameters per token.
In a mixture-of-experts, or MoE, model, only a selected group of specialist subnetworks is used for each token rather than the entire parameter set. This can reduce computation per token compared with a dense model of the same total size, although the complete checkpoint still requires substantial storage and multi-GPU infrastructure.
Where it fits in Cerebras's model lineup
Step-3.5-Flash-REAP-149B-A11B is a compressed derivative of Cerebras's Step-3.5-Flash model, not a separate lightweight model family. Its main purpose is to make a large sparse model more practical to deploy by pruning lower-impact experts while retaining much of the base model's behavior.
The model card describes a 25% expert-pruning ratio. REAP uses router gate values and expert activation norms to identify experts that are selected less often or contribute less to layer outputs. The remaining router-controlled experts continue to provide sparse specialization. This positioning makes the checkpoint most relevant to organizations and researchers that want the capabilities of a large MoE model but are willing to manage their own hardware and serving stack.
Architecture, parameters, and context length
The verified configuration describes a 45-layer sparse MoE architecture with 216 experts and 8 experts activated for each token. The model has a 262,144-token context window, allowing it to process very long prompts or document collections when the serving environment has enough memory and supports the required context length.
The published repository identifies the architecture as Step3p5ForCausalLM and lists bfloat16 and float32 tensor representations. These formats are not the same as a guarantee that every deployment should use full precision. Actual memory requirements depend on precision, quantization, tensor parallelism, batch size, context length, and the inference framework.
| Specification | Verified detail |
|---|---|
| Provider | Cerebras |
| Model type | Open-weight sparse mixture-of-experts causal language model |
| Total parameters | 149 billion |
| Activated parameters | Approximately 11 billion per token |
| MoE configuration | 216 experts, with 8 activated per token |
| Layers | 45 |
| Context length | 262,144 tokens |
| License | Apache 2.0 |
The activated-parameter figure should not be mistaken for the storage requirement. Although roughly 11 billion parameters are used for an individual token, the complete 149-billion-parameter checkpoint still needs to be stored and made available to the serving system. Consequently, this is not a suitable model for low-memory laptops or typical consumer devices without substantial optimization and compatible hardware.
Coding, reasoning, and tool use
Step-3.5-Flash-REAP-149B-A11B is intended for text generation and conversational applications. The published model information specifically identifies code generation, mathematics, reasoning, and tool calling as retained capabilities. It is therefore a plausible fit for coding assistants, agentic coding systems, mathematical problem solving, and applications in which the model must request actions from external tools.
The repository supplies deployment settings for Step-3.5 reasoning and tool-call parsers in vLLM. Tool calling allows a serving application to expose functions or external operations to the model; it does not mean that the checkpoint independently accesses the web, databases, or software systems. Those integrations must be implemented by the application hosting the model.
The model card reports a HumanEval score of 97.0 and a HumanEval+ score of 90.9 for this checkpoint. These are published evaluation results for coding benchmarks, not a guarantee of performance on every codebase or programming task. Benchmark outcomes can vary with prompting, sampling settings, evaluation harnesses, and whether generated code is tested in a realistic environment.
Supported modalities and output types
This is a text-only language model. It accepts text input and produces text output. The supplied research does not identify image, audio, or video input, nor does it support direct image, audio, or video generation. Its tool-calling capability is structured text interaction with application-defined functions rather than a separate media output modality.
The context window is documented as 262,144 tokens, but no authoritative maximum output-token limit is provided for this exact checkpoint in the supplied research. The usable prompt and completion lengths also depend on the serving framework and the total context limit configured by the operator.
Deployment and inference
Cerebras provides or documents deployment paths through Transformers, vLLM, SGLang, Docker Model Runner, and compatible quantization workflows. The model is available from the official Cerebras Hugging Face repository.
For vLLM, the published configuration uses tensor parallelism and expert parallelism, along with the Step-3.5 tool-call parser, the Step-3.5 reasoning parser, automatic tool choice, and disabled cascade attention. These settings are useful starting points, but deployment still requires testing against the operator's GPU layout, precision, quantization method, context length, and workload.
The sparse architecture can reduce active computation per token compared with processing all 149 billion parameters, but it does not turn the model into a small or inexpensive deployment. Large-model serving normally requires multiple GPUs or other substantial infrastructure. Longer contexts and larger batches increase memory pressure, while quantization may reduce memory use at the cost of additional compatibility or quality trade-offs.
Pricing and hosted API availability
No official Cerebras hosted per-token price was identified for this exact checkpoint. It is distributed as an open-weight model under the Apache 2.0 license, so the primary cost model is infrastructure: hardware, electricity, storage, operations, and engineering time for deployment.
The repository information states that the model is not deployed by a Hugging Face Inference Provider. This means readers should not assume that a normal managed API endpoint, provider-managed uptime, or fixed input and output token pricing is available for this checkpoint. A third party could potentially offer hosted access independently, but no such pricing or service specification is verified here.
Main strengths and limitations
The clearest strength is the combination of a very long context window, sparse activation, open weights, and retained coding, reasoning, mathematical, and tool-use functionality. REAP also provides a way to reduce the total size of the Step-3.5-Flash family checkpoint without a full retraining process, which may be useful for research into model compression and expert pruning.
Its main limitation is operational scale. A 149-billion-parameter checkpoint remains difficult to run, especially in bfloat16 or float32. The model is not a low-resource option, and sparse activation does not eliminate the need to load or distribute the full model across suitable hardware. The absence of a confirmed hosted API price also makes it less convenient for teams that want immediate pay-as-you-go inference.
Other limitations follow from its modality and documentation. It is text-only, has no verified image or audio generation capability, and has no authoritative maximum output-token figure supplied for this exact release. Hardware and performance claims should therefore be validated in the intended serving environment rather than inferred solely from the activated-parameter count.
When to choose this model
Choose Step-3.5-Flash-REAP-149B-A11B when you need an open-weight model for self-hosted coding assistance, agentic workflows, tool-enabled applications, mathematical reasoning, or experimentation with compressed sparse MoE systems. It is particularly relevant when data-control, local deployment, or customization matters more than the convenience of a managed API.
- Choose it for local control: the checkpoint can be integrated into a deployment stack instead of sending prompts to a third-party hosted service.
- Choose it for long documents: the documented 262,144-token context window is useful for large codebases, long technical documents, and extended agent state, subject to hardware and framework limits.
- Choose it for coding and tools: the model card reports strong coding benchmark results and the repository includes reasoning and tool-call parser settings.
- Choose another type of option for simplicity: a managed API is more appropriate when the team does not want to operate multi-GPU inference infrastructure.
- Choose a smaller model for constrained hardware: a compact model is more appropriate for laptops, edge devices, or low-cost deployments where a 149-billion-parameter checkpoint is impractical.
- Choose a multimodal model for media tasks: this checkpoint is not intended for image, audio, or video understanding or generation.
Bottom line
Step-3.5-Flash-REAP-149B-A11B is best understood as a deployment-oriented compressed version of a large Cerebras MoE model, not as a small local chatbot. Its 149-billion-parameter open-weight checkpoint, approximately 11-billion-parameter active footprint, long context, coding performance, and tool support make it interesting for capable self-hosted systems. The trade-off is substantial infrastructure: users must manage the hardware, serving framework, scaling, and operating costs themselves, and no official hosted price or maximum output limit is confirmed for this exact model.