What is GLM-4.6-REAP-268B-A32B?
GLM-4.6-REAP-268B-A32B is an open-weight sparse Mixture-of-Experts, or MoE, causal language model published by Cerebras on Hugging Face. It is derived from zai-org/GLM-4.6 and is intended for people who want to download, run, inspect, or adapt a large language model rather than consume it through a documented hosted API.
The name describes both the model family and the compression method. REAP stands for Router-weighted Expert Activation Pruning. In an MoE model, different input tokens are routed to selected expert networks instead of every parameter being used for every token. REAP removes experts that contribute less to computation, then keeps the router responsible for choosing among the remaining experts.
This process reduces the checkpoint from approximately 355 billion total parameters in the GLM-4.6 base model to approximately 268 billion. The resulting model still activates about 32 billion parameters per token. That does not make it a small model, but it reduces the total memory and infrastructure requirement compared with the unpruned model.
Architecture and verified specifications
The published BF16 checkpoint has 120 experts, compared with 160 in the base configuration, and activates eight experts per token. The model contains 92 layers and uses grouped-query attention with 96 query heads and eight key-value heads. Its stated context length is 202,752 tokens, which is sufficient for very large source repositories, lengthy technical documents, or extended agent sessions when the serving stack and available memory can handle that workload.
| Specification | GLM-4.6-REAP-268B-A32B |
|---|---|
| Provider | Cerebras |
| Base model | GLM-4.6 |
| Model type | BF16 sparse Mixture-of-Experts causal language model |
| Total parameters | Approximately 268B |
| Active parameters per token | Approximately 32B |
| Expert pruning | 25%; 160 experts reduced to 120 |
| Experts activated per token | 8 |
| Context length | 202,752 tokens |
| License | MIT |
| Output modality | Text |
These are model-card specifications rather than independent performance guarantees. Cerebras publishes an FP8 variant separately, but the subject of this page is the BF16 checkpoint. The BF16 model is also described as a useful source checkpoint for downstream low-bit quantization.
Capabilities for coding and agentic work
The model is primarily relevant to demanding text and software tasks. Its documented intended uses include code generation, code editing, repository-scale programming, mathematical reasoning, agentic coding workflows, and function calling. For example, it can be considered for generating an implementation from a detailed specification, reviewing changes across a large codebase, proposing patches, or producing structured tool calls within an agent loop.
The model card describes preserved code-generation and tool-use behavior after pruning. Function calling is therefore a central supported use case, although the exact behavior still depends on the serving framework, tool schema, prompt format, and application-side validation. A model that emits a tool call does not itself execute the tool; the surrounding application must interpret the output and perform the requested operation.
Its long context is especially relevant to repository-scale work. A developer may be able to provide more files, documentation, or prior interaction history in one request than with a shorter-context model. However, a large context window is a capacity specification, not a guarantee that every task will receive equally strong results. It also increases memory and serving demands, particularly when many requests are processed concurrently.
Deployment and infrastructure requirements
GLM-4.6-REAP-268B-A32B can be loaded with Transformers and served through vLLM, SGLang, or compatible OpenAI-style local inference servers. The published vLLM configuration uses tensor parallelism, expert parallelism, automatic tool choice, and the GLM45 tool-call parser. Tensor parallelism distributes model computation across GPUs, while expert parallelism helps distribute the MoE experts across the available hardware.
This is not a practical single-GPU model for ordinary local experimentation. Although only approximately 32B parameters are activated for each token, the checkpoint still contains approximately 268B parameters and must be stored or distributed accordingly. BF16 weights also require substantially more memory than low-bit representations. Multi-GPU infrastructure is the realistic baseline for full-precision-style deployment, while quantization may reduce memory requirements at the cost of an additional quality and compatibility trade-off.
Operators should verify compatibility between the selected model revision, Transformers or serving framework version, tokenizer, chat template, tool-call parser, and parallelism settings. The supplied research confirms deployment patterns, but it does not specify a universal hardware configuration, guaranteed tokens-per-second rate, or maximum concurrency level.
Reasoning, speed, and cost trade-offs
Cerebras presents the pruning approach as near-lossless for important generative workloads, but this should be treated as a provider or model-card claim rather than a universal benchmark conclusion. The BF16 model card does not provide a separate evaluation table for this exact checkpoint; comparative benchmark results are associated with the separately published FP8 variant.
Editorially, this model is best understood as a high-capability, infrastructure-heavy option. Its coding and reasoning potential is attractive for complex repository tasks and long agent workflows, while the REAP reduction can make deployment more practical than using the full GLM-4.6 configuration. It is still much larger and likely slower or more expensive to operate than a compact dense model, especially when the deployment does not have optimized multi-GPU serving.
The research data gives editorial scores of 8 for reasoning, 9 for coding, 5 for speed, and 8 for cost. These scores are editorial assessments, not Cerebras-published benchmark results or pricing claims. They summarize the practical trade-off: strong emphasis on coding and complex generation, but significant infrastructure overhead and no documented hosted price for this exact checkpoint.
Modalities, tools, and API availability
GLM-4.6-REAP-268B-A32B is a text-only model. It accepts text and generates text; it does not natively generate images, audio, or video, and the supplied research does not identify image, audio, or video input support. Its function-calling capability is relevant to tool-using applications, but the checkpoint should not be confused with a complete hosted agent platform.
No hosted token price, maximum output-token limit, knowledge-cutoff date, managed fine-tuning service, batch API, prompt-caching feature, or independent JSON-schema output specification is documented for this particular downloadable checkpoint. The context length is stated as 202,752 tokens, but that should not be treated as a documented maximum output length. Pricing is therefore not applicable as a published recurring API rate for this checkpoint; users must account for their own GPU, storage, electricity, hosting, and engineering costs.
When to choose GLM-4.6-REAP-268B-A32B
This model is a strong candidate when the priority is self-hosted access to a very large coding-oriented language model and the operator can provide multi-GPU infrastructure. Its open-weight MIT-licensed distribution is useful for research, private inference, internal development tools, and experiments with sparse-model compression. The long context is particularly relevant when prompts must include substantial repository material or extended technical history.
- Choose it for repository-scale code generation and editing, long-context analysis, agentic programming, function-calling experiments, and research into MoE pruning.
- Choose it when downloadable weights and deployment control matter more than a simple hosted API bill.
- Choose it when your serving environment supports tensor parallelism, expert parallelism, and the required tool-call parsing configuration.
- Consider another option when you need a small single-GPU model, predictable low-latency responses, a managed API, or clearly published per-token pricing and service limits.
- Consider another option when image, audio, or video understanding or generation is a core requirement.
A smaller model may be more economical for routine code completion, short questions, or high-concurrency services. A hosted model may also be preferable when the total cost of operating a 268B-parameter checkpoint exceeds the value of local control. Conversely, the full GLM-4.6 model may be relevant to users who specifically want the unpruned base configuration, while the separately published FP8 REAP variant may be more suitable when lower-precision deployment is the immediate priority.
Limitations and availability
The most important limitation is the gap between reduced size and genuinely small deployment requirements. REAP removes 25% of the base model's experts, but 268B total parameters remains a substantial operational burden. The BF16 format further favors well-equipped infrastructure over casual local use.
There is also limited public evidence for this exact BF16 checkpoint beyond its model card. No separate BF16 benchmark table, knowledge cutoff, maximum output limit, hosted pricing, or managed API feature set is supplied. Users evaluating it for production should test their own coding, tool-use, latency, and reliability workloads rather than relying only on the base model's reputation or the pruning claim.
Cerebras currently lists the checkpoint in its Hugging Face model catalog, alongside newer GLM-4.7 REAP variants. That catalog positioning makes GLM-4.6-REAP-268B-A32B a current downloadable model rather than an obsolete alias, but the presence of newer variants means teams should compare the exact task quality, memory footprint, framework support, and license terms before standardizing on it.