Granite-SWASH-2B

What is Granite-SWASH-2B?

Granite-SWASH-2B is a small, open-weight language model from IBM’s Granite team. It is a decoder-only dense model, which means it processes text and generates text one token at a time rather than accepting or producing images, audio, or video. The model is intended for general text generation, experimentation, and research rather than for serving as IBM’s flagship production model.

IBM describes it as an experimental, small-scale preview for future Granite work. The model has 2,144,463,840 parameters and is distributed in BF16 format under the Apache 2.0 license through IBM Granite’s official Hugging Face organization. Because the weights are available for self-hosted use, developers can run it locally or deploy it in their own compatible inference environment. The supplied research does not identify a separate IBM-hosted API price for Granite-SWASH-2B.

Where it fits in IBM’s Granite lineup

Granite-SWASH-2B belongs to IBM Granite’s experimental collection rather than being presented as a mature, general-purpose flagship. Its main distinction is architectural: it investigates whether a model can reduce attention-related memory costs by using local attention in most layers while retaining occasional full-attention processing.

This positioning matters when evaluating the model. It is not primarily differentiated by a large context window, multimodal input, tool use, or a hosted service ecosystem. Instead, it is aimed at users who want an Apache-licensed text model and are interested in how attention design affects local inference speed and memory consumption.

How the SWASH architecture works

SWASH stands for Sliding Window Attention + Sinks Hybrid. In a conventional full-attention layer, each new token can attend broadly across the available sequence. That flexibility can increase computation and the size of the key-value cache used during generation. Granite-SWASH-2B changes this pattern by combining two layer types.

• Seven full-attention layers: These periodically give the model access to the complete available context.
• Seventeen sliding-window layers: These use a 128-token attention window, limiting most attention operations to nearby tokens.
• Learnable attention sinks: Per-head sink values help preserve the behavior needed by the hybrid attention mechanism and rescale attention outputs through a log-sum-exp-based calculation.

In practical terms, most layers do not need to process the entire history in the same way. The full-attention layers provide broader context access, while the local layers can reduce the amount of active attention and key-value-cache state. The design is therefore a memory-efficiency experiment, not a claim that every task can be handled with only 128 tokens of total context.

The model also uses grouped-query attention, rotary position embeddings, SwiGLU activations, RMSNorm, shared input and output embeddings, and a vocabulary of 100,352 tokens. These details are relevant to users working with model configuration or inference engines, but they do not change the basic user-facing role: Granite-SWASH-2B is a text-in, text-out language model.

Technical specifications and limits

The documented 8,192-position configuration is the relevant overall sequence limit. The research does not provide a separate maximum-output-token value, so users should not assume that the model supports a particular completion length beyond what their chosen runtime and the remaining context capacity allow.

Capabilities and evaluation results

Granite-SWASH-2B accepts text input and produces text output. Suitable tasks include text completion, drafting, lightweight generation, controlled experiments, and testing inference systems that support the Granite SWASH architecture. It is not natively multimodal and does not generate images, audio, video, music, embeddings, or speech.

The model card reports a WikiText-2 word perplexity of 15.29, MMLU five-shot accuracy of 0.5519, and GSM8K chain-of-thought exact-match accuracy of 0.4223. These are provider-reported results from the model documentation, not guarantees for every prompt or deployment. They suggest a usable general language capability and some basic mathematical-reasoning performance, but the model is not presented as a specialized reasoning system.

For practical evaluation, this model is better understood as a compact base model than as a polished assistant. The supplied research does not identify instruction tuning, safety alignment, a dedicated coding specialization, or a verified structured-output mode. Coding is possible as a general text-generation task, but it should not be treated as a primary strength without task-specific validation.

Deployment and inference considerations

Granite-SWASH-2B can be loaded with Hugging Face Transformers and served through compatible inference systems such as vLLM or SGLang. The model documentation states that Transformers support requires a version newer than 5.8.0. Users should check the current model card and runtime documentation before deployment because support for the attention implementation is important.

Standard scaled dot-product attention, commonly called SDPA, is not supported for this model because it cannot express the attention-sink mechanism used by SWASH. The documented recommendations are Flash Attention 3 or 4, with eager attention available as a fallback for debugging or CPU-oriented use. A backend that does not correctly implement the model’s attention behavior may fail to load the model or produce incorrect results.

The architecture can reduce memory use during generation compared with applying full attention uniformly across all layers, especially when the key-value cache would otherwise become a major constraint. However, the exact speed and memory benefit depends on hardware, quantization, batch size, sequence length, and runtime implementation. The research does not provide a universal tokens-per-second figure or a hardware-specific memory requirement.

Pricing and availability

No IBM-hosted inference price is identified for Granite-SWASH-2B. The model is available as open weights under the Apache 2.0 license, so there is no reported per-token input or output tariff for downloading and running the weights yourself. Self-hosting still involves hardware, storage, electricity, and infrastructure costs, and those costs vary substantially by deployment.

The open license may be useful for teams that need to inspect, adapt, or operate a model without depending on a hosted API. It does not eliminate the engineering work associated with selecting a compatible runtime, managing model files, configuring hardware, and evaluating generated content.

Main strengths and limitations

Strengths include:

• A relatively small 2.14-billion-parameter footprint compared with larger language models.
• An Apache 2.0 license that supports self-hosted experimentation and deployment subject to the license terms.
• A hybrid attention design specifically intended to reduce memory demands during inference.
• Periodic full-attention layers that preserve broader context access instead of restricting every layer to a local window.
• Compatibility with documented open inference tooling, including Transformers, vLLM, and SGLang, when the required architecture support is available.

Limitations include:

• The model is an experimental preview, not a mature production flagship.
• The maximum documented position configuration is 8,192, and no separate maximum output-token limit is supplied.
• It is text-only and has no native image, audio, video, speech, or embedding output.
• Standard SDPA is unsupported because of the attention-sink implementation.
• The model has not undergone safety alignment according to the supplied documentation.
• It may produce biased, false, unsafe, or otherwise problematic content, and IBM notes potential memorization and verbatim-copying risks.
• No verified native tool use, function calling, streaming, fine-tuning, caching, batch API, or JSON-mode capability is identified in the supplied research.

When to choose Granite-SWASH-2B

Choose Granite-SWASH-2B when the main requirement is a compact, open-weight English text model that can be run under your own control and when experimenting with memory-efficient attention is valuable. It is a reasonable candidate for local text completion, research into sliding-window attention, lightweight language applications, and inference experiments where an Apache-licensed model is preferable to a closed hosted service.

The model may also suit developers who want to study how periodic full attention and local attention interact in a small language model. Its relatively modest size can make experimentation more accessible than working with a much larger model, although actual hardware requirements depend on the runtime and precision configuration.

Another option is more appropriate when you need a production-hardened assistant, strong safety controls, verified function calling, a large or extended context window, multimodal generation, a hosted API with predictable per-token pricing, or specialized high-end reasoning and coding performance. It is also worth choosing a different model if your infrastructure cannot support Flash Attention 3 or 4, or if the runtime does not yet implement Granite SWASH correctly.

Bottom line

Granite-SWASH-2B is best viewed as an IBM research-oriented preview with a practical deployment angle. Its defining feature is not a broad set of assistant capabilities but the SWASH attention design: 17 local sliding-window layers, seven full-attention layers, and learnable attention sinks intended to lower inference-memory pressure. For local text generation and architecture research, that makes it distinctive. For safety-sensitive, multimodal, tool-using, or highly demanding production workloads, its experimental status and limited documented capabilities call for a more mature alternative.