IBM watsonx.ai API

What is the IBM watsonx.ai API?

The IBM watsonx.ai API is the developer interface for IBM's enterprise AI platform. It lets an application send prompts and messages to supported foundation models, receive generated responses, and use related AI operations such as embeddings, reranking, document extraction, evaluations, forecasting and model tuning.

The main interface is a versioned REST API. IBM also provides the official ibm-watsonx-ai Python package and @ibm-cloud/watsonx-ai Node.js SDK. Requests normally identify a model with model_id and identify the IBM Cloud project or deployment context with a project_id or, for some workflows, a space identifier.

watsonx.ai is aimed primarily at enterprise applications rather than casual chatbot use. It is a good fit for teams building governed internal assistants, retrieval-augmented generation systems, AI agents, code-generation tools, document workflows, machine-learning applications and applications that need IBM Cloud, hybrid-cloud or selected on-premises deployment options.

How to get access and credentials

Start by creating or using an IBM Cloud account and provisioning the relevant watsonx.ai service in a supported region. Depending on the product and plan, access may be provided through the watsonx.ai SaaS experience, IBM Cloud, AWS or another supported enterprise deployment arrangement.

1. Choose a supported region and provision watsonx.ai or watsonx.ai Runtime.
2. Open the watsonx.ai Developer access area and record the regional service endpoint.
3. Create an IBM Cloud API key and store it in an environment variable or secret manager, not in application source code.
4. Record the project ID or deployment-space identifier required by the selected operation.
5. Choose a currently available model from the foundation-model catalog and verify its supported features.

For raw HTTP requests, an IBM Cloud API key is exchanged for a short-lived IAM access token. The resulting token is sent in an Authorization: Bearer header. SDKs can simplify authentication, but applications still need the correct service URL, project context and model identifier.

Choosing a model and API operation

For a new conversational or text-generation application, start with the current text chat API, POST /ml/v1/text/chat, or its SDK equivalent. This operation is intended for messages with roles such as system, user and assistant. The selected model must support the features you plan to use.

Direct foundation-model inference is different from deployment-specific inference. A catalog model can be called through the standard model inference path, while a model deployed into a project or deployment space may require a deployment-specific endpoint. Prompt Lab and the current API reference can help identify the appropriate request shape for the selected model.

Model capabilities are not uniform. Before implementation, check whether the chosen model supports chat, vision input, streaming, tools, JSON output, tuning, context requirements and the parameters used by your application. Availability can vary by model, region, plan and deployment type.

Making a first request with REST

The following example uses the current text chat pattern. It exchanges an IBM Cloud API key for an IAM token and then sends a chat request to a regional watsonx.ai endpoint. The version query parameter is required so IBM can evolve the API while maintaining versioned behavior.

set -euo pipefail

: "${IBM_CLOUD_API_KEY:?Set IBM_CLOUD_API_KEY}"
: "${WATSONX_PROJECT_ID:?Set WATSONX_PROJECT_ID}"

WATSONX_URL="${WATSONX_URL:-https://us-south.ml.cloud.ibm.com}"
WATSONX_MODEL_ID="${WATSONX_MODEL_ID:-ibm/granite-4-h-small}"

TOKEN=$(curl --fail-with-body -sS -X POST 
  "https://iam.cloud.ibm.com/identity/token" 
  -H "Content-Type: application/x-www-form-urlencoded" 
  --data-urlencode "grant_type=urn:ibm:params:oauth:grant-type:apikey" 
  --data-urlencode "apikey=${IBM_CLOUD_API_KEY}" 
  | python -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')

curl --fail-with-body -sS -X POST 
  "${WATSONX_URL}/ml/v1/text/chat?version=2025-02-11" 
  -H "Authorization: Bearer ${TOKEN}" 
  -H "Content-Type: application/json" 
  -H "Accept: application/json" 
  -d "$(cat <<JSON
{
  "model_id": "${WATSONX_MODEL_ID}",
  "project_id": "${WATSONX_PROJECT_ID}",
  "messages": [
    {"role": "system", "content": "You are a concise technical assistant."},
    {"role": "user", "content": "Explain what an API gateway does in one paragraph."}
  ],
  "max_tokens": 200,
  "temperature": 0.2
}
JSON
)"

The example assumes that IBM_CLOUD_API_KEY and WATSONX_PROJECT_ID are already set. Change WATSONX_URL for the region where the service is provisioned and change WATSONX_MODEL_ID to a model currently available to the account.

Understanding the response

A successful chat response contains a choices collection. The generated assistant message is typically read from choices[0].message.content. Applications should not assume that every response is identical: streaming responses arrive incrementally, tool calls use message fields containing function information, and error responses require separate handling.

Production code should inspect HTTP status codes and preserve useful request or correlation information for troubleshooting without logging API keys or sensitive prompt content. A response can also contain usage information or other metadata depending on the endpoint, model and service configuration.

How watsonx.ai API pricing works

watsonx.ai combines plan fees with usage-based charges. Foundation-model inference may be metered using resource units equivalent to 1,000 input and output tokens, while some models and deployment modes are billed hourly. IBM offers free playground or trial allocations, an Essentials pay-as-you-go plan and a Standard enterprise plan.

The supplied pricing information lists a published Standard starting instance fee of USD 1,110 per month, but this is not a universal cost for every API workload. Model usage, capacity, hosting, tuning, text extraction and other features can add separate charges. Prices and availability vary by country, region, model, GPU configuration, plan and service location, so review IBM's current pricing page before committing to a production design.

Core capabilities available to developers

• Chat and text generation: Send conversational or instruction-following messages to supported foundation models.
• Streaming: Receive incremental output events instead of waiting for the complete response, which can improve perceived responsiveness in interactive applications.
• Tool or function calling: Provide function definitions that a model can select and populate with JSON arguments. The application, not the model, executes the external function and should validate its arguments.
• Structured responses: Request JSON-object output with response_format for compatible models and APIs. Guided JSON or schema-oriented controls may depend on the model or gateway path, so verify support before relying on strict schemas.
• Selected multimodal input: Vision-capable models can process image content when the model and request format support it. This does not mean that every watsonx.ai model accepts images.
• File-backed workflows: Files can be uploaded or registered as data assets or connections for document extraction, retrieval, tuning and related workflows. The API should not be treated as a universal multipart file-ingestion endpoint.
• Agents: watsonx.ai agent workflows and Agent Lab support applications that use tools and external data.
• Additional operations: APIs and platform workflows cover embeddings, reranking, document text extraction, evaluations, time-series forecasting and foundation-model tuning.

Using the Python SDK

The official Python SDK is useful for notebooks, data-science workflows, inference and tuning-related development. The following example uses the current ModelInference interface and keeps credentials in environment variables.

import os
from ibm_watsonx_ai import Credentials
from ibm_watsonx_ai.foundation_models import ModelInference

credentials = Credentials(
    url=os.getenv("WATSONX_URL", "https://us-south.ml.cloud.ibm.com"),
    api_key=os.environ["IBM_CLOUD_API_KEY"],
)

model = ModelInference(
    model_id=os.getenv("WATSONX_MODEL_ID", "ibm/granite-4-h-small"),
    credentials=credentials,
    project_id=os.environ["WATSONX_PROJECT_ID"],
)

response = model.chat(
    messages=[
        {"role": "system", "content": "You are a concise technical assistant."},
        {"role": "user", "content": "What is an API gateway?"},
    ],
    params={"max_tokens": 200, "temperature": 0.2},
)

print(response["choices"][0]["message"]["content"])

For streaming, the SDK exposes chat_stream. Each event should be inspected for incremental content before it is displayed or forwarded to a client. SDK method names and supported parameters can change across releases, so keep the package updated and check its current documentation when adding less common operations.

Structured output and tool calling

Structured output is useful when the next part of an application expects machine-readable data rather than prose. A compatible request can include response_format with a JSON-object type. The application should still parse the returned text, handle invalid output, and confirm that the selected model supports the requested format.

json_response = model.chat(
    messages=[
        {"role": "system", "content": "Return a JSON object with a key named benefits."},
        {"role": "user", "content": "Give two API gateway benefits."},
    ],
    params={
        "response_format": {"type": "json_object"},
        "max_tokens": 200,
    },
)

print(json_response["choices"][0]["message"]["content"])

Tool calling follows a different control flow. The application supplies a function name, description and parameter definition. If the model requests the function, the application validates the JSON arguments, performs the operation, and sends the result back in the conversation. Do not allow model-generated arguments to bypass authorization, input validation or business rules.

Node.js and other language support

IBM provides the @ibm-cloud/watsonx-ai Node.js SDK for JavaScript and TypeScript services. It exposes current watsonx.ai operations such as text chat and can be appropriate for web backends. PHP and other languages can call the REST API directly with standard HTTP clients such as cURL; a provider-specific package is not required.

Prompt Lab is the main interactive developer tool. It provides a place to test prompts and models and can generate request examples through its code panel. Treat generated examples as a starting point: verify the model, API version, endpoint, parameters and SDK syntax against the current documentation before deploying them.

Streaming, files and multimodal requests

Streaming is appropriate for user-facing chat interfaces because text can be displayed as it arrives. It does not eliminate model or network latency, and applications still need cancellation, timeout and partial-response handling.

File workflows are generally based on IBM Cloud data assets, connections or other registered resources. This is useful for document extraction, retrieval and tuning, but it introduces project storage, permissions and retention considerations. Confirm how the selected workflow stores and accesses the file rather than assuming that a file is processed only in memory.

Image input is available only for selected vision-capable models. Check the model-specific content format and regional availability before building an image-dependent feature. watsonx.ai does not provide a verified first-party general image-generation or video-generation product in the supplied information.

Important limits and production considerations

There is no single public rate limit or fixed latency value that applies to every watsonx.ai model and deployment. Quotas and request limits depend on the IBM Cloud account, service plan, region, model, deployment and gateway configuration. Latency varies with the model, prompt and output size, queueing, region and whether inference is shared or dedicated.

• Handle transient HTTP 429, 503 and 504 responses with bounded retries and exponential backoff.
• Track token or resource-unit usage and set application-level budgets.
• Keep API keys, project identifiers and deployment configuration outside source control.
• Use a regional endpoint that matches data-residency and deployment requirements.
• Check context, output, streaming, tool, vision and JSON support for each selected model.
• Test error handling for malformed requests, unavailable models, quota exhaustion and service interruptions.
• Review the retention and access behavior of saved assets, monitoring, evaluation and governance features.

IBM states that watsonx customer prompts, tuning data, training data and foundation-model outputs are not used by IBM to train or improve IBM-developed models. However, saved project assets, configured monitoring systems, connected data sources and third-party model providers can have separate access, retention and contractual terms. Review the current service documentation for the exact deployment.

Advantages and limitations

watsonx.ai's main advantage is its enterprise scope. It combines model access with IBM Cloud IAM, regional and hybrid deployment choices, governance, monitoring, data connections, agents, tuning and broader machine-learning operations. Official REST, Python and Node.js interfaces make it possible to integrate the platform into existing services rather than using only the web interface.

The trade-off is complexity. Developers must understand projects or spaces, regional endpoints, IAM tokens, versioned requests and model-specific behavior. Pricing is often usage-based or enterprise-oriented, and features may differ between plans, regions and deployment modes. Teams seeking a simple, low-cost consumer chatbot or a single uniform API across all models may find the platform heavier than necessary.

When should you use this API?

Choose the watsonx.ai API when you need governed enterprise AI development, IBM Granite or selected third-party models, regional or hybrid-cloud deployment, retrieval and document workflows, agents, tuning, model lifecycle controls or integration with IBM Cloud projects.

It is a poorer choice when the requirement is casual personal chat, a minimal consumer-facing integration, standalone image or video generation, or a predictable one-price API with identical capabilities across all models. In those cases, the account setup, service selection and usage-based pricing may outweigh the platform's enterprise features.