Create a Deployment

You can create a deployment from the dashboard or via the REST API.

From the Dashboard

1. Navigate to Models in the left sidebar
2. Find the model you want to deploy and click Deploy
3. Select the desired quantization (if multiple options exist)
4. Click Confirm Deployment

The deployment card appears immediately in Deployments with status pending.

Via the API

Endpoint

POST /api/v1/model-deployments

Request Body

Example

Authentication uses the session cookie set when you sign in (see Authentication).

import requests

COOKIES = {"session": "<your-session-cookie>"}

response = requests.post(
    "https://api.xinference.co/api/v1/model-deployments",
    cookies=COOKIES,
    json={
        "model_name": "qwen2.5-instruct",
        "model_format": "pytorch",
        "model_size_in_billions": 7,
        "quantization": "int4",
    },
)

deployment = response.json()
print(deployment)
# {
#   "id": "dep_abc123",
#   "status": "requested",
#   "model_name": "qwen2.5-instruct",
#   "selected_instance_type": "ml.g5.xlarge",
#   ...
# }

Polling for Ready State

Deployments are asynchronous. Poll the deployment status endpoint until status becomes running:

import time

deployment_id = deployment["id"]

while True:
    r = requests.get(
        f"https://api.xinference.co/api/v1/model-deployments/{deployment_id}",
        cookies=COOKIES,
    )
    status = r.json()["status"]
    print(f"Status: {status}")

    if status == "running":
        break
    elif status in ("failed", "failed_cleaned", "cleanup_failed"):
        print("Deployment failed:", r.json().get("failure_reason"))
        break

    time.sleep(15)

Instance Selection

Xinference automatically selects the GPU instance type based on the model's requirements. The chosen instance type and the reason are returned in:

• selected_instance_type — e.g. g5.xlarge
• selection_reason — human-readable explanation

If no suitable instance type is available for the requested model configuration, the deployment fails with status: failed and a descriptive failure_reason.

Billing Start

Billing begins when the cluster starts provisioning (status transitions from pending to provisioning). It stops when the deployment reaches terminated.