Xinference Docs
Get started

Quickstart

Get your first model running in under five minutes.

1. Create an Account

Go to xinference.co and sign up with your email or via Google SSO.

After registration you will be asked to verify your email address. Once verified, you are taken directly to the dashboard.

2. Your Organization

An organization is created for you during sign-up (the organization field is required at registration). If you are working with a team, you can invite teammates to share deployments and a single billing account.

3. Browse Available Models

Navigate to Models in the left sidebar to see the full catalog of deployable models. Each entry shows:

  • Model name and family
  • Model type (LLM, Embedding)
  • Available quantizations and sizes
  • Estimated instance type

4. Deploy a Model

Click Deploy on any model card, or use the API:

import requests

BASE_URL = "https://api.xinference.co"
# Authentication uses the session cookie set when you sign in.
COOKIES = {"session": "<your-session-cookie>"}

response = requests.post(
    f"{BASE_URL}/api/v1/model-deployments",
    cookies=COOKIES,
    json={
        "model_name": "qwen2.5-instruct",
        "model_format": "pytorch",
        "model_size_in_billions": 7,
        "quantization": "int4",
    },
)
deployment = response.json()
print(deployment["id"], deployment["status"])

The deployment advances through these statuses:

requested → provisioning_supervisor → provisioning_worker → waiting_for_xinference → launching_model → running

See Deployment Status for the full lifecycle.

5. Run Inference

Once the deployment reaches running, call the OpenAI-compatible endpoint. Include the same session cookie and a model field matching your deployment:

response = requests.post(
    f"{BASE_URL}/v1/chat/completions",
    cookies=COOKIES,
    json={
        "model": "qwen2.5-instruct",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is Xinference?"},
        ],
    },
)
print(response.json()["choices"][0]["message"]["content"])

6. Terminate When Done

Deployments are billed for the time they run. Terminate as soon as you are done to avoid unnecessary charges:

requests.delete(
    f"{BASE_URL}/api/v1/model-deployments/{deployment['id']}",
    cookies=COOKIES,
)

Next Steps