Inference API

Once a deployment is running, Xinference exposes OpenAI-compatible inference endpoints. The platform proxies your request to that deployment's Xinference server and returns its response unchanged, so OpenAI-style request and response bodies work as-is.

Base URL

https://api.xinference.co

Inference endpoints are served under the /v1 prefix.

Authentication

Inference requests are authenticated with the same HttpOnly session cookie as the rest of the platform — there is no separate API key. Sign in first (see Authentication) and send the session cookie with each request.

Supported Endpoints

How It Works

Every request must include a model field. The platform uses it to locate your matching running deployment and forwards the request to that deployment's supervisor endpoint. For chat completions, setting "stream": true returns a streamed response.

curl https://api.xinference.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  --cookie "session=<your-session-cookie>" \
  -d '{
    "model": "qwen2.5-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Response Format

Responses are returned exactly as produced by the underlying Xinference server, following the OpenAI response schema (including usage fields):

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1718000000,
  "model": "qwen2.5-instruct",
  "choices": [ ... ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 47,
    "total_tokens": 59
  }
}

Errors

If no running deployment matches the requested model, the proxy responds 404:

{ "detail": "Model is not running." }

Any error produced by the underlying Xinference server is passed back to the caller.

Next Steps

• Chat Completions →
• Embeddings →
• OpenAI Compatibility →