OpenAI Compatibility

Xinference's inference endpoints accept OpenAI-style request bodies and return OpenAI-style responses. Requests are proxied to your running deployment's Xinference server, so payloads and responses match the OpenAI schema. Note that the platform authenticates with a session cookie — not an API key — so it is not a literal drop-in for clients that only support Bearer API-key auth.

Endpoints

POST /v1/chat/completions
POST /v1/embeddings

The base URL is https://api.xinference.co. Every request must include a model field, which the platform uses to route to your matching running deployment.

Authentication

Send the session cookie issued at sign-in (see Authentication). There is no API-key or Bearer-token scheme.

curl https://api.xinference.co/v1/chat/completions \
  --cookie "session=<your-session-cookie>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Using OpenAI Client Libraries

Because authentication uses a session cookie rather than a Bearer key, the standard OpenAI SDKs (and frameworks like LangChain or LlamaIndex that rely on OPENAI_API_KEY) cannot authenticate against the platform out of the box. To use such a client you must configure it to send the session cookie — for example via a custom HTTP client or default headers. The request and response payloads themselves remain OpenAI-compatible.

Supported OpenAI Parameters

The proxy validates model, messages, input, and stream, and forwards any additional OpenAI parameters to the underlying Xinference server. Support for a given parameter ultimately depends on the deployed model:

Parameter	Chat	Embeddings	Notes
`model`	✅	✅	Required
`messages`	✅	—
`temperature`	✅	—
`max_tokens`	✅	—
`stream`	✅	—
`stop`	✅	—
`top_p`	✅	—
`n`	✅	—
`input`	—	✅
`encoding_format`	—	✅	`float` or `base64`

Known Differences

Authentication uses a session cookie rather than an OpenAI API key.
The model field selects your running deployment and is reflected in responses as the Xinference model name, not an OpenAI model ID.
Which OpenAI features and parameters work depends on the deployed model and the underlying Xinference server, since the platform forwards requests to it.