Deployments API

Create and manage model deployments. All routes are mounted under /api/v1/model-deployments and require an authenticated session — the session cookie set when the user signs in or completes SSO. Each request is scoped to the current user.

List Deployments

GET /api/v1/model-deployments

Query parameters:

Parameter	Type	Default	Description
`limit`	integer	100	Page size (1–500)
`offset`	integer	0	Pagination offset (≥ 0)

Response:

{
  "items": [
    {
      "id": "dep_abc123",
      "user_id": "usr_xyz",
      "organization_id": "org_abc",
      "cluster_id": "clu_123",
      "instance_id": "i-0abc123",
      "gpu_allocation_id": "gpa_123",
      "model_name": "qwen2.5-instruct",
      "model_format": "pytorch",
      "model_size_in_billions": 7,
      "quantization": "int4",
      "xinference_model_uid": "qwen2.5-instruct-0",
      "status": "running",
      "selected_instance_type": "ml.g5.xlarge",
      "selection_reason": "Best fit for the requested model",
      "failure_stage": null,
      "failure_reason": null,
      "created_at": "2025-06-17T10:00:00Z",
      "updated_at": "2025-06-17T10:05:30Z",
      "terminated_at": null
    }
  ],
  "total": 3,
  "limit": 100,
  "offset": 0
}

Get Deployment

GET /api/v1/model-deployments/{deployment_id}

Returns a single deployment object (same schema as the items above). Responds 404 Not Found if the deployment does not exist or does not belong to the current user.

Create Deployment

POST /api/v1/model-deployments

Request body:

{
  "model_name": "qwen2.5-instruct",
  "model_format": "pytorch",
  "model_size_in_billions": 7,
  "quantization": "int4"
}

Field	Type	Required	Notes
`model_name`	string	Yes	Must be non-empty
`model_format`	string	No	e.g. `pytorch`
`model_size_in_billions`	integer / number / string	No	Parameter count
`quantization`	string	No	e.g. `int4`

Unknown fields are rejected (the request body is strict). On success the platform records the request, returns the deployment with an initial status of requested, and starts provisioning in the background.

Status	Meaning
`201 Created`	Deployment accepted; provisioning started
`422 Unprocessable Entity`	Requested model is not deployable
`409 Conflict`	Blocked by a billing condition

Terminate Deployment

DELETE /api/v1/model-deployments/{deployment_id}

Initiates termination and returns the updated deployment object. Responds 404 Not Found if the deployment does not exist or does not belong to the current user. See Deployment Status for the full lifecycle.

OpenAI-Compatible Inference

Once a deployment is running, inference is served through the OpenAI-compatible proxy (mounted under /v1):

POST /v1/chat/completions
POST /v1/embeddings

The proxy identifies the target deployment from the model field in the request body and forwards the request to the deployment's supervisor endpoint. See the Inference API for request and response details.