Models

Xinference maintains a curated catalog of deployable models — pre-validated model configurations that are tested and ready to serve production traffic.

What Is a Deployable Model?

A deployable model is a specific combination of:

• Model name — e.g. qwen2.5-instruct
• Model format — e.g. pytorch, gguf
• Size — parameter count in billions (e.g. 7, 14, 72)
• Quantization — precision level (e.g. none, int4, int8, gptq)
• Type — model_type (defaults to LLM; embedding for embedding models)

Each deployable model variant maps to a specific GPU instance family that Xinference will provision when you create a deployment.

Model Catalog

The catalog is visible in the dashboard under Models. You can filter by:

• Type — LLM / Embedding
• Provider — Alibaba, Meta, Mistral AI, etc.
• Family — Qwen, Llama, Mistral, etc.
• Tags — chat, multilingual, code, vision, etc.

Only enabled models can be deployed. Models that are disabled may be under evaluation or temporarily unavailable.

Model Metadata

Each model card displays:

Next Steps

• Supported Models → — full list of available models
• Model Types → — details on LLM and embedding types
• Create a Deployment → — launch a model