Supported Models
The table below lists the models currently available in the Xinference catalog. The catalog is updated regularly as new models are evaluated and approved.
Tip
The live catalog is always available via the API: GET /v1/deployable-models
Large Language Models (LLM)
| Model Name | Display Name | Provider | Size | Quantization | Tags |
|---|---|---|---|---|---|
qwen2.5-instruct |
Qwen2.5 Instruct | Alibaba | 7B, 14B, 32B, 72B | int4, int8, none | chat, multilingual, code |
qwen2.5-coder-instruct |
Qwen2.5 Coder | Alibaba | 7B, 14B | int4, int8 | chat, code |
llama-3.1-instruct |
Llama 3.1 Instruct | Meta | 8B, 70B | int4, int8, none | chat |
llama-3.2-instruct |
Llama 3.2 Instruct | Meta | 1B, 3B | int4, none | chat, edge |
mistral-instruct |
Mistral Instruct | Mistral AI | 7B | int4, int8 | chat |
mixtral-instruct |
Mixtral Instruct | Mistral AI | 8×7B | int4 | chat |
deepseek-r1-distill |
DeepSeek R1 Distill | DeepSeek | 7B, 14B | int4 | chat, reasoning |
gemma-2-instruct |
Gemma 2 Instruct | 9B, 27B | int4, int8 | chat |
Embedding Models
| Model Name | Display Name | Provider | Dimension | Tags |
|---|---|---|---|---|
bge-m3 |
BGE M3 | BAAI | 1024 | embedding, multilingual |
bge-large-en-v1.5 |
BGE Large EN v1.5 | BAAI | 1024 | embedding, english |
text-embedding-3-small |
Text Embedding 3 Small | OpenAI-compat | 1536 | embedding |
gte-qwen2-7b-instruct |
GTE Qwen2 7B | Alibaba | 3584 | embedding, multilingual |
Instance Type Mapping
Xinference automatically selects the most cost-effective GPU instance for each model. The general mapping:
| Model Size | Typical Instance |
|---|---|
| ≤ 7B (int4) | Single A10G (24 GB) |
| 7B–14B (int4) | Single A10G or A100 40 GB |
| 14B–32B (int4) | A100 40 GB |
| 70B+ (int4) | 2× A100 80 GB |
| Embedding | CPU or T4 |
Actual selection depends on availability and model format. See Deployment Overview → for details.