Supported Models List

Lightllm supports most mainstream open-source large language models and multimodal models, and will continue to expand the list of supported models. In future versions, lightllm will support more types of models (such as reward models).

Note

Due to its lightweight design, Lightllm is highly extensible, which means that adding new model support is very simple. For more information, please refer to the Adding New Models section.

---

Large Language Models

Model	Notes
BLOOM
LLaMA
LLaMA V2
StarCoder
Qwen-7b	--eos_id 151643 --trust_remote_code
ChatGLM2-6b	--trust_remote_code
InternLM-7b	--trust_remote_code
Yi-34b
Mixtral
Stablelm	--trust_remote_code
MiniCPM
Phi-3	Only supports Mini and Small.
CohereForAI	--data_type bfloat16
DeepSeek-V2-Lite	--data_type bfloat16
DeepSeek-V2	--data_type bfloat16
DeepSeek-V3
Qwen3
Qwen3-Moe

Multimodal Models

Model	Notes
Qwen-VL	--trust_remote_code --enable_multimodal
Qwen-VL-Chat	--trust_remote_code --enable_multimodal
Llava-7b	--enable_multimodal
Llava-13b	--enable_multimodal
Qwen2-VL	--enable_multimodal
Google Gemma3	--enable_multimodal

Reward Models

Model	Notes
internLM-reward	--use_reward_model
Qwen2-Reward	--use_reward_model