Reward Model Deployment Configuration

LightLLM supports inference for various reward models, used for evaluating conversation quality and generating reward scores. Currently supported reward models include InternLM2 Reward and Qwen2 Reward, etc.

Basic Launch Command

python -m lightllm.server.api_server \
--port 8080 \
--model_dir ${MODEL_PATH} \
--trust_remote_code \
--use_reward_model # Enable reward model functionality (required parameter)

Testing Examples

Python Testing Code

import json
import requests

# InternLM2 Reward test
query = "<|im_start|>user\nHello! What's your name?<|im_end|>\n<|im_start|>assistant\nMy name is InternLM2! A helpful AI assistant. What can I do for you?<|im_end|>\n<|reward|>"

url = "http://127.0.0.1:8000/get_score"
headers = {'Content-Type': 'application/json'}

data = {
    "chat": query,
    "parameters": {
        "frequency_penalty": 1
    }
}

response = requests.post(url, headers=headers, data=json.dumps(data))

if response.status_code == 200:
    result = response.json()
    print(f"Reward score: {result['score']}")
    print(f"Input tokens: {result['prompt_tokens']}")
else:
    print(f"Error: {response.status_code}, {response.text}")

cURL Testing Command

curl http://localhost:8000/get_score \
     -H "Content-Type: application/json" \
     -d '{
       "chat": "<|im_start|>user\nHello! What is AI?<|im_end|>\n<|im_start|>assistant\nAI stands for Artificial Intelligence, which refers to the simulation of human intelligence in machines.<|im_end|>\n<|reward|>",
       "parameters": {
         "frequency_penalty": 1
       }
     }'