Structured Output

Code

1from typing import List
2
3from kern.agent import Agent
4from kern.models.vllm import VLLM
5from pydantic import BaseModel, Field
6
7class MovieScript(BaseModel):
8    name: str = Field(..., description="Give a name to this movie")
9    setting: str = Field(
10        ..., description="Provide a nice setting for a blockbuster movie."
11    )
12    ending: str = Field(
13        ...,
14        description="Ending of the movie. If not available, provide a happy ending.",
15    )
16    genre: str = Field(
17        ...,
18        description="Genre of the movie. If not available, select action, thriller or romantic comedy.",
19    )
20    characters: List[str] = Field(..., description="Name of characters for this movie.")
21    storyline: str = Field(
22        ..., description="3 sentence storyline for the movie. Make it exciting!"
23    )
24
25agent = Agent(
26    model=VLLM(id="Qwen/Qwen2.5-7B-Instruct", top_k=20, enable_thinking=False),
27    description="You write movie scripts.",
28    output_schema=MovieScript,
29)
30
31agent.print_response("Llamas ruling the world")

Usage

Set up your virtual environment

1uv venv --python 3.12
2source .venv/bin/activate

1uv venv --python 3.12
2.venv\Scripts\activate

Install Libraries

1uv pip install -U kern-ai pydantic vllm openai

Start vLLM server

1vllm serve Qwen/Qwen2.5-7B-Instruct \
2    --enable-auto-tool-choice \
3    --tool-call-parser hermes \
4    --dtype float16 \
5    --max-model-len 8192 \
6    --gpu-memory-utilization 0.9

Run Agent

1python cookbook/11_models/vllm/structured_output.py