LiteLLM OpenAI

Use LiteLLM with Kern with an openai-compatible proxy server.

Proxy Server Integration

LiteLLM can also be used as an OpenAI-compatible proxy server, allowing you to route requests to different models through a unified API.

Starting the Proxy Server

First, install LiteLLM with proxy support:

1uv pip install 'litellm[proxy]'

Start the proxy server:

1litellm --model gpt-5-mini --host 127.0.0.1 --port 4000

Using the Proxy

The LiteLLMOpenAI class connects to the LiteLLM proxy using an OpenAI-compatible interface:

1from kern.agent import Agent
2from kern.models.litellm import LiteLLMOpenAI
3
4agent = Agent(
5    model=LiteLLMOpenAI(
6        id="gpt-5-mini",  # Model ID to use
7    ),
8    markdown=True,
9)
10
11agent.print_response("Share a 2 sentence horror story")

Configuration Options

The LiteLLMOpenAI class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier	"gpt-5-mini"
`name`	str	Display name for the model	"LiteLLM"
`provider`	str	Provider name	"LiteLLM"
`api_key`	str	API key (falls back to LITELLM_API_KEY environment variable)	None
`base_url`	str	URL of the LiteLLM proxy server	"http://0.0.0.0:4000"

LiteLLMOpenAI is a subclass of the OpenAILike class and has access to the same params.

Examples

Check out these examples in the cookbook:

Proxy Examples

Note View more examples here.