LiteLLM

Integrate LiteLLM with Kern for a unified LLM experience.

LiteLLM provides a unified interface for various LLM providers, allowing you to use different models with the same code.

Kern integrates with LiteLLM in two ways:

Direct SDK integration - Using the LiteLLM Python SDK
Proxy Server integration - Using LiteLLM as an OpenAI-compatible proxy

Prerequisites

For both integration methods, you'll need:

1# Install required packages
2uv pip install kern-ai litellm

Set up your API key: Regardless of the model used(OpenAI, Hugging Face, or XAI) the API key is referenced as LITELLM_API_KEY.

1export LITELLM_API_KEY=your_api_key_here

SDK Integration

The LiteLLM class provides direct integration with the LiteLLM Python SDK.

Basic Usage

1from kern.agent import Agent
2from kern.models.litellm import LiteLLM
3
4# Create an agent with GPT-4o
5agent = Agent(
6    model=LiteLLM(
7        id="gpt-5-mini",  # Model ID to use
8        name="LiteLLM",  # Optional display name
9    ),
10    markdown=True,
11)
12
13# Get a response
14agent.print_response("Share a 2 sentence horror story")

Using Hugging Face Models

LiteLLM can also work with Hugging Face models:

1from kern.agent import Agent
2from kern.models.litellm import LiteLLM
3
4agent = Agent(
5    model=LiteLLM(
6        id="huggingface/mistralai/Mistral-7B-Instruct-v0.2",
7        top_p=0.95,
8    ),
9    markdown=True,
10)
11
12agent.print_response("What's happening in France?")

Configuration Options

The LiteLLM class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier (e.g., "gpt-5-mini" or "huggingface/mistralai/Mistral-7B-Instruct-v0.2")	"gpt-5-mini"
`name`	str	Display name for the model	"LiteLLM"
`provider`	str	Provider name	"LiteLLM"
`api_key`	Optional[str]	API key (falls back to LITELLM_API_KEY environment variable)	None
`api_base`	Optional[str]	Base URL for API requests	None
`max_tokens`	Optional[int]	Maximum tokens in the response	None
`temperature`	float	Sampling temperature	0.7
`top_p`	float	Top-p sampling value	1.0
`request_params`	Optional[Dict[str, Any]]	Additional request parameters	None

Examples

Note View more examples here.