Groq

Use Groq's fast inference API with Kern agents.

Groq offers blazing-fast API endpoints for large language models.

See all the Groq supported models here.

We recommend using llama-3.3-70b-versatile for general use
We recommend llama-3.1-8b-instant for a faster result.
We recommend using llama-3.2-90b-vision-preview for image understanding.

Multimodal Support

With Groq we support Image as input

Authentication

Set your GROQ_API_KEY environment variable. Get your key from here.

1export GROQ_API_KEY=***

1setx GROQ_API_KEY ***

Example

Use Groq with your Agent:

1from kern.agent import Agent
2from kern.models.groq import Groq
3
4agent = Agent(
5    model=Groq(id="llama-3.3-70b-versatile"),
6    markdown=True
7)
8
9# Print the response in the terminal
10agent.print_response("Share a 2 sentence horror story.")

Note View more examples here.

Parameters

Parameter	Type	Default	Description
`id`	`str`	`"llama-3.3-70b-versatile"`	The id of the Groq model to use
`name`	`str`	`"Groq"`	The name of the model
`provider`	`str`	`"Groq"`	The provider of the model
`api_key`	`Optional[str]`	`None`	The API key for Groq (defaults to GROQ_API_KEY env var)
`base_url`	`str`	`"https://api.groq.com/openai/v1"`	The base URL for the Groq API
`timeout`	`Optional[int]`	`None`	Request timeout in seconds

Groq extends the OpenAI-compatible interface and supports most parameters from the OpenAI model.