HuggingFace Embedder

The HuggingfaceCustomEmbedder class is used to embed text data into vectors using the Hugging Face API. You can get one from here.

Usage

1from kern.knowledge.knowledge import Knowledge
2from kern.vectordb.pgvector import PgVector
3from kern.knowledge.embedder.huggingface import HuggingfaceCustomEmbedder
4
5# Embed sentence in database
6embeddings = HuggingfaceCustomEmbedder().get_embedding("The quick brown fox jumps over the lazy dog.")
7
8# Print the embeddings and their dimensions
9print(f"Embeddings: {embeddings[:5]}")
10print(f"Dimensions: {len(embeddings)}")
11
12# Use an embedder in a knowledge base
13knowledge = Knowledge(
14    vector_db=PgVector(
15        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
16        table_name="huggingface_embeddings",
17        embedder=HuggingfaceCustomEmbedder(),
18    ),
19    max_results=2,
20)

Params

Parameter	Type	Default	Description
`dimensions`	`int`	-	The dimensionality of the generated embeddings
`model`	`str`	`all-MiniLM-L6-v2`	The name of the HuggingFace model to use
`api_key`	`str`	-	The API key used for authenticating requests
`client_params`	`Optional[Dict[str, Any]]`	-	Optional dictionary of parameters for the HuggingFace client
`huggingface_client`	`Any`	-	Optional pre-configured HuggingFace client instance

Developer Resources

View Cookbook