embed

embed

Embedding provider — pure-numpy TF-IDF vectorizer.

No external ML dependencies needed. Works for any language (Greek, English, etc.). Tokenizes on whitespace and punctuation for language-agnostic word-level features.

Embedder

Pure-numpy TF-IDF vectorizer for language-agnostic text embedding.

Tokenizes on whitespace and punctuation for word-level features, then applies TF-IDF weighting with cosine normalization. Supports dtype configuration (float16/float32) and optional vocabulary size capping for memory-constrained environments.

No external ML dependencies needed. Works for any language including Greek and English.

vocab property writable

vocab: dict[str, int]

Get the current vocabulary mapping (term → index).

idf property writable

idf: ndarray | None

Get the IDF weights array, or None if not yet fitted.

__init__

__init__(dtype: str = 'float32', max_vocab_size: int = 0)

Initialize the embedder.

Parameters:

Name	Type	Description	Default
dtype	str	NumPy dtype for embedding vectors. Use "float16" for memory-constrained deployments (e.g., Render free tier). Defaults to "float32".	'float32'
max_vocab_size	int	Maximum number of vocabulary terms to retain. Terms are selected by document frequency (most frequent kept). If 0, retains all terms. Defaults to 0.	0

fit

fit(texts: list[str])

Fit the TF-IDF vectorizer on a corpus of documents.

Builds vocabulary from all terms found in the corpus, optionally capping vocabulary size to max_vocab_size (keeping most frequent terms), then computes IDF weights.

Parameters:

Name	Type	Description	Default
texts	list[str]	List of document strings to fit on.	required

embed

embed(texts: list[str]) -> list[list[float]]

Compute TF-IDF embeddings for one or more texts.

Each text is tokenized, TF-IDF weighted, and L2-normalized before being returned as a list of floats.

Parameters:

Name	Type	Description	Default
texts	list[str]	List of text strings to embed.	required

Returns:

Type	Description
list[list[float]]	List of embedding vectors, each as a list of floats.
list[list[float]]	Vectors are L2-normalized (unit length).

Raises:

Type	Description
ValueError	If fit() has not been called yet (vocab/idf not set).