embed

Embedding provider — pure-numpy TF-IDF vectorizer.

No external ML dependencies needed. Works for any language (Greek, English, etc.). Tokenizes on whitespace and punctuation for language-agnostic word-level features.

Embedder

Pure-numpy TF-IDF vectorizer for language-agnostic text embedding.

Tokenizes on whitespace and punctuation for word-level features, then applies TF-IDF weighting with cosine normalization. Supports dtype configuration (float16/float32) and optional vocabulary size capping for memory-constrained environments.

No external ML dependencies needed. Works for any language including Greek and English.

idf property writable

Get the IDF weights array, or None if not yet fitted.

vocab property writable

Get the current vocabulary mapping (term → index).

__init__(dtype='float32', max_vocab_size=0)

Initialize the embedder.

Parameters:

Name	Type	Description	Default
dtype	str	NumPy dtype for embedding vectors. Use "float16" for memory-constrained deployments (e.g., Render free tier). Defaults to "float32".	'float32'
max_vocab_size	int	Maximum number of vocabulary terms to retain. Terms are selected by document frequency (most frequent kept). If 0, retains all terms. Defaults to 0.	0

embed(texts)

Compute TF-IDF embeddings for one or more texts.

Each text is tokenized, TF-IDF weighted, and L2-normalized before being returned as a list of floats.

Parameters:

Name	Type	Description	Default
texts	list[str]	List of text strings to embed.	required

Returns:

Type	Description
list[list[float]]	List of embedding vectors, each as a list of floats.
list[list[float]]	Vectors are L2-normalized (unit length).

Raises:

Type	Description
ValueError	If fit() has not been called yet (vocab/idf not set).

fit(texts)

Fit the TF-IDF vectorizer on a corpus of documents.

Builds vocabulary from all terms found in the corpus, optionally capping vocabulary size to max_vocab_size (keeping most frequent terms), then computes IDF weights.

Parameters:

Name	Type	Description	Default
texts	list[str]	List of document strings to fit on.	required