Skip to content
tnsaiDocs
GitHub

Embeddings

Vector-backed retrieval needs an EmbeddingFunction, the interface com.tnsai.memory.advanced.EmbeddingFunction that turns text into a float[]. There are two ways to supply one:

  • Programmatic. Pass it to the store you build, for example new VectorMemoryStore(embedding). That store uses it and nothing else does.
  • Declarative. @Knowledge sources on a Role find one process-wide provider through Java ServiceLoader. Without one they fall back to a hash embedding that ranks by token overlap.

Install an embedding provider

Register one public EmbeddingFunction implementation with a no-argument constructor in this service file:

META-INF/services/com.tnsai.memory.advanced.EmbeddingFunction

The file holds the implementation's fully qualified class name. Resolution is deterministic:

  • No provider: the bundled 128-dimensional hash embedding is used. See Hash-embedding fallback.
  • One provider: it is wrapped, cached for the process, and shared by every vector-backed declarative strategy and by embedding-based deduplication.
  • More than one: binding initialization fails. Classpath order never picks one.

The wrapper admits one call at a time. A concurrent initialization or embed call fails at once instead of waiting, so control concurrency in your application and return promptly from the provider.

Returned vectors must be non-null, non-empty, at most 16,384 values, finite, and of a stable length. The first vector of a valid length fixes the dimension, even if its finite-value check then fails. The wrapper copies the array before it validates and returns it. A RetrievalException from the provider passes through with its reason; any other exception becomes an execution failure.

Hash-embedding fallback

The bundled hash embedding is a normalised bag of words. Two documents that say the same thing in different words score near zero, so @Search(strategy = SEMANTIC) behaves lexically.

When the hash embedding is in use and a Role asks for SEMANTIC, HYBRID, HIERARCHICAL or TEMPORAL, the runtime logs one WARN per Role class for the life of the process. The warning names the Role and the remedies. KEYWORD, GRAPH, MULTI_QUERY and REASONING don't use vector similarity and don't warn. Nothing throws: the fallback is a deliberate zero-setup default, so offline tests and examples keep running.

HYBRID is only partly affected, since its BM25 half still runs, but its vector half is lexical.

To act on the warning, do one of these:

  1. Install an embedding provider.
  2. Use strategy = KEYWORD if lexical matching is what you want.
  3. Keep the hash embedding on purpose with -Dtnsai.rag.acknowledgeHashEmbedding=true. That removes the warning without silencing the logger.

RagDiagnostics reports whether a Role resolved to the bundled fallback.

Shorter vectors from one model

EmbeddingFunctions.matryoshka(delegate, dimensions) keeps the first dimensions values of each vector and L2-normalizes them. Use it only when the model is Matryoshka-trained or otherwise prefix-compatible; cutting other vectors changes their geometry.

EmbeddingFunction full = /* application provider, e.g. 1024-d */;
EmbeddingFunction reduced = EmbeddingFunctions.matryoshka(full, 256);
VectorMemoryStore store = new VectorMemoryStore(reduced);

Each embed call asks the delegate for the full vector, copies the prefix (the delegate's array is never changed or returned) and normalizes it. An all-zero prefix stays zero.

dimensions must be positive. A non-finite prefix value throws IllegalArgumentException. A delegate vector shorter than dimensions throws EmbeddingDimensionMismatchException, which reports expectedDimension and actualDimension.

Store dimension

VectorMemoryStore fixes its dimension from the first vector it admits. Later add calls and non-empty search vectors must have the same length, or EmbeddingDimensionMismatchException is thrown. clear() removes the entries but keeps the dimension.

Build the store with the reduced function from the first add. A 1024-d query against a 256-d index is a typed failure, not a silent reshape.