Embedding models for the Virtuoso vector index

An embedding model converts text into a fixed-length float vector (an "embedding") so that semantically similar texts land close together in vector space. Virtuoso's vector index is model-bound: every index is created against a named model in DB.DBA.ONNX_MODELS, and that model is used to embed both your stored data and your query text. Choosing and managing the right model is therefore central to getting good search results.

This reference covers:

• Overview: What embedding models are and the two categories Virtuoso supports
• The ONNX_MODELS table: The model registry, its schema, and how to inspect it
• Local ONNX models: The bundled granite model, bootstrap, load, and embed functions
• Remote HTTP embedding providers: OpenAI, Ollama, LM Studio, and custom endpoints
• API key management: Storing provider keys
• Registering a provider model: The registration procedure and examples
• Choosing a model: A comparison of latency, cost, quality, privacy, and dimensions
• Using a custom ONNX model: Exporting and registering your own ONNX model

Overview

An embedding model is a neural network that maps a piece of text to a vector of real numbers — for example, a 384-dimensional float vector for the bundled granite model. Texts that mean similar things produce vectors that are close together under a distance metric such as cosine similarity. The Virtuoso vector index stores these vectors in a native HNSW graph and uses the same model to embed query text at search time, so the query vector can be matched against the stored vectors.

Because the index is bound to a single model, the model determines three things up front:

• Output dimension — the length of every vector (e.g. 384 for the bundled granite model, 1536 for OpenAI text-embedding-3-small). The index's dimensions must match exactly, and you cannot change models after the index is created.
• Quality — how well the embeddings capture semantic similarity. Larger or more recent models generally produce better rankings.
• Operational characteristics — latency, cost, privacy, and memory footprint, which differ sharply between local and remote models.

Virtuoso supports two categories of embedding models:

Both categories are represented as rows in the same DB.DBA.ONNX_MODELS table and are referenced by model name from CREATE VECTOR INDEX (SQL) or create vector index (SPARQL). The difference is in how the row is populated and how embeddings are produced at runtime.

The ONNX_MODELS table

DB.DBA.ONNX_MODELS is the central model registry. Every model a vector index can use — local or remote — has a row here. You do not normally populate this table by hand; the bundled local model is auto-provisioned at startup, and remote models are registered through a stored procedure.

Auto-provisioning at startup

When Virtuoso starts with the vonnx plugin loaded, the bundled granite-384-local model is automatically registered and loaded. This happens inside the bootstrap routine ONNX_BOOTSTRAP_MODELS(), which:

1. Ensures the ONNX_MODELS table exists (creating it if necessary).
2. Registers granite-384-local. It looks for granite-embedding-30m-english/model.onnx and vocab.json (with merges.txt beside it) under [ONNX] ModelRoot — or at [ONNX] GraniteModelPath and GraniteVocabPath when those are set — and for the helper program at [ONNX] HelperPath, falling back to the plugin LoadPath and the DirsAllowed directories. The paths are resolved again at every start, so a moved installation still finds them. If a file cannot be found, the server log names it and says what to configure, and an existing granite-384-local row is marked unavailable.
3. Loads every registered model and stores each local model's handle in the handle column. The server log reports ONNX bootstrap refreshed N model(s).

After startup, the bundled model is ready to use with no extra configuration. The relevant virtuoso.ini settings, as make install writes them:

[ONNX]
ModelRoot   = <hosting directory>/onnx-models
;HelperPath = <hosting directory>/onnx_helper

[Plugins]
LoadPath    = <hosting directory>
Load17      = plain, vonnx

With HelperPath commented out, the helper is found in the plugin LoadPath. Relative paths are resolved against the server root — the directory the server runs in, which holds its database — so use absolute paths when in doubt.

Schema

How to inspect the table

SELECT model_name, model_type, handle, output_shape
FROM DB.DBA.ONNX_MODELS
ORDER BY model_name;

Example output after a default startup:

To see the full picture including remote provider details:

SELECT model_name, model_type, handle, output_shape,
       provider_endpoint, provider_api_key_name,
       provider_timeout, provider_dimensions
FROM DB.DBA.ONNX_MODELS
ORDER BY model_name;

Local ONNX models

A local ONNX model runs on the Virtuoso host itself. The vonnx plugin wraps ONNX Runtime and runs each loaded model in a pool of helper processes that the server starts and talks to over pipes. There are no network calls and no text ever leaves the machine.

The bundled granite-384-local model

Virtuoso ships with a single bundled embedding model:

This is a general-purpose English embedding model. It is small, free to run, and needs no setup, which makes it the default for development and for production workloads where keeping data on the host is a priority. Its files — model.onnx, vocab.json and merges.txt — are installed under onnx-models/granite-embedding-30m-english/ in the hosting directory, with a README that records how the model was exported and checked, and a SHA256SUMS file.

model.onnx is not kept in the source repository. A build with --enable-onnxruntime downloads it from OpenLink's model repository (https://nexus.openlinksw.com/repository/onnx-models) when it is missing and checks every file against SHA256SUMS; a mismatch fails the build. A tree that already holds a verified copy builds without the network.

ONNX_BOOTSTRAP_MODELS()

ONNX_BOOTSTRAP_MODELS() is called automatically during server startup (after the vonnx plugin loads). It creates the ONNX_MODELS table if it does not exist, registers the bundled granite-384-local row, and loads every registered local model so their handles are ready immediately.

You normally never call this procedure yourself. It is safe to call it again manually if you need to re-bootstrap after a table was damaged, but in normal operation startup handles it.

ONNX_LOAD_MODEL

ONNX_LOAD_MODEL(model_path, options) loads an ONNX model file from disk into ONNX Runtime and returns an integer handle that identifies it. For a registered model, the bootstrap and ONNX_MODEL_REFRESH store that handle in the handle column of ONNX_MODELS.

This is the low-level load primitive used by the bootstrap and by custom-model registration, and it is DBA-only. The options string tells the runtime how to feed the model — see Using a custom ONNX model for the format. A file already loaded with the same options returns the existing handle rather than a new one.

ONNX_UNLOAD_MODEL

ONNX_UNLOAD_MODEL(handle) (DBA-only) closes a handle and stops its helper processes; it returns 1. It refuses while calls are in flight on that handle (Model N has M in-flight calls). Unload a handle before reloading a model file you replaced on disk — see ONNX_MODEL_REFRESH.

ONNX_EMBED_TEXT

ONNX_EMBED_TEXT(handle, text) is the lower-level BIF that produces an embedding from an already-loaded model. You pass the handle (the integer returned by ONNX_LOAD_MODEL), not the model name, so you must look up the handle first:

SELECT ONNX_EMBED_TEXT(
  (SELECT handle FROM DB.DBA.ONNX_MODELS WHERE model_name = 'granite-384-local'),
  'hello world'
);
-- Expect: a 384-element float vector

ONNX_EMBED_TEXT_FOR_MODEL

DB.DBA.ONNX_EMBED_TEXT_FOR_MODEL(model_name, text) is the recommended wrapper. You pass the model name instead of a handle, and the procedure resolves the model, checks whether it is local or remote, and dispatches accordingly. A local model that is registered but not loaded yet is loaded on first use.

-- Local model (uses the ONNX handle):
SELECT DB.DBA.ONNX_EMBED_TEXT_FOR_MODEL('granite-384-local', 'hello world');

-- Remote model (issues an HTTP call to the provider):
SELECT DB.DBA.ONNX_EMBED_TEXT_FOR_MODEL('openai-small', 'hello world');

This is the procedure you should call when you want to embed text manually — for example, to precompute a query vector and reuse it across multiple columns with a remote provider. During normal vector queries you do not call it yourself; the index embeds your query text automatically with its bound model.

DB.DBA.ONNX_EMBED_BATCH_FOR_MODEL(model_name, texts) is its batch form: it takes a vector of strings, returns a vector of embeddings in the same order, and splits the work for you — into requests of provider_max_batch inputs for a remote model, and into calls of 32 texts for a local one.

ONNX_MODEL_REFRESH

DB.DBA.ONNX_MODEL_REFRESH(model_name) loads a registered local model with the options its ONNX_MODELS row describes — paths resolved afresh, plus input_names, output_name, vocab_path, max_seq_len, intra_threads and inter_threads — stores the resulting handle in the row, and returns it:

SELECT DB.DBA.ONNX_MODEL_REFRESH('granite-384-local');

Use it after you change the row's load settings. Because a file already loaded with the same options returns the existing handle, a refresh alone does not pick up a model file you replaced on disk — it returns the old handle, and the old weights keep serving. Unload the old handle first:

SELECT ONNX_UNLOAD_MODEL ((SELECT handle FROM DB.DBA.ONNX_MODELS
                           WHERE model_name = 'my-custom-model'));
SELECT DB.DBA.ONNX_MODEL_REFRESH ('my-custom-model');   -- a new handle

Vectors from different weights are not comparable: if the new file is a different model, rebuild the indexes that use it.

Performance characteristics

Helper thread counts

Each ONNX helper process runs its ONNX Runtime session with one intra-op and one inter-op thread by default. That is deliberate: a helper pool gets its parallelism from running several helpers at once, and an intra-op pool per helper on top of that would oversubscribe the machine and compete with Virtuoso's own threads.

Set them per model with the intra_threads / inter_threads load options:

SELECT ONNX_LOAD_MODEL (
  '/path/to/model.onnx',
  'input=input_ids;input2=attention_mask;output=pooled_output;vocab=/path/to/vocab.json;max_seq_len=512;intra_threads=6');

Note the options are separated by semicolons, not commas. Give the full set of options a model needs: a load without max_seq_len, for example, truncates every text at the helper's default of 128 tokens.

For a registered model — the bundled one included — the row carries them:

UPDATE DB.DBA.ONNX_MODELS SET intra_threads = 4 WHERE model_name = 'granite-384-local';
SELECT DB.DBA.ONNX_MODEL_REFRESH ('granite-384-local');

The refresh loads a new model and moves the row's handle to it; the old one stays loaded until ONNX_UNLOAD_MODEL or a restart. Re-registration at start-up keeps the setting. Measured through the row, ONNX_EMBED_TEXT_FOR_MODEL on 200 FiQA passages one at a time (704 characters on average): 20.6 a second with the column empty, 66.2 with intra_threads = 4 (two rounds each, alternating; 10-core arm64, 2026-10-10).

Measured, bundled granite on a 10-core arm64 machine (Apple M1 Pro), aggregate helper CPU while embedding batches of 32 FiQA passages, 2026-10-10:

Raise intra_threads on a handle you call directly when a single call's latency matters and there is no concurrency to exploit. For building an index, raise __vec_embed_concurrency instead — process level concurrency scales better than intra-op threading, and the two combine.

Remote HTTP embedding providers

Instead of (or in addition to) a local ONNX model, you can register external embedding APIs. Virtuoso presents them through a unified OpenAI-compatible interface: every provider speaks the same request and response shape, so the only differences are the endpoint URL, the model name, the dimensions, and whether a key is required.

Built-in providers

• OpenAI is a hosted commercial API. You must supply an API key, and usage is billed.
• Ollama and LM Studio are local model servers you run yourself. They default to loopback endpoints and need no key, but the server process must be running and must have the model loaded.
• openai_compatible is the catch-all for any service that implements the OpenAI embeddings request/response contract — for example, a self-hosted vLLM, TEI, or LiteLLM gateway. You provide the endpoint explicitly; registering one without an endpoint fails with An endpoint is required for openai_compatible models.

Endpoint security

• HTTPS, except on a private network. An endpoint must use https://; plain http:// is accepted only for a loopback or private-network host (localhost, 127.*, 10.*, 172.16–172.31.*, 192.168.*, link-local and IPv6 private addresses). The host is classified by its literal text, so any other host name counts as public. Registration and every request check it:

  *** Error 42000: [Virtuoso Driver][Virtuoso Server]HTTP embedding endpoint http://embeddings.example.com/v1/embeddings must use https:// -- plain http is allowed only to a loopback or private-network host (refusing to send an API key in cleartext to a public host).: ONNXP

• Redirects are not followed, so a key is never re-sent to another host.

API key management

Provider API keys are stored in the Virtuoso registry, never in the ONNX_MODELS table. The table holds only the registry name of a key.

DB.DBA.ONNX_API_KEY_SET

DB.DBA.ONNX_API_KEY_SET(key_name, api_key);

DB.DBA.ONNX_API_KEY_SET stores a key under a registry name of your choosing. It is DBA-only. The conventional name for the OpenAI key is ONNX_API_KEY_OPENAI, which is the name the openai provider auto-defaults to when you register a model without specifying a key.

DB.DBA.ONNX_API_KEY_SET('ONNX_API_KEY_OPENAI', '<your-secret-key>');

DB.DBA.ONNX_PROVIDER_API_KEY_SET

DB.DBA.ONNX_PROVIDER_API_KEY_SET(provider, api_key);

DB.DBA.ONNX_PROVIDER_API_KEY_SET is a convenience wrapper (also DBA-only). It maps a provider name to its conventional registry name, stores the key there, and returns the name it used:

DB.DBA.ONNX_PROVIDER_API_KEY_SET('openai', '<your-secret-key>');
-- Equivalent to: DB.DBA.ONNX_API_KEY_SET('ONNX_API_KEY_OPENAI', '<your-secret-key>');

Only the openai provider falls back to its conventional name when a model is registered without one. For the other providers, pass the registry name as _api_key_name when you register the model, or no key is sent.

Key visibility and safety

Registering a provider model

Remote models are registered with DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (DBA-only). The procedure creates (or replaces) a row in ONNX_MODELS with the provider connection details. It does not contact the provider.

Signature

DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (
  _model_name,        -- alias used in USING '...' for CREATE VECTOR INDEX
  _provider,          -- 'openai', 'ollama', 'lmstudio', or 'openai_compatible'
  _remote_model,      -- the provider's own model name (e.g. 'text-embedding-3-small')
  _dimensions,        -- expected output dimension; must match the provider model
  _endpoint        := NULL,   -- null = use the provider's default endpoint
  _api_key_name    := NULL,   -- registry NAME of the key, NOT the key itself
  _timeout         := 30,     -- HTTP timeout in seconds, 1 to 300
  _description     := NULL,   -- optional free-text description
  _insecure        := 0);     -- passed to the HTTP client's insecure option

Examples

-- 1. OpenAI text-embedding-3-small (1536 dimensions)
--    Store the key first (DBA-only):
DB.DBA.ONNX_API_KEY_SET('ONNX_API_KEY_OPENAI', '<your-secret-key>');

DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (
  'openai-small', 'openai',
  'text-embedding-3-small', 1536,
  NULL, 'ONNX_API_KEY_OPENAI', 30,
  'OpenAI text-embedding-3-small');

-- 2. Local Ollama (no key needed; default endpoint)
DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (
  'ollama-mxbai', 'ollama',
  'mxbai-embed-large', 1024,
  NULL, NULL, 30,
  'Ollama mxbai-embed-large');

-- 3. Local LM Studio (no key needed; default endpoint)
DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (
  'lmstudio-bge', 'lmstudio',
  'bge-large-en-v1.5', 1024,
  NULL, NULL, 30,
  'LM Studio BGE large');

-- 4. Any OpenAI-compatible custom endpoint (key as needed)
DB.DBA.ONNX_API_KEY_SET('ONNX_API_KEY_MYHOST', '<your-secret-key>');

DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL (
  'my-remote', 'openai_compatible',
  'bge-large-en', 1024,
  'https://my-host.example.com/v1/embeddings', 'ONNX_API_KEY_MYHOST', 30,
  'Self-hosted BGE via LiteLLM');

After registration, reference the model by its _model_name alias in DDL:

CREATE VECTOR INDEX DOCS_VEC ON DB.DBA.DOCS (content)
USING 'openai-small'
WITH (dimensions = 1536, distance_metric = 'cosine',
      hnsw_m = 32, ef_construction = 400);

HTTP request details

When the server needs an embedding from a remote model, it performs the following exchange with the provider endpoint:

Request — an HTTP POST to the configured endpoint with a JSON body. A single text sends a string; a batch sends an array (see Request batching):

{
  "model": "text-embedding-3-small",
  "input": ["first text to embed", "second text to embed"]
}

Authorization — when the model has a key name, Virtuoso sends an Authorization: Bearer <key> header, where <key> is the value stored in the registry under the provider_api_key_name of the model row. Store the key before the model is first used; a provider that requires one rejects the requests (401) otherwise. A model registered with no key name (Ollama or LM Studio, typically) sends no Authorization header.

Response parsing — the server reads the JSON response and extracts every data[].embedding, placing each vector by the index the provider reports rather than by the order the items arrive. This is the OpenAI embeddings response shape, which all supported providers implement. Providers are not required to answer in order, and pairing by arrival would silently attach one text's vector to another text, so a response whose item count does not match the request, or which repeats or omits an index, fails with an error instead.

Dimension validation — the server checks that the length of every returned embedding matches the _dimensions value registered for the model, and fails the call rather than storing a mismatched vector:

HTTP embedding dimension 3 at batch position 0 does not match registered model dimension 4

Retries and errors — a 429, 500, 502, 503 or 504, a timeout and a connection error are retried, up to three attempts in all, waiting for a numeric Retry-After (at most 30 seconds) or else 0.5 s and then 1 s. Any other status fails at once. The caller then gets a generic error such as HTTP embedding provider request failed after 1 attempt(s); see the server log for the provider response., the server log gets one line per failed attempt, and the model row's provider_consecutive_failures, provider_last_error and provider_last_error_at record the failure until the next success.

Request batching

A backfill or refresh embeds texts in batches of __vec_embed_batch_size (32 by default) and sends each batch as one request per provider_max_batch inputs, rather than one request per text. With the defaults, 57,600 documents take 1,800 requests instead of 57,600; with __vec_embed_batch_size raised to 128 — OpenAI's default request size — they take 450.

Defaults are deliberately well below each provider's documented ceiling:

OpenAI accepts 2048 inputs per request but also caps a request at 300k tokens, so a 2048-input batch of ordinary documents breaks the token limit long before the input limit — the win is in going from 1 to ~100, not in reaching 2048. Local servers hold the whole batch in memory, so they get a smaller default.

Override per model when a gateway or self-hosted server has a different limit:

UPDATE DB.DBA.ONNX_MODELS
   SET provider_max_batch = 64
 WHERE model_name = 'my-openai-compatible-model';

Choosing a model

The right model depends on your priorities across latency, cost, quality, privacy, and vector dimensions. Use this comparison to decide.

Comparison

The remote latencies are typical ranges, not measurements; measure from your own host.

Factors in detail

• Latency — Local ONNX embedding of a short text takes about 5 ms (measured), including the round trip to the helper process on the same host. A remote provider adds a network round-trip: hosted OpenAI is the slowest (hundreds of milliseconds from a typical data center), while a local Ollama or LM Studio server on the same host is faster but still slower than the bundled model's helper. This matters most for interactive query latency, since each query embeds the query text.
• Cost — The bundled granite model and any self-hosted Ollama/LM Studio model are free to run (you own the hardware). OpenAI is a paid API billed per token.
• Quality — The bundled model has 30M parameters. Larger models — OpenAI's text-embedding-3-small, or larger self-hosted BGE or mxbai-embed-large checkpoints — can rank better, especially on longer or more technical text; measure on your own data before switching, since a model change means rebuilding every index that uses it.
• Privacy — With a local ONNX model, your text never leaves the Virtuoso host. With Ollama or LM Studio, text goes to that server — over loopback when it runs on the same host. With OpenAI, your text is sent to a third-party API. If your data is sensitive or regulated, prefer a local model.
• Dimensions — Dimension count affects index memory and HNSW build quality. 384-dim vectors from the bundled model are compact; 1536-dim OpenAI vectors are larger and benefit from higher hnsw_m / ef_construction. The dimensions you choose are fixed for the life of the index. Some models support Matryoshka Representation Learning (MRL), which allows you to truncate vectors to a lower dimension while retaining most of the semantic quality — for example, OpenAI text-embedding-3-small (1536-dim native) can be reduced to 256 or 512 dimensions via WITH (reduce = 'truncate', dimensions = 256). See Index parameters: Matryoshka reduction.

Recommendation

Remember that a vector index is bound to one model for its lifetime. To change models, drop the index, register or confirm the new model, and recreate the index with the new model name and dimensions.

Using a custom ONNX model

You are not limited to the bundled granite model. You can register any ONNX text-embedding model that meets the requirements below. This is useful when you need a different language, a larger model for better quality, or a quantized model for a memory-constrained host.

Model file requirements

• Format: an ONNX graph (.onnx).
• Inputs: two int64 tensors of shape [batch, sequence] — token ids and attention mask, by default named input_ids and attention_mask. The helper tokenizes the text and fills both.
• Output: one pooled vector per input, shape [batch, dimension]. The helper does no pooling and no normalisation: it returns the output tensor's row as it is, so a graph whose output is last_hidden_state (one vector per token) does not work. A stock Hugging Face ONNX export has exactly that output; the export tool adds the pooling.
• A tokenizer the helper implements: a WordPiece vocab.txt (BERT-style, uncased), or a byte-level BPE vocab.json with merges.txt in the same directory (RoBERTa- and GPT-2-style). The file extension decides which: a path ending in .json is BPE. SentencePiece tokenizers are not supported.
• A maximum sequence length: longer text is truncated to it. The helper accepts 2–8192 tokens and defaults to 128 when none is given.

How text is normalised before tokenisation

This section applies to WordPiece models (vocab.txt). A byte-level BPE model — the bundled granite among them — is cased: the helper strips leading and trailing whitespace and then tokenises exactly as Hugging Face's RoBERTa tokenizer does, which the test suite checks against Hugging Face's own token ids.

For WordPiece, the server normalises text the way BERT's own tokenizer does with do_lower_case=True — lowercase, then decompose and drop combining marks — so that cafe, CAFE, café and CAFÉ all reach the model as the same token. Text that arrives already decomposed (cafe + U+0301) normalises identically. Because lowercasing is always applied, the WordPiece path suits uncased vocabularies.

Covered:

For these, Virtuoso produces the same token ids — and so the same embedding — as transformers does on the same vocabulary.

Not covered, so worth knowing before you index such a corpus:

• CJK. BERT inserts spaces around CJK codepoints so each character becomes its own token; Virtuoso does not. In practice this matters less than it sounds with an English vocabulary: bert-base-uncased holds only 488 CJK tokens, so transformers itself tokenises most Chinese to [UNK] and Virtuoso's answer is already close to it. It becomes a real limitation with a CJK-capable vocabulary — which in practice means a multilingual checkpoint, and those usually want SentencePiece rather than WordPiece.
• Greek final sigma. A word-final capital sigma lowercases to U+03C2 in Python and to U+03C3 here; a per-codepoint table cannot see word position. Both are in the vocabulary, so a word ending in sigma may tokenise one piece differently.
• Full Unicode NFD. Only the blocks above decompose.

Registering a custom model

Place the .onnx model file and the tokenizer files on the Virtuoso host where the server process can read them. Then register the model in ONNX_MODELS and load it, or load it directly with ONNX_LOAD_MODEL.

The options string of ONNX_LOAD_MODEL tells the runtime how to feed and read the model. The format is a semicolon-separated list of key=value pairs:

input=input_ids;input2=attention_mask;output=pooled_output;vocab=/path/to/vocab.txt;max_seq_len=256

The export tool that ships with the plugin, virtuoso-export-embedding-model (installed in bin/), turns a Hugging Face sentence-transformers model into this form. It reads the model's own pooling and normalisation configuration rather than assuming them, wraps the transformer in them, exports a graph with the inputs input_ids and attention_mask and the output pooled_output, and copies the tokenizer files beside it. --dry-run reads the configuration only — no PyTorch, and no network for a local directory:

virtuoso-export-embedding-model ./granite-embedding-30m-english /dev/null --dry-run
# plan: pooling=cls normalize=1 inferred=0 inputs=input_ids,attention_mask output=pooled_output max_seq_len=512 tokenizer=bpe

A real export needs torch and transformers, and verifies its output against the source model. For a model published in bf16, --bf16-weights halves the file without changing a single output value; the bundled model is stored that way. When it finishes, the tool prints the ONNX_LOAD_MODEL call and the registration for the files it wrote. Register the model with its output_shape as well — a vector index needs the dimension from it:

INSERT INTO DB.DBA.ONNX_MODELS (model_name, model_path, model_type,
    input_names, output_name, output_shape, vocab_path, max_seq_len)
  VALUES ('my-custom-model', '/opt/models/my-model/model.onnx', 'text_embedding',
    'input_ids,attention_mask', 'pooled_output', '1,384',
    '/opt/models/my-model/vocab.json', 512);

-- Load it; returns the handle:
SELECT DB.DBA.ONNX_MODEL_REFRESH ('my-custom-model');

-- Verify it produces the expected dimension (384 here):
SELECT length (DB.DBA.ONNX_EMBED_TEXT_FOR_MODEL ('my-custom-model', 'hello world'));

Without output_shape, the model embeds text but cannot back a vector index: CREATE VECTOR INDEX … USING 'my-custom-model' fails with VEC13: Model my-custom-model does not declare an output shape, so this index needs WITH (dimensions = <n>)., and even with dimensions given, the backfill fails with Vector index DOCS_C_VEC: its model my-custom-model is not available: VEC52. Set the column and the index builds:

UPDATE DB.DBA.ONNX_MODELS SET output_shape = '1,384' WHERE model_name = 'my-custom-model';

To replace the model file later, unload the old handle before refreshing — see ONNX_MODEL_REFRESH.

Next steps

• SQL vector index how-to — Create, query, and maintain a relational SQL vector index bound to one of these models
• SPARQL vector index how-to — Use vector indexes with RDF data and the bif:vector_search magic predicate
• Index parameters — HNSW build and query parameters that control the accuracy/speed trade-off

Related content

Reference:

• ONNX Runtime — Microsoft ONNX Runtime project
• granite-embedding-30m-english — The bundled embedding model on Hugging Face
• OpenAI embeddings docs — OpenAI embeddings API reference

PreviousVector search over RDF literals (knnSimilar) NextIndex parameters and tuning for the Virtuoso vector index