--- title: "Tutorial: Semantic Search Over OpenLink FAQs with a SPARQL Vector Index" description: "Learn how to load OpenLink's FAQ Turtle documents into Virtuoso, create a SPARQL vector index over schema.org Question entities, and perform semantic search using bif:vector_search." url: urn:osdi:this dateModified: 2026-10-31T21:57:28Z language: en --- One page of the Vector Index documentation, MERGED by build_vector_index_docs.py from this template and the markdown in fixture/vector-index/. Do not edit the generated file: change this template or the generator and rebuild. It opens on the documentation header (dxshell.py), not on the site hero. [Home](urn:osdi:site)/[Virtuoso](urn:osdi:sitevirtuoso/)/[Vector index](urn:osdi:sitevirtuoso/docs/vector-index/)/Tutorial: Semantic search over OpenLink FAQs with a SPARQL vector index # Tutorial: Semantic search over OpenLink FAQs with a SPARQL vector index Learn how to load OpenLink's FAQ Turtle documents into Virtuoso, create a SPARQL vector index over schema.org Question entities, and perform semantic search using bif:vector_search. Start [Overview](urn:osdi:sitevirtuoso/docs/vector-index/) [Quickstart](urn:osdi:sitevirtuoso/docs/vector-index/quickstart/) Tutorial: FAQ semantic search Build [SQL vector index how-to](urn:osdi:sitevirtuoso/docs/vector-index/howto-sql-vector-index/) [SPARQL vector index how-to](urn:osdi:sitevirtuoso/docs/vector-index/howto-sparql-vector-index/) [Search RDF literals (knnSimilar)](urn:osdi:sitevirtuoso/docs/vector-index/knn-similar/) Configure [Embedding models](urn:osdi:sitevirtuoso/docs/vector-index/embedding-models/) [Index parameters](urn:osdi:sitevirtuoso/docs/vector-index/index-parameters/) [Similarity metrics](urn:osdi:sitevirtuoso/docs/vector-index/similarity-metrics/) Operate [VVec Console (web UI)](urn:osdi:sitevirtuoso/docs/vector-index/vvec-console/) [Troubleshooting](urn:osdi:sitevirtuoso/docs/vector-index/troubleshooting/) Reference [API reference](urn:osdi:sitevirtuoso/docs/vector-index/api-reference/) This tutorial walks through a complete, real-world example: loading OpenLink's public FAQ documents (published as Turtle/RDF using schema.org vocabulary), creating a SPARQL vector index over the `schema:Question` entities, and performing semantic similarity search with `bif:vector_search`. This tutorial covers: - **Loading RDF data**: Bulk-loading FAQ Turtle documents via `SPARQL LOAD` - **Creating a vector index**: Using a general selector with `schema:Question` entities - **Refreshing and draining**: Populating the HNSW graph with embeddings - **Semantic search**: Querying FAQs by meaning, not keyword matching - **Interpreting results**: Understanding scores and distances Note This tutorial uses the SPARQL vector index surface. For the SQL surface, see [SQL vector index how-to](urn:osdi:sitevirtuoso/docs/vector-index/howto-sql-vector-index/). For an overview of both surfaces, see [Vector index overview](urn:osdi:sitevirtuoso/docs/vector-index/). --- ## Prerequisites Before you start, make sure you have: - **Virtuoso Open Source** built with ONNX support and the `vonnx` plugin loaded - **The bundled `granite-384-local` model** auto-provisioned at startup (384 dimensions) - **Network access** to `virtuoso.openlinksw.com` and `www.openlinksw.com` for loading the FAQ documents - **DBA access** to the Virtuoso instance (via iSQL on port 1111 or similar) **Verify the model is loaded:** SQL ``` SELECT model_name, handle FROM DB.DBA.ONNX_MODELS WHERE model_name = 'granite-384-local'; -- Expect: handle = 1 (non-null means loaded) ``` --- ## Step 1: Load the FAQ documents OpenLink publishes a collection of FAQ documents as Turtle/RDF using the schema.org vocabulary. Each document contains `schema:Question` entities with `schema:name` (the question text) and associated `schema:Answer` entities. Load all FAQ sources into a single graph (``): SQL ``` SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; ``` This query runs as follows: 1. **`SPARQL LOAD`** fetches each Turtle document from the remote URL. 2. **`INTO GRAPH`** places all triples into the named graph ``. 3. Multiple loads into the same graph accumulate — all FAQ sources merge into one searchable collection. **Verify the data loaded:** SQL ``` SPARQL DEFINE input:storage "" SELECT (COUNT(*) AS ?total) WHERE { GRAPH { ?s ?p ?o } } ; ``` SQL ``` SPARQL DEFINE input:storage "" SELECT (COUNT(*) AS ?questions) WHERE { GRAPH { ?q a . } } ; ``` Tip The number of questions will vary as the FAQ documents are updated. Expect dozens to hundreds of `schema:Question` entities across all sources — on 2026-10-20 the eight documents gave 1,350 triples and 87 questions, 68 of which have a `schema:name`. Those 68 are what the index below embeds. --- ## Step 2: Create the vector index Create a vector index over the `schema:Question` entities, embedding the question text from `schema:name`. This uses a **general selector** because we need a graph pattern to filter for `schema:Question` types specifically. SQL ``` SPARQL DEFINE input:storage "" CREATE VECTOR INDEX KEY ?question EMBEDDING ?questionName WHERE { GRAPH { ?question a ; ?questionName . } } USING MODEL 'granite-384-local' WITH (dimensions = 384, distance = 'cosine') ; ``` This query runs as follows: 1. **`key ?question`** — the subject IRI of each FAQ question becomes the searchable entity identifier. 2. **`embedding ?questionName`** — the `schema:name` literal (the question text) is what gets embedded into a vector. 3. **`where { ... }`** — the graph pattern selects only `schema:Question` entities with a `schema:name` from the `` graph. 4. **`using model 'granite-384-local'`** — binds the index to the bundled 384-dimensional granite model. 5. **`with (dimensions = 384, distance = 'cosine')`** — cosine distance is the standard choice for text similarity. **Key concepts:** - The `KEY` variable must identify the entity IRI — here `?question`. - The `EMBEDDING` variable must be the text literal to embed — here `?questionName`. - The two variables must be different. - The catalog row is stored in `DB.DBA.SPARQL_VEC_INDEX` with `SVI_STATE = 'active'`. **Verify the index was created:** SQL ``` SELECT SVI_IRI, SVI_STATE, SVI_SELECTOR_TYPE, SVI_MODEL_NAME, SVI_DIMENSIONS FROM DB.DBA.SPARQL_VEC_INDEX WHERE SVI_IRI = 'urn:openlink:faqs:embeddings'; ``` --- ## Step 3: Refresh the index After creating the index, run a refresh to evaluate the selector pattern, embed each question's text, and populate the HNSW graph: SQL ``` SPARQL DEFINE input:storage "" refresh VECTOR INDEX ; ``` Output ``` Vector index refreshed, 68 rows processed, 0 stale keys deleted ``` Refresh evaluates the `WHERE` pattern, embeds each `?questionName` value with the bound model, and inserts the resulting vectors into the HNSW graph — one row per question that has a name. For a local model like the bundled one, no network is involved: the embeddings come from helper processes on the same host. For a remote provider, the questions are sent in batches of HTTP requests. Note This index uses the default `maintenance = 'manual'`: when the FAQ graph changes, run `REFRESH VECTOR INDEX` again — it embeds only the questions that changed. An index created with `maintenance = 'async'` queues such changes instead; they become searchable when the queue is drained with `SELECT DB.DBA.VEC_INDEX_QUEUE_DRAIN(1000);`. --- ## Step 4: Search the FAQs Now you can search the FAQ collection by semantic meaning. The `bif:vector_search` magic predicate embeds your query text with the same model and searches the HNSW graph for the most similar questions: SQL ``` SPARQL PREFIX schema: SELECT ?doc ?name ?score ?distance WHERE { ?doc schema:name ?name ; bif:vector_search "Virtuoso Data Virtualization Support" option ( INDEX , score ?score, distance ?distance, score_limit 0.85, distance_limit 0.30, LIMIT 10, ef_search 100 ) . } ORDER BY DESC(?score) ; ``` The first three of the ten rows it returned on 2026-10-20: | ?name | ?score | ?distance | | --- | --- | --- | | Does OpenLink Virtuoso support data virtualization? | 0.937 | 0.126 | | How does Virtuoso aid digital transformation? | 0.903 | 0.194 | | What are the key use cases for Virtuoso? | 0.899 | 0.202 | This query runs as follows: 1. **`bif:vector_search "Virtuoso Data Virtualization Support"`** — the query text is embedded once using the index's bound model, `granite-384-local`. 2. **HNSW search** finds the 10 nearest neighbor questions in the vector space. 3. **`?doc`** binds to the matching `schema:Question` IRI. 4. **`?name`** binds to the question's `schema:name` text. 5. **`?score`** and **`?distance`** bind the similarity score and raw distance. 6. **`score_limit 0.85`** filters out low-similarity results. 7. **`ORDER BY DESC(?score)`** returns the most similar questions first. ### Search options explained | Option | Value | Purpose | | --- | --- | --- | | `index ` | Explicit index IRI | Required when multiple indexes exist | | `score ?score` | Bind variable | Higher = more similar, from 0 to 1; for cosine it is `1 - ?distance / 2` (1 = identical), the score `X_SIMILAR` gives in SQL | | `distance ?distance` | Bind variable | Lower = more similar (0 = identical) | | `score_limit 0.85` | Threshold | Exclude results scoring below 0.85 | | `distance_limit 0.30` | Threshold | Exclude results with distance > 0.30 | | `limit 10` | Result count | Return at most 10 matches | | `ef_search 100` | Search depth | HNSW beam width — higher = better recall | Tip Adjust `score_limit` and `distance_limit` to control how strict the matching is. Lower thresholds return more results (including less relevant ones); higher thresholds return only the closest matches. For a cosine index the two are the same condition — the score is `1 - distance / 2`, so `score_limit 0.85` is `distance_limit 0.30` — and one is enough. Set them from your own data: for this query 31 of the 68 questions clear `score_limit 0.85` with the bundled model, so `limit 10` is what bounds the result here. --- ## Step 5: Try different queries The power of semantic search is that it matches **meaning**, not keywords. Try these queries and observe how different phrasings find the same FAQs: SQL ``` -- Search for scalability questions SPARQL PREFIX schema: SELECT ?doc ?name ?score WHERE { ?doc schema:name ?name ; bif:vector_search "how to scale virtuoso for more data" option (INDEX , score ?score, LIMIT 5, ef_search 100) . } ORDER BY DESC(?score) ; ``` SQL ``` -- Search for high availability SPARQL PREFIX schema: SELECT ?doc ?name ?score WHERE { ?doc schema:name ?name ; bif:vector_search "disaster recovery and high availability" option (INDEX , score ?score, LIMIT 5, ef_search 100) . } ORDER BY DESC(?score) ; ``` SQL ``` -- Search for RDF views / data virtualization SPARQL PREFIX schema: SELECT ?doc ?name ?score WHERE { ?doc schema:name ?name ; bif:vector_search "expose SQL tables as RDF graphs" option (INDEX , score ?score, LIMIT 5, ef_search 100) . } ORDER BY DESC(?score) ; ``` **Key concepts:** - The model captures meaning, not words. With the bundled model on 2026-10-20, "expose SQL tables as RDF graphs" ranked "How do I Generate and Deploy RDF-based Linked Data from Tables in a Relational Database?" first (0.896) and "How do I use R2RML to map RDB to RDF with Virtuoso?" fourth (0.848) — neither says "expose" or "SQL". "disaster recovery and high availability" put the High-Availability and Disaster Recovery question first (0.918), well clear of the next (0.792). - `ef_search 100` provides good recall for most FAQ collections. Increase to 200+ for larger datasets or when you need maximum accuracy. --- ## Step 6: Using a remote embedding model To use a hosted model instead — for example OpenAI's `text-embedding-3-small` (1536 dimensions) in place of the bundled local model — first register the provider, then recreate the index: SQL ``` -- Store the API key: DB.DBA.ONNX_API_KEY_SET('ONNX_API_KEY_OPENAI', ''); -- Register the model: DB.DBA.ONNX_REGISTER_HTTP_EMBED_MODEL( 'openai-small', 'openai', 'text-embedding-3-small', 1536, NULL, 'ONNX_API_KEY_OPENAI', 30); -- Drop the old index: SPARQL DEFINE input:storage "" DROP SILENT VECTOR INDEX ; -- Recreate with the OpenAI model: SPARQL DEFINE input:storage "" CREATE VECTOR INDEX KEY ?question EMBEDDING ?questionName WHERE { GRAPH { ?question a ; ?questionName . } } USING MODEL 'openai-small' WITH (dimensions = 1536, distance = 'cosine', hnsw_m = 32, ef_construction = 400) ; -- Refresh to embed all questions via OpenAI: SPARQL DEFINE input:storage "" refresh VECTOR INDEX ; ``` Important With a remote provider, `REFRESH VECTOR INDEX` sends the questions in batches of 32 texts (`__vec_embed_batch_size`), each batch one HTTP request to the provider, so the 68 questions here take three requests. For hundreds of FAQs this takes seconds; for thousands, it may take minutes. The `hnsw_m = 32` and `ef_construction = 400` parameters are recommended for 1536-dimensional vectors. --- ## Cleanup When you're done with the tutorial, you can drop the index and clear the graph: SQL ``` SPARQL DEFINE input:storage "" DROP SILENT VECTOR INDEX ; SPARQL DEFINE input:storage "" CLEAR GRAPH ; ``` --- ## Complete script Here is the full end-to-end script you can paste into iSQL: SQL ``` -- Step 1: Load FAQ documents SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; SPARQL LOAD INTO GRAPH ; -- Step 2: Create the vector index SPARQL DEFINE input:storage "" CREATE VECTOR INDEX KEY ?question EMBEDDING ?questionName WHERE { GRAPH { ?question a ; ?questionName . } } USING MODEL 'granite-384-local' WITH (dimensions = 384, distance = 'cosine') ; -- Step 3: Refresh to populate the HNSW graph SPARQL DEFINE input:storage "" refresh VECTOR INDEX ; -- Step 4: Semantic search SPARQL PREFIX schema: SELECT ?doc ?name ?score ?distance WHERE { ?doc schema:name ?name ; bif:vector_search "Virtuoso Data Virtualization Support" option ( INDEX , score ?score, distance ?distance, score_limit 0.85, distance_limit 0.30, LIMIT 10, ef_search 100 ) . } ORDER BY DESC(?score) ; ``` --- ## Next steps - **[SPARQL vector index how-to](urn:osdi:sitevirtuoso/docs/vector-index/howto-sparql-vector-index/)** — Complete reference for all SPARQL vector index features - **[SQL vector index how-to](urn:osdi:sitevirtuoso/docs/vector-index/howto-sql-vector-index/)** — Use vector indexes with relational tables - **[Vector index overview](urn:osdi:sitevirtuoso/docs/vector-index/)** — Architecture and concepts ## Related content **Reference:** - [schema.org Question](https://schema.org/Question) — The Question vocabulary used in the FAQ data - [OpenLink Virtuoso FAQ](https://virtuoso.openlinksw.com/faq/) — The live FAQ page - [granite-embedding-30m-english](https://huggingface.co/ibm-granite/granite-embedding-30m-english) — The bundled embedding model [PreviousQuickstart: Vector index for Virtuoso](urn:osdi:sitevirtuoso/docs/vector-index/quickstart/) [NextHow-To: Build and use the relational SQL vector index (Virtuoso)](urn:osdi:sitevirtuoso/docs/vector-index/howto-sql-vector-index/) On this page 1. [Prerequisites](#prerequisites) 2. [Step 1: Load the FAQ documents](#step-1-load-the-faq-documents) 3. [Step 2: Create the vector index](#step-2-create-the-vector-index) 4. [Step 3: Refresh the index](#step-3-refresh-the-index) 5. [Step 4: Search the FAQs](#step-4-search-the-faqs) 6. [Step 5: Try different queries](#step-5-try-different-queries) 7. [Step 6: Using a remote embedding model](#step-6-using-a-remote-embedding-model) 8. [Cleanup](#cleanup) 9. [Complete script](#complete-script) 10. [Next steps](#next-steps) 11. [Related content](#related-content)