Usage on Vchord

Indexing

Mon, 01 Jan 0001 00:00:00 +0000

VectorChord’s index type vchordrq divides vectors into lists and searches only a subset of lists closest to the query vector. It provides fast build time and low memory consumption, while delivering significantly better performance than both hnsw and ivfflat.

Let’s start by creating a table named items with an embedding column of type vector(n), and then populate it with sample data.

CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(3));
INSERT INTO items (embedding) SELECT ARRAY[random(), random(), random()]::real[] FROM generate_series(1, 1000);

To create a vchordrq index, you can use the following SQL.

Multi-Vector Retrieval

Mon, 01 Jan 0001 00:00:00 +0000

since v0.3.0

Multi-Vector Retrieval is an advanced technique used in Retrieval-Augmented Generation (RAG) systems to enhance document retrieval by leveraging multiple vectors per document.

Unlike Single-Vector Retrieval, Multi-Vector Retrieval using the MaxSim operator enables capturing fine-grained semantic relationships.

Refer to our blog for more details on building an end-to-end Multi-Vector Retrieval application.

How to start

Late interaction models embed documents and queries as vector arrays separately and compute similarity through an operator called MaxSim. The definition of the MaxSim operator is $\sum_i \max_j q_i \cdot d_j$ , meaning that for each vector in the query vector array, the closest vector in the document vector array is found, their dot product is calculated, and the results are summed.

Graph Index

Mon, 01 Jan 0001 00:00:00 +0000

since v0.5.0

VectorChord’s index type vchordg is a disk-based graph index. It has low memory consumption.

Let’s start by creating a table named items with an embedding column of type vector(n), and then populate it with sample data.

To create a vchordg index, you can use the following SQL.

CREATE INDEX ON items USING vchordg (embedding vector_l2_ops);

Tip

This feature is in preview.

Quantization Types

Mon, 01 Jan 0001 00:00:00 +0000

since v1.1.0

Quantization can be applied not only to indexes but also directly to vectors. To this end, VectorChord provides data types for quantized vectors.

rabitq8 is the data type of a quantized vector. For the same dimension, its cost is about $\frac{1}{4}$ that of a full-precision vector (vector) and $\frac{1}{2}$ that of a half-precision vector (halfvec). This type is named after its quantization algorithm.¹

Similarity Filter

Mon, 01 Jan 0001 00:00:00 +0000

Usually, ORDER BY and LIMIT clauses are used to perform vector search. In some cases, you may want to use a WHERE clause to filter out vectors that are far from the query. Naturally, you might write the following SQL.

EXPLAIN (COSTS FALSE) 
SELECT * FROM items WHERE embedding <-> '[0, 0, 0]' < 0.1 ORDER BY embedding <-> '[0, 0, 0]' LIMIT 10;
                                  QUERY PLAN                                   
-------------------------------------------------------------------------------
 Limit
   ->  Index Scan using items_embedding_idx on items
         Order By: (embedding <-> '[0,0,0]'::vector)
         Filter: ((embedding <-> '[0,0,0]'::vector) < '0.1'::double precision)

The returned results meet expectations, while this approach suffers from poor performance. This is because if fewer than $10$ vectors fall within the distance threshold, PostgreSQL forces the index to continue to search to get $10$ results, exhausting the entire search space.

PostgreSQL Tuning

Mon, 01 Jan 0001 00:00:00 +0000

Indexing may be parallelized. Parallelism increases server load and is therefore not enabled by default in PostgreSQL. To enable it, refer to the following SQL.

-- Sets the maximum number of background processes that the cluster can support.
-- It's recommended to set it to at least the sum of your desired parallelism plus 8.
-- For example, if you want to use 16 threads for parallel processing, set the value at least to 24.
-- Depending on your workload, this value might be useful to be higher.
-- Note: A restart is required for this setting to take effect.
ALTER SYSTEM SET max_worker_processes = 24;

-- Sets the maximum number of workers that the cluster can support for parallel operations.
-- It's recommended to set it to the sum of your desired parallelism minus 1.
-- For example, if you want to use 16 threads for parallel processing, set the value to 15.
SET max_parallel_workers = 15;

-- Sets the maximum number of parallel workers that can be started by a single utility command.
-- It's recommended to set it to the sum of your desired parallelism minus 1.
-- For example, if you want to use 16 threads for parallel processing, set the value to 15.
SET max_parallel_maintenance_workers = 15;

The number of parallel workers also depends on the table’s configuration. By default, this is automatically determined by PostgreSQL. If PostgreSQL identifies it as disabled, parallel indexing will not take effect. You can override PostgreSQL’s behavior by adjusting the table’s parallel_workers setting to enable parallel indexing.

Monitoring

Mon, 01 Jan 0001 00:00:00 +0000

You can monitor the progress of building a PostgreSQL index by querying the pg_stat_progress_create_index view.

SELECT command, phase, round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS "%" FROM pg_stat_progress_create_index;

Reference

Phases (`vchordrq`)

vchordrq

There are 5 steps in the index construction process of vchordrq. The phase name of each step are as follows:

Fallback Parameters

Mon, 01 Jan 0001 00:00:00 +0000

since v1.1.0

Search parameters are typically configured via GUCs. VectorChord also offers an alternative method, allowing these parameters to be attached directly to an index instead of a session or transaction. In other words, you can store them as part of the index’s storage parameters.

You could set probes at index building using the following syntax.

CREATE INDEX items_embedding_idx ON items USING vchordrq (embedding vector_l2_ops) WITH (options = $$
build.internal.lists = [1000]
$$, probes = '10');

You can now perform searches without setting GUCs:

Measure Recall

Mon, 01 Jan 0001 00:00:00 +0000

since v0.5.0

In the context of vector search, recall is the ratio of true nearest neighbors to approximate nearest neighbors returned by the index. For example, if the index retrieves $100$ approximate nearest neighbors and $97$ of them are true nearest neighbors, then the recall is $\frac{97}{100} = 0.97$ .

Tip

This feature is in preview.

Prewarm

Mon, 01 Jan 0001 00:00:00 +0000

After PostgreSQL starts, tables and indexes are loaded from storage into the buffer pool only when they are accessed for the first time. As queries are executed, the frequently accessed parts of the index are naturally cached in the buffer pool. However, as initial queries are executed, the buffer pool is empty, which causes the queries to be slow.

The PostgreSQL extension pg_prewarm includes the pg_prewarm function, which can partially address this issue. After PostgreSQL starts, you simply provide the name of the index to this function, and it will load the index from storage into the buffer pool.

Prefilter

Mon, 01 Jan 0001 00:00:00 +0000

since v0.4.0

In a filtered index scan, the index performs the scan first and then PostgreSQL checks whether the filter conditions are satisfied. On this page, “filter” refers to the filter on the index scan node of the query plan in PostgreSQL. This is usually constructed from the WHERE clause in the SQL statement. The filter does not need to have a specific form.

EXPLAIN (COSTS FALSE)
SELECT * FROM items WHERE id <= 5000 AND vector_norm(embedding) < 0.5;
                          QUERY PLAN                          
--------------------------------------------------------------
 Index Scan using items_pkey on items
   Index Cond: (id <= 5000)
   Filter: (vector_norm(embedding) < '0.5'::double precision)

However, this is not always the most efficient approach for vector search. Consider the following query.

Prefetch

Mon, 01 Jan 0001 00:00:00 +0000

since v0.4.0

Prefetch dramatically accelerates queries that involve I/O operations. It works by proactively reading buffers into operating system page cache or PostgreSQL’s buffer pool based on the actual needs of the index.

Prefetch helps overlap I/O with computation, as well as I/O with other I/O operations, significantly reducing the time spent waiting for buffers during query execution, thereby improving overall query performance.

Tip

This feature is enabled by default. Any query can benefit from prefetching with VectorChord 0.4.0 and later versions.

Rerank in Table

Mon, 01 Jan 0001 00:00:00 +0000

since v0.2.1

PostgreSQL indexes store row identifiers and the values of the indexed columns. Typically, indexed columns have simple data types, for example, integers or strings with short length limits.

However, if the indexed column is of a vector type, both the table and the index store vectors, which means you pay twice the storage cost for the vectors. If you would like to reduce this cost, you can enable rerank_in_table when building the index. The index will then use a tricky approach where it does not store the vectors itself but reads them from the table when needed.

Partitioning Tuning

Mon, 01 Jan 0001 00:00:00 +0000

For large tables, the partitioning phase dominates both build time and memory consumption.

Time complexity of partitioning is $O(FC^{2}DL)$ , and the memory consumption in bytes can be approximated by $4DC(F + T + 1)$ ,

where:

$D$ is the vector dimension, or build.internal.kmeans_dimension if set.
$F$ is build.internal.sampling_factor.
$C$ is build.internal.lists[-1].
$T$ is build.internal.build_threads.
$L$ is build.internal.kmeans_iterations.

This page introduces some tuning techniques to improve build time and memory consumption.

External Build

Mon, 01 Jan 0001 00:00:00 +0000

In addition to using the methods in Partitioning Tuning to achieve shorter build time and lower memory consumption at the cost of QPS and recall, you can also choose to run the partitioning phase offline on other machines or on GPUs. This feature is not supported by vchordg, since it does not have a partitioning phase.

Assume the table is t and the column to be indexed is val.

CREATE TABLE t (val vector(3));

Specifically, you need to sample the column for which the index will be built.

Usage on Vchord

Indexing

Multi-Vector Retrieval

How to start

Graph Index

Quantization Types

Similarity Filter

PostgreSQL Tuning

Monitoring

Reference

Phases (vchordrq)

Fallback Parameters

Measure Recall

Prewarm

Prefilter

Prefetch

Rerank in Table

Partitioning Tuning

External Build

Phases (`vchordrq`)