Benchmarks on Vchord

Price Benchmark

Mon, 01 Jan 0001 00:00:00 +0000

Using VectorChord, you can achieve a QPS of 131 with 0.95 precision on 100 million 768-dimensional vectors for the top 10 queries. This setup costs only $250 monthly and can be hosted on a single machine. If you are interested in the details of our implementation, please refer to our blog VectorChord: Store 400k Vectors for $1 in PostgreSQL.

This means you can store 400k vectors for only $1, allowing you to save significantly: 6x more vectors compared to Pinecone (storage optimized instance) and 26x more than pgvector/pgvecto.rs for the same price.

Performance Benchmarking

Mon, 01 Jan 0001 00:00:00 +0000

Here we present a comprehensive benchmark comparing our implementation of VectorChord with other vector databases, focusing on QPS and scalability. If you are interested in the details of our implementation, please refer to our blog VectorChord: Store 400k Vectors for $1 in PostgreSQL.

QPS

LAION 5M

In LAION 5M dataset, VectorChord achieved more than 3x QPS at 0.95 recall compared to different vector databases, including MyScale, pgvector, pinecone, weaviate, qdrant and zilliz(milvus).

Benchmark with pgvector

Mon, 01 Jan 0001 00:00:00 +0000

In this section, we compare VectorChord with pgvector, a PostgreSQL extension that provides vector similarity search capabilities. We will focus on four key metrics: QPS (Queries Per Second), Recall@100, Index Building Time, and Insertion Time. If you are interested in the details of our implementation, please refer to our blog Vector Search Over PostgreSQL: A Comparative Analysis of Memory and Disk Solutions.

QPS & Recall

Here are some initial benchmark results for the GIST dataset, which consists of 1 million vectors in 960 dimensions. With equivalent recall, VectorChord’s QPS could be twice that of pgvector. More details will be provided in the Benchmark section.

Benchmark with Elasticsearch

Mon, 01 Jan 0001 00:00:00 +0000

Here we present a comprehensive benchmark comparing our implementation of VectorChord-BM25 with ElasticSearch, focusing on two key metrics: QPS (Queries Per Second) and NDCG@10 (Normalized Discounted Cumulative Gain at rank 10). If you are interested in the details of our implementation, please refer to our blog VectorChord-BM25: Revolutionize PostgreSQL Search with BM25 Ranking — 3x Faster Than ElasticSearch.

RPS

We compare the Top1000 Queries Per Second (QPS)—a metric that measures how many queries a system can process per second while retrieving the top 1000 results for each query—between our implementation and ElasticSearch across various datasets from bm25-benchmarks. On average, our implementation achieves 3 times higher QPS compared to ElasticSearch across the tested datasets, showcasing its efficiency and scalability.

Benchmark with pgvectorscale

Mon, 01 Jan 0001 00:00:00 +0000

In this section, we compare VectorChord with pgvectorscale, a PostgreSQL extension that provides vector similarity search capabilities. We will focus on four key metrics: QPS (Queries Per Second), Recall@10/100, Index Building Time, and Insertion Time. Memory and disk performance will also be discussed. If you are interested in the details of our implementation, please refer to our blog Vector Search Over PostgreSQL: A Comparative Analysis of Memory and Disk Solutions.