Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

Management

Manage projects, clusters, RBAC authorization, backups, upgrades, API keys, Terraform, and external index workflows.

1 - Overview of the cloud object hierarchy

Managing your VectorChord Cloud project requires an understanding of the object hierarchy. The following diagram shows how objects in cloud are related. See bel

Managing your VectorChord Cloud project requires an understanding of the object hierarchy. The following diagram shows how objects in cloud are related. See below for a description of each object.

VectorChord Cloud Account

This is the account you used to sign up with VectorChord Cloud. Now supports signing up with an GitHub account.

Projects

A project is the top-level object in the hierarchy. It is a container for all clusters in the project and billing information. For more information, see Projects.

Clusters

A project can contain one or more clusters. A cluster is a collection of nodes that run one Postgres cluster. For more information, see Clusters.

Compute Instances

A cluster can contain one or more compute instances. A compute instance is a virtual machine that runs a Postgres instance.

Databases

Every cluster contains two default databases: postgres and a database what you named when you created the cluster. You can create additional databases in a cluster with psql after the cluster is ready.

2 - Projects

With VectorChord Cloud, everything starts with the project. It is the top-level object in the hierarchy. A project can hold as many Postgres clusters as needed.

With VectorChord Cloud, everything starts with the project. It is the top-level object in the hierarchy. A project can hold as many Postgres clusters as needed. The maximum number of projects you can create and join is 5.

Learn more about projects and how to manage them in these sections:

Project Setting

In project setting, you can view and edit the project information. You can switch to another project by clicking the project name in the bottom left corner of the page. The default project is the first project we created for you when you signed up. In below diagram, you can change the name, invite members to the project, and leave the project.

Project Permissions

You can invite other users to your project by clicking the Invite button. You can invite users by their email address. The invited user will receive an email with an invitation link. The invited user can accept the invitation by clicking the link.

Warning

If the invited user does not have a VectorChord Cloud account, they will need to sign up for an account before they can accept the invitation.

For more information on access levels of each role, see Authorization.

Project Billing

Every project has a billing page where you can view the billing information. In the billing page, you can add a payment method, view the billing invoices.

3 - Clusters

One or more clusters can be created in a project. A cluster is a collection of nodes that run a Postgres cluster.

One or more clusters can be created in a project. A cluster is a collection of nodes that run a Postgres cluster.

Create a cluster

To create a new cluster, navigate to the VectorChord Cloud Console and click on the “Create Cluster” button.

You need to specify the following information:

  • Required:
    • Plan: The plan you want to use for your cluster, for information about Cloud’s paid plan options, see Cloud plans.
    • Cluster Name: A unique name for your cluster.
    • Extension Settings:

      • Extension: We support two Vector Search extensions, pgvectors and its successor vectorchord

        • vectorchord: Scalable, Fast, and Disk-friendly Vector search in Postgres
        • pgvectors: Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.
        Warning

        The vectorchord only support server resource with NVMe SSD.

      • Image:

        • VectorChord, we provide a tag in a semantic version format, for example 16-v0.1.0:
          • 16 indicates the version number of Postgres major
          • v0.1.0 indicates VectorChord version number
        • pgvecto.rs, we provide a tag in a semantic version format, for example 16-v0.4.0-vectors-exts:
          • 16 indicates the version number of Postgres major
          • v0.4.0 indicates pgvecto.rs version number
          • vectors indicates the installation location of pgvecto.rs scheme, optional vectors,extensions, public. Please choose public if you want to use VectorChord Cloud on AWS RDS. If you’re going to use fdw on Supabase, please choose extensions.
          • exts indicates the cluster can install extensions from https://pgt.dev/.
    • Cloud Provider Settings:
      • Cloud Provider: The cloud provider where your cluster will be deployed, currently only AWS is supported.
      • Region: The region where your cluster will be deployed.
    • Resource Settings:
      • Type: We provide two types of resources, Performance and Capacity.
      • Server Resource:
        • Performance:
          • 1C/2G: Free tier, 1 core CPU, 2GB memory and 3Gi disk space.
          • 2C/8G: Enterprise tier, 2 core CPU, 8GB memory and 15Gi disk space.
          • 2C/16G: Enterprise tier, 2 core CPU, 16GB memory and 15Gi disk space.
          • 4C/32G: Enterprise tier, 4 core CPU, 32GB memory and 90Gi disk space.
        • Capacity:
          • 4C/32G NVMe SSD: Enterprise tier, 4 core CPU, 32GB memory and 850Gi NVMe SSD disk space.
    • Postgres Settings:
      • Database Name: The name of the database that you will store vector data in.
      • Enable Connection Pooler: (default: false): Enable the connection pooler for your cluster.
      • Enable Restore Config: (default: false): Enable the restore config for your backup created in other clusters.
      • Optional:
        • Disk Size: Postgres PGData disk size, will change the default disk size.
        • Instances: The number of instances in the cluster. The default is 1.
        Warning

        Currently, vector index not support streaming replication.

Delete a cluster

If you want to delete a cluster, navigate to the VectorChord Cloud Console and click on the “Delete” button.

Warning

Once you delete this cluster, all data including WAL in Postgres and automatically or on-demand created backups in the cluster will be deleted as well.

Cluster Monitoring

You can view the cpu and memory usage of your cluster in the cluster info page, and you can also view the index info for example the number of vectors in the index, the dimension of the vectors etc. For detailed information about monitoring, see Monitoring.

4 - Authorization

Topics related to role-based access control (RBAC) on your VectorChord Cloud clusters.

Topics related to role-based access control (RBAC) on your VectorChord Cloud clusters.

User Roles

We provide two types of roles for the project: Owner and Member.

  • Owner: The owner has full access to the project. The owner can invite members, create or delete clusters, and view the billing information.
  • Member: The member has read-only access to the project. The member can view the cluster information, but cannot create or delete clusters.

API Key Roles

API Key provide more fine-grained control over access to resources by assigning roles with specific permissions to users within the projects. We provide two types of roles for the API Key: Owner and Member.

  • Owner: The owner has full access to the project cluster.
  • Member: The member has read-only access to the project cluster. However, you can specify clusters to which the member has full access.

Resource Hierarchy

For more information on the object hierarchy, see Overview of the cloud object hierarchy.

5 - Backup & Restore

Database backups are an integral part of any disaster recovery plan. Disasters come in many shapes and sizes. It could be as simple as accidentally deleting a t

Database backups are an integral part of any disaster recovery plan. Disasters come in many shapes and sizes. It could be as simple as accidentally deleting a table column, the database crashing, or even a natural calamity wiping out the underlying hardware a database is running on. The risks and impact brought by these scenarios can never be fully eliminated, but only minimized or even mitigated. Having database backups is a form of insurance policy. They are essentially snapshots of the database at various points in time. When disaster strikes, database backups allow the project to be brought back to any of these points in time, therefore averting the crisis.

Warning

Once you delete this cluster or your project, all data including WAL in Postgres and automatically or on-demand created backups in the cluster will be deleted as well.

Backup

Concepts

  • WAL archive: a location containing the WAL files (transactional logs) that are continuously written by Postgres and archived for data durability
  • Physical base backups: a copy of all the files that PostgreSQL uses to store the data in the database (primarily the PGDATA and any tablespace)

Automatic Backups

Once the cluster is created, automatic backups are enabled by default. Automatic physical base backups are taken daily and retained for 7 days. The backups are stored in AWS S3, and the retention policy for AWS S3 is set to 90 days.

At the same time, the WAL (Write-Ahead Logging) will be archived every 5 minutes. If these files cross a certain file size threshold, they are backed up immediately. As such, during periods of high amount of transactions, WAL file backups become more frequent. Conversely, when there is no activity in the database, WAL file backups are not made. Overall, this would mean that at the worst case scenario or disaster, the PITR(Point-In-Time Recovery) achieves a Recovery Point Objective (RPO) of five minutes.

On-demand Backups

If you want to create a backup at a specific point in time, you can create an on-demand backup. On-demand backups are physical base backups of the database cluster and retained for 7 days. They are stored in AWS S3 and can be retained for up to 90 days.

You can follow the steps below to create an on-demand backup:

  • Clicking the Actions button and selecting Backup.
  • Set the unique name for the backup.

You can get the backup info in the Backups tab. The information includes:

  • Backup name
  • The cluster that the backup belongs to
  • Backup start time
  • Backup end time
  • The begin LSN(Log Sequence Number) of the backup
  • The end LSN of the backup
  • The begin WAL segment of the backup
  • The end WAL segment of the backup
  • The backup phase

Restore

Restore from a cluster

Point-in-Time Recovery (PITR) allows a cluster to be backed up at much shorter intervals. This provides users an option to restore to any chosen point of up to seconds in granularity. Even with daily backups, a day’s worth of data could still be lost. With PITR, backups could be performed up to the point of disaster.

If you want to do a PITR for a cluster, you can follow the steps below:

  • Enter cluster details page.
  • Click the Actions button and select Restore from Cluster.
  • Select the server resource that you want to restore to.
  • Select the target time that you want to restore to.
    Warning

    The target time should be within the first recoverability point and last archived wal time.

Restore from an on-demand backup

If you want to restore from an on-demand backup, you can follow the steps below:

  • Enter the backups details page of a cluster.
  • Click the Actions button and select Restore
  • Select the server resource that you want to restore to.
alt text
Warning

The cluster you are restoring to should have the same image,database name and region as the backup, but recovery from the cluster can cross regions.

6 - Upgrade

You may need to upgrade your cluster and do not want to lose any data. You can upgrade the server resource like CPU, memory. Or you only want to upgrade the sto

You may need to upgrade your cluster and do not want to lose any data. You can upgrade the server resource like CPU, memory. Or you only want to upgrade the storage. The following sections will guide you through the process of upgrading your cluster.

Increasing server resource

If you want to increase the server resource like CPU and memory, the status of the cluster need to be Ready. And click the Upgrade button in the cluster detail page.

In the upgrade page, you can select the new plan and server resource. The price will be calculated based on the new server resource. Click the Submit button to upgrade the cluster.

After submitting the upgrade request, the cluster will be in the Upgrading status. The cluster will be unavailable during the upgrade process. The upgrade process may take a few minutes to complete. After the upgrade process is completed, the cluster will be in the Ready status. And the cpu, memory will be updated to the new server resource, and the storage will remain the same.

Increasing only storage

If you only want to increase the storage, the status of the cluster need to be Ready. And click the Upgrade button in the cluster detail page.

In the upgrade page, you can insert the new disk size in advanced options. The price will be calculated based on the new storage size. Click the Submit button to upgrade the cluster.

After a few minutes, the status of cluster will change from Upgrading to Ready. And the storage will be updated to the new storage size, and the cpu, memory will remain the same.

If you want to upgrade the server resource and storage at the same time, you can directly select the server resource and disk size in the upgrade page. The price will be calculated based on the new server resource and storage size. Currently, we not support downgrade the server resource and storage size. If you want to downgrade the server resource and storage size, you can contact us to help you.

7 - API Keys

On VectorChord Cloud, every project comes with a set of API keys. These keys are essential authentication tokens required for initiating terraform operations.

On VectorChord Cloud, every project comes with a set of API keys. These keys are essential authentication tokens required for initiating terraform operations.

Role-based Access Control (RBAC)

Role-based Access Control (RBAC) is a crucial security mechanism in VectorChord Cloud, applicable to managing API calls. This system allows for fine-grained control over access to resources by assigning roles with specific permissions to users within the projects.

For more information on access levels of each role, see Authorization.

Create an API Key

To create an API key, follow these steps:

  • Navigate to the API Key section in the left-hand menu.
  • Click the Create API Key button.
  • Enter a name for the API key.
  • Select the role for the API key.
    • Owner: The owner has full access to the project cluster.
    • Member: The member has read-only access to the project cluster. However, you can specify clusters to which the member has full access.
  • Click the Create button.

Delete an API Key

To delete an API key, follow these steps:

  • Navigate to the API Key section in the left-hand menu.
  • Click the Delete button next to the API key you want to delete.
Warning

Notice that once you delete an API key, you cannot recover it.

List of API Keys

To view the list of API keys, follow these steps:

  • Navigate to the API Key section in the left-hand menu.
  • You can view the list of API keys with their respective roles.
Warning

If you are not the project owner, you will not have access to the list of API keys.

8 - Terraform Integration Overview with VectorChord Cloud

This guide walks you through the process of installing and configuring the PGVecto.rs Cloud provider.

This guide walks you through the process of installing and configuring the PGVecto.rs Cloud provider.

1. Prerequisites

Before you begin, ensure you have the following:

  1. Terraform Installed: Download and install Terraform from here by following the provided instructions.

  2. VectorChord Cloud Account: Access to VectorChord Cloud and your API Key are essential. Go to the VectorChord Cloud Console and create an API key. The API key must have the ProjectOwner permissions. For more information, see API Key.

2. Download VectorChord Cloud Terraform Provider

Start by configuring the VectorChord Cloud provider within your Terraform configuration file (main.tf). Follow these steps:

terraform {
  required_providers {
    pgvecto-rs-cloud = {
      source = "tensorchord/pgvecto-rs-cloud"
    }
  }
}

3. Initialize Terraform Configuration

Initialize the Terraform configuration by running:

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of tensorchord/pgvecto-rs-cloud from the dependency lock file
- Using previously-installed tensorchord/pgvecto-rs-cloud v0.0.2

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Terraform will download the pgvecto-rs-cloud provider and install it in a hidden subdirectory of your current working directory, named .terraform.

4. Authenticate VectorChord Cloud Terraform Provider

Your VectorChord Cloud API Key is required to use the Terraform Provider. There are two ways to configure this.

Option 1: Specify API Key in Provider Block

Append the following code to your main.tf file:

provider "pgvecto-rs-cloud" {
  api_key = "<your-api-key>"
}

Replace <your-api-key> with your VectorChord Cloud API Key.

Option 2: Use Environment Variable

Set the API key as an environment variable:

export PGVECTORS_CLOUD_API_KEY="<your-api-key>"

Then the provider declaration in your main.tf file is simply:

provider "pgvecto-rs-cloud" {
}

By following these steps, you should have the VectorChord Cloud Terraform provider configured and ready to move on to the next steps.

4. Manage VectorChord Cloud Cluster

Create a Enterprise Plan

The following example demonstrates how to create a PostgreSQL cluster with the Enterprise plan. For more information about the options, refer to the Resource Schema.

resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
  account_id        = "8364ded2-5580-45c4-a394-edfa582e35a0"
  cluster_name      = "enterprise-plan-cluster"
  plan              = "Enterprise"
  image             = "15-v0.3.0-public"
  server_resource   = "aws-m7i-large-2c-8g"
  region            = "eu-west-1"
  cluster_provider  = "aws"
  database_name     = "test"
  pg_data_disk_size = "10"
}

output "psql_endpoint_enterprise" {
  description = "Endpoint for the VectorChord Cloud Enterprise PostgreSQL database"
  value       = pgvecto-rs-cloud_cluster.enterprise_plan_cluster.connect_endpoint
}
$ terraform validate && terraform plan  && terraform output  
Success! The configuration is valid.


Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be created
  + resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      + account_id                 = "8364ded2-5580-45c4-a394-edfa582e35a0"
      + cluster_name               = "enterprise-plan-cluster"
      + cluster_provider           = "aws"
      + connect_endpoint           = (known after apply)
      + database_name              = "test"
      + first_recoverability_point = (known after apply)
      + id                         = (known after apply)
      + image                      = "15-v0.3.0-public"
      + last_archived_wal_time     = (known after apply)
      + last_updated               = (known after apply)
      + pg_data_disk_size          = "10"
      + plan                       = "Enterprise"
      + region                     = "eu-west-1"
      + server_resource            = "aws-m7i-large-2c-8g"
      + status                     = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + psql_endpoint_enterprise = (known after apply)

─────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly
these actions if you run "terraform apply" now.
╷
│ Warning: No outputs found
│ 
│ The state file either has no outputs defined, or all the defined outputs are empty. Please define an
│ output in your configuration with the `output` keyword and run `terraform refresh` for it to become
│ available. If you are using interpolation, please verify the interpolated value is not empty. You can
│ use the `terraform console` command to assist.
$ terraform apply   

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be created
  + resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      + account_id                 = "8364ded2-5580-45c4-a394-edfa582e35a0"
      + cluster_name               = "enterprise-plan-cluster"
      + cluster_provider           = "aws"
      + connect_endpoint           = (known after apply)
      + database_name              = "test"
      + first_recoverability_point = (known after apply)
      + id                         = (known after apply)
      + image                      = "15-v0.3.0-public"
      + last_archived_wal_time     = (known after apply)
      + last_updated               = (known after apply)
      + pg_data_disk_size          = "10"
      + plan                       = "Enterprise"
      + region                     = "eu-west-1"
      + server_resource            = "aws-m7i-large-2c-8g"
      + status                     = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + psql_endpoint_enterprise = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Creating...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [10s elapsed]
...[snip]...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [2m0s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Creation complete after 2m9s [id=9986cfff-9758-4cbc-9433-084fe6e7936f]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

psql_endpoint_enterprise = "postgres://test_user:plvpx1YvXmBu@enterprise-plan-cluster-hp2goemtvtd5q0ew.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require"

Connect to the cluster 

$ psql "postgres://test_user:plvpx1YvXmBu@enterprise-plan-cluster-hp2goemtvtd5q0ew.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require"
psql (15.3, server 15.8 (Debian 15.8-1.pgdg120+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

test=> \dx
                                                 List of installed extensions
  Name   | Version |   Schema   |                                         Description
---------+---------+------------+----------------------------------------------------------------------------------------------
 pgaudit | 1.7     | public     | provides auditing functionality
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
 vectors | 0.3.0   | public     | vectors: Vector database plugin for Postgres, written in Rust, specifically designed for LLM
(3 rows)

test=> CREATE TABLE test (id integer PRIMARY KEY, embedding vector(3) NOT NULL);
CREATE TABLE
test=> INSERT INTO test SELECT i, ARRAY[random(), random(), random()]::real[] FROM generate_series(1, 100) i;
INSERT 0 100
test=> CREATE INDEX ON test USING vectors (embedding vector_l2_ops) WITH (options = "[indexing.hnsw]");
CREATE INDEX
test=> SELECT * FROM test ORDER BY embedding <-> '[0.40671515, 0.24202824, 0.37059402]' LIMIT 10;
 id |               embedding
----+---------------------------------------
 10 | [0.3603687, 0.2597973, 0.48529512]
 68 | [0.341106, 0.36290976, 0.4206507]
 46 | [0.29137495, 0.10839028, 0.2966823]
 56 | [0.5840259, 0.3336522, 0.33045283]
 54 | [0.2123173, 0.21955383, 0.2764878]
 97 | [0.37704793, 0.45084634, 0.3079026]
 64 | [0.33564404, 0.46575224, 0.33372045]
  8 | [0.5397384, 0.41881782, 0.26093292]
 66 | [0.42560518, 0.0136409905, 0.2709207]
 26 | [0.37973955, 0.24594037, 0.61934566]
(10 rows)

Upgrade the cluster

Up to now, we only support upgrading the server resource, plan and disk size. There are some restrictions on the upgrade operation, such as the cpu and memory of the server resource must be greater or equal to the original one, the disk size must be greater or equal to the original one. And the plan upgrade must from the Starter to Enterprise.

+ server_resource   = "aws-r7i-large-2c-16g"
- server_resource   = "aws-m7i-large-2c-8g"
+ pg_data_disk_size = "20"
- pg_data_disk_size = "10" 

$ terraform apply   
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Refreshing state... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b]

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be updated in-place
  ~ resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      ~ connect_endpoint           = "postgres://test_user:q7SdfHYVqkGB@enterprise-plan-cluster-tiqmlrllv36fmh4h.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require" -> (known after apply)
      ~ first_recoverability_point = "2024-09-20T03:53:04Z" -> (known after apply)
        id                         = "bbfb4ed2-b434-4e20-994f-6627e8f3785b"
      ~ last_archived_wal_time     = "2024-09-20T03:53:07Z" -> (known after apply)
      ~ last_updated               = "2024-09-20T03:52:38Z" -> (known after apply)
      ~ pg_data_disk_size          = "10" -> "20"
      ~ server_resource            = "aws-m7i-large-2c-8g" -> "aws-r7i-large-2c-16g"
      ~ status                     = "Ready" -> (known after apply)
        # (7 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Changes to Outputs:
  ~ psql_endpoint_enterprise = "postgres://test_user:q7SdfHYVqkGB@enterprise-plan-cluster-tiqmlrllv36fmh4h.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require" -> (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b, 10s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b, 20s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b, 30s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b, 40s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still modifying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b, 50s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Modifications complete after 53s [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Outputs:

psql_endpoint_enterprise = "postgres://test_user:q7SdfHYVqkGB@enterprise-plan-cluster-tiqmlrllv36fmh4h.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require"

Destroy the cluster 

$ terraform destroy
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Refreshing state... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b]

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be destroyed
  - resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      - account_id                 = "8364ded2-5580-45c4-a394-edfa582e35a0" -> null
      - cluster_name               = "enterprise-plan-cluster" -> null
      - cluster_provider           = "aws" -> null
      - connect_endpoint           = "postgres://test_user:q7SdfHYVqkGB@enterprise-plan-cluster-tiqmlrllv36fmh4h.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require" -> null
      - database_name              = "test" -> null
      - first_recoverability_point = "2024-09-20T03:53:04Z" -> null
      - id                         = "bbfb4ed2-b434-4e20-994f-6627e8f3785b" -> null
      - image                      = "15-v0.3.0-public" -> null
      - last_archived_wal_time     = "2024-09-20T03:53:07Z" -> null
      - last_updated               = "2024-09-20T03:55:04Z" -> null
      - pg_data_disk_size          = "20" -> null
      - plan                       = "Enterprise" -> null
      - region                     = "eu-west-1" -> null
      - server_resource            = "aws-r7i-large-2c-16g" -> null
      - status                     = "Ready" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Changes to Outputs:
  - psql_endpoint_enterprise = "postgres://test_user:q7SdfHYVqkGB@enterprise-plan-cluster-tiqmlrllv36fmh4h.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require" -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Destroying... [id=bbfb4ed2-b434-4e20-994f-6627e8f3785b]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Destruction complete after 1s

Destroy complete! Resources: 1 destroyed.

5. Recovery

In this section, we will show you how to restore your PostgreSQL database from existing backup or point-in-time recovery (PITR).

Recovery from the backup

Warning

The cluster you are restoring to should have the same image,database name and region as the backup.

  1. Get the backup id from dashboard

  2. Create a new cluster with the backup id

terraform {
  required_providers {
    pgvecto-rs-cloud = {
      source = "tensorchord/pgvecto-rs-cloud"
    }
  }
}

provider "pgvecto-rs-cloud" {
  api_key = "pgrs-57cc0d484e7e1d00444201ee846455b5"
  api_url = "https://cloud-dev.vectorchord.ai/api/v1"
}

resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
  account_id        = "8364ded2-5580-45c4-a394-edfa582e35a0"
  cluster_name      = "enterprise-plan-cluster"
  plan              = "Enterprise"
  image             = "15-v0.3.0-public"
  server_resource   = "aws-r7i-large-2c-16g"
  region            = "eu-west-1"
  cluster_provider  = "aws"
  database_name     = "test"
  pg_data_disk_size = "20"
  enable_restore    = "true"
  backup_id         = "4f53b451-8e19-443c-bcd7-43420d105696"
}

output "psql_endpoint_enterprise" {
  description = "Endpoint for the VectorChord Cloud Enterprise PostgreSQL database"
  value       = pgvecto-rs-cloud_cluster.enterprise_plan_cluster.connect_endpoint
}
$ terraform apply   

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be created
  + resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      + account_id                 = "8364ded2-5580-45c4-a394-edfa582e35a0"
      + backup_id                  = "4f53b451-8e19-443c-bcd7-43420d105696"
      + cluster_name               = "enterprise-plan-cluster"
      + cluster_provider           = "aws"
      + connect_endpoint           = (known after apply)
      + database_name              = "test"
      + enable_restore             = true
      + first_recoverability_point = (known after apply)
      + id                         = (known after apply)
      + image                      = "15-v0.3.0-public"
      + last_archived_wal_time     = (known after apply)
      + last_updated               = (known after apply)
      + pg_data_disk_size          = "20"
      + plan                       = "Enterprise"
      + region                     = "us-east-1"
      + server_resource            = "aws-r7i-large-2c-16g"
      + status                     = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + psql_endpoint_enterprise = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Creating...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [10s elapsed]
...[snip]...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [2m10s elapsed]
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Creation complete after 2m19s [id=8ba9e5b4-f06c-4ef0-8e10-b6663237a532]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

psql_endpoint_enterprise = "postgres://test_user:j4Xu6TIstD1F@enterprise-plan-cluster-kxy2lovbc4crarmn.modelz.tech:5432/test?sslmode=require"
# you can check the table and index
$ psql "postgres://test_user:j4Xu6TIstD1F@enterprise-plan-cluster-kxy2lovbc4crarmn.modelz.tech:5432/test?sslmode=require"
psql (15.3, server 15.8 (Debian 15.8-1.pgdg120+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

test=> \dt
         List of relations
 Schema | Name | Type  |   Owner
--------+------+-------+-----------
 public | test | table | test_user
(1 row)

test=> SELECT * FROM test ORDER BY embedding <-> '[0.40671515, 0.24202824, 0.37059402]' LIMIT 10;
 id |              embedding
----+--------------------------------------
 34 | [0.4047166, 0.2152912, 0.30516902]
 56 | [0.39068437, 0.3153389, 0.40011033]
 42 | [0.43917453, 0.22915891, 0.44726416]
 45 | [0.47589824, 0.33998397, 0.39237767]
 43 | [0.46702152, 0.09506769, 0.4343289]
 95 | [0.4987348, 0.08346876, 0.28716254]
 60 | [0.5421089, 0.3585022, 0.25259662]
 80 | [0.61970145, 0.2186208, 0.36950335]
 53 | [0.3399207, 0.3417044, 0.55435336]
 54 | [0.3198571, 0.445044, 0.36041933]
(10 rows)

PITR

Warning

The cluster you are restoring to should have the same image,database name as the cluster.

  1. Get cluster id from dashboard

  2. Create a new cluster with the cluster id and target time

terraform {
  required_providers {
    pgvecto-rs-cloud = {
      source = "tensorchord/pgvecto-rs-cloud"
    }
  }
}

provider "pgvecto-rs-cloud" {
  api_key = "pgrs-57cc0d484e7e1d00444201ee846455b5"
  api_url = "https://cloud-dev.vectorchord.ai/api/v1"
}

resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
  account_id        = "8364ded2-5580-45c4-a394-edfa582e35a0"
  cluster_name      = "enterprise-plan-cluster"
  plan              = "Enterprise"
  image             = "15-v0.3.0-public"
  server_resource   = "aws-r7i-large-2c-16g"
  region            = "eu-west-1"
  cluster_provider  = "aws"
  database_name     = "test"
  pg_data_disk_size = "20"
  enable_restore    = "true"
  target_cluster_id = "43f17a52-0d07-43d5-91b6-9fbe4641578c"
  target_time       = "2024-09-20T12:00:00+08:00"
}

output "psql_endpoint_enterprise" {
  description = "Endpoint for the VectorChord Cloud Enterprise PostgreSQL database"
  value       = pgvecto-rs-cloud_cluster.enterprise_plan_cluster.connect_endpoint
}
$ terraform apply   

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # pgvecto-rs-cloud_cluster.enterprise_plan_cluster will be created
  + resource "pgvecto-rs-cloud_cluster" "enterprise_plan_cluster" {
      + account_id                 = "8364ded2-5580-45c4-a394-edfa582e35a0"
      + cluster_name               = "enterprise-plan-cluster"
      + cluster_provider           = "aws"
      + connect_endpoint           = (known after apply)
      + database_name              = "test"
      + enable_restore             = true
      + first_recoverability_point = (known after apply)
      + id                         = (known after apply)
      + image                      = "15-v0.3.0-public"
      + last_archived_wal_time     = (known after apply)
      + last_updated               = (known after apply)
      + pg_data_disk_size          = "20"
      + plan                       = "Enterprise"
      + region                     = "eu-west-1"
      + server_resource            = "aws-r7i-large-2c-16g"
      + status                     = (known after apply)
      + target_cluster_id          = "43f17a52-0d07-43d5-91b6-9fbe4641578c"
      + target_time                = "2024-09-20T12:00:00+08:00"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + psql_endpoint_enterprise = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Creating...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [10s elapsed]
...[snip]...
pgvecto-rs-cloud_cluster.enterprise_plan_cluster: Still creating... [5m20s elapsed]

# you can check the table and index
$ psql 'postgres://test_user:RKC4wr7MRcwm@enterprise-plan-cluster-4paxqvbzwxudmvja.eu-west-1-dev.aws.vectorchord.com:5432/test?sslmode=require'
psql (15.3, server 15.8 (Debian 15.8-1.pgdg120+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

test=> \dt
         List of relations
 Schema | Name | Type  |   Owner   
--------+------+-------+-----------
 public | test | table | test_user
(1 row)

test=> SELECT * FROM test ORDER BY embedding <-> '[0.40671515, 0.24202824, 0.37059402]' LIMIT 10;
 id |              embedding               
----+--------------------------------------
 34 | [0.4047166, 0.2152912, 0.30516902]
 56 | [0.39068437, 0.3153389, 0.40011033]
 42 | [0.43917453, 0.22915891, 0.44726416]
 45 | [0.47589824, 0.33998397, 0.39237767]
 43 | [0.46702152, 0.09506769, 0.4343289]
 95 | [0.4987348, 0.08346876, 0.28716254]
 60 | [0.5421089, 0.3585022, 0.25259662]
 80 | [0.61970145, 0.2186208, 0.36950335]
 53 | [0.3399207, 0.3417044, 0.55435336]
 54 | [0.3198571, 0.445044, 0.36041933]
(10 rows)

9 - Regions

With VectorChord Cloud, you can deploy clusters to different Regions. When creating an cluster, pick a Region using the drop-down selector.

Deploying to Regions

With VectorChord Cloud, you can deploy clusters to different Regions. When creating an cluster, pick a Region using the drop-down selector.

Tip

Don’t see the Region or Provider that you want? Please contact us at Discord, we are happy to help.

10 - Workflow

Workflow is a feature that allow you can use some pre-defined functions to complete some tasks. For example, you can use ExternalBuildIndex workflow to build in

Overview

Workflow is a feature that allow you can use some pre-defined functions to complete some tasks. For example, you can use ExternalBuildIndex workflow to build index externally for your Postgres cluster. There are some advantages of using workflow:

  • Out Of Box: You can use the pre-defined functions to complete some tasks without writing any code.
  • Accelerate Tasks: You can use a server with a higher resource configuration than the PostgreSQL cluster to accelerate the task.
  • Cluster Stability: The workflow is executed in a separate server, which will not affect the stability of the PostgreSQL cluster.
  • Low Cost: You do not need to pay continuously like a database for high configuration resources for short-term tasks.

Supported Workflows

Supported workflows are as follows:

ExternalBuildIndex

The ExternalBuildIndex workflow is used to build vchord index externally for your Postgres cluster, details can be found here.

In dashboard, you can click the Create Workflow button to create a new workflow.

You need to specify the following information:

  • Workflow Name: A unique name for your workflow.
  • Option: The option for ExternalBuildIndex workflow
    • Table: The table where the vector is located
    • Column: Vector column name
    • Dimension: The dimension of the vector
    • Lists: KMeans n list
    • DistanceMetric: The distance metric for the vector
      • DistanceMetricCosine: Cosine distance
      • DistanceMetricL2: Euclidean distance
      • DistanceMetricDot: Dot product distance

After submitting the workflow, the workflow will be in the Running status. The workflow will be unavailable during the running process. The running process may take a few minutes to complete. After the running process is completed, the workflow will be in the Succeeded status. You can list all the workflows in the dashboard.

You can check the detail of every step in the workflow detail page.

When the workflow is in the Succeeded status, you can validate the index is ready to use.

$ psql 'postgres://vectorchord_user:C8bTXXFi2vxA@vectorchord-oews3hvirnsq3vi7.us-east-1-dev.aws.vectorchord.com:5432/vectorchord?sslmode=require'
psql (15.3, server 16.5)
WARNING: psql major version 15, server major version 16.
         Some psql features might not work.
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.
vectorchord=# EXPLAIN ANALYZE SELECT * FROM sift ORDER BY embedding <-> '[12,47,14,25,2,3,4,7,14,122,90,7,0,0,6,14,0,24,122,22,2,4,6,0,0,10,93,10,6,6,0,0,122,31,9,23,19,9,8,56,122,100,29,19,3,0,0,25,3,9,43,59,76,32,0,0,8,6,10,7,24,58,1,1,81,23,32,68,14,19,10,23,122,13,2,1,4,9,86,122,3,0,0,8,122,95,68,30,9,2,0,2,26,50,44,13,0,0,0,3,12,82,18,7,6,0,0,0,2,20,112,122,6,5,1,0,3,69,122,43,15,1,0,0,0,27,29,21]' LIMIT 5;
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.00..1.85 rows=5 width=532) (actual time=0.644..0.683 rows=5 loops=1)
   ->  Index Scan using sift_embedding_idx on sift  (cost=0.00..73962.25 rows=200012 width=532) (actual time=0.643..0.681 rows=5 loops=1)
         Order By: (embedding <-> '[12,47,14,25,2,3,4,7,14,122,90,7,0,0,6,14,0,24,122,22,2,4,6,0,0,10,93,10,6,6,0,0,122,31,9,23,19,9,8,56,122,100,29,19,3,0,0,25,3,9,43,59,76,32,0,0,8,6,10,7,24,58,1,1,81,23,32,68,14,19,10,23,122,13,2,1,4,9,86,122,3,0,0,8,122,95,68,30,9,2,0,2,26,50,44,13,0,0,0,3,12,82,18,7,6,0,0,0,2,20,112,122,6,5,1,0,3,69,122,43,15,1,0,0,0,27,29,21]'::vector)
 Planning Time: 0.069 ms
 Execution Time: 0.710 ms
(5 rows)