Leveraging AlloyDB and Vertex AI for an LLM and RAG Based Chat Application

Task 1. Initialize the database environment

Before you begin

1. Make sure you have a Google Cloud project and billing is enabled.

2. Set your PROJECT_ID environment variable:

export PROJECT_ID=<YOUR_PROJECT_ID>

3. Set gcloud project:

gcloud config set project $PROJECT_ID

4. Enable APIs:

gcloud services enable alloydb.googleapis.com \
                       compute.googleapis.com \
                       cloudresourcemanager.googleapis.com \
                       servicenetworking.googleapis.com \
                       vpcaccess.googleapis.com \
                       aiplatform.googleapis.com \
                       artifactregistry.googleapis.com

Enable private services access

In this step, we will enable Private Services Access so that AlloyDB is able to connect to your VPC. You should only need to do this once per VPC (per project).

1. Set environment variables:

export RANGE_NAME=my-allocated-range-default
export DESCRIPTION="peering range for alloydb-service"

2. Create an allocated IP address range:

gcloud compute addresses create $RANGE_NAME \
    --global \
    --purpose=VPC_PEERING \
    --prefix-length=16 \
    --description="$DESCRIPTION" \
    --network=default

3. Create a private connection:

gcloud services vpc-peerings connect \
    --service=servicenetworking.googleapis.com \
    --ranges="$RANGE_NAME" \
    --network=default

Create an AlloyDB cluster

1. Set environment variables. For security reasons, use a different password for $DB_PASS and note it for future use:

export CLUSTER=my-alloydb-cluster
export INSTANCE=my-alloydb-instance
export REGION=us-central1
export DB_USER=postgres
export DB_PASS=my-alloydb-pass

2. Create an AlloyDB cluster:

gcloud alloydb clusters create $CLUSTER \
    --password=$DB_PASS\
    --network=default \
    --region=$REGION \
    --project=$PROJECT_ID

3. Create a primary instance:

gcloud alloydb instances create $INSTANCE \
    --instance-type=PRIMARY \
    --cpu-count=8 \
    --region=$REGION \
    --cluster=$CLUSTER \
    --project=$PROJECT_ID \
    --ssl-mode=ALLOW_UNENCRYPTED_AND_ENCRYPTED \
    --database-flags=password.enforce_complexity=on

Create a VM Instance and Install a PostgreSQL client

1. Create a VM Instance

gcloud compute instances create app-vm \
  --machine-type=e2-medium \
  --zone=us-central1-c \
  --scopes=https://www.googleapis.com/auth/cloud-platform

2. Connect to the VM Instance

gcloud compute ssh app-vm --zone=us-central1-c

3. Install PostgreSQL Client

sudo apt-get update
sudo apt-get install --yes postgresql-client

Connect to the AlloyDB instance

export ADBCLUSTER=my-alloydb-cluster
export ADBINSTANCE=my-alloydb-instance
export PGUSER=postgres
export PGPASSWORD=my-alloydb-pass
export PROJECT_ID=$(gcloud config get-value project)
export REGION=us-central1
export INSTANCE_IP=$(gcloud alloydb instances describe $ADBINSTANCE --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")

echo $INSTANCE_IP
psql "host=$INSTANCE_IP user=$PGUSER sslmode=require"

exit

Task 2. Create the vector database

Create the database

1. To create a new database, in the VM session, run the following command:

export PGPASSWORD=my-alloydb-pass
psql "host=$INSTANCE_IP user=$PGUSER" -c "CREATE DATABASE assistantdemo"

2. To enable vector embeddings in this database, run the following command:

psql "host=$INSTANCE_IP user=$PGUSER dbname=assistantdemo" -c "CREATE EXTENSION vector"

Task 3. Install Python

1. To install Python and Git, in the VM, run the following commands:

sudo apt install -y python3.11-venv git
python3 -m venv .venv
source ~/.venv/bin/activate
pip install --upgrade pip

2. To confirm the python version, run the following command:

python -V

Task 4. Populate the example database

1. To clone the repo, in the VM, run the following command:

git clone https://github.com/GoogleCloudPlatform/genai-databases-retrieval-app.git

2. To see the data model, run the following command:

cd ~/genai-databases-retrieval-app
cat retrieval_service/models/models.py

3. To see an example of the airport data, run the following commands:

head -1 data/airport_dataset.csv; grep SFO data/airport_dataset.csv

4. To see an example of the flight data, run the following commands:

head -1 data/flights_dataset.csv; grep -m10 "SFO" data/flights_dataset.csv

5. To see an example of the amenities data, run the following command:

head -2 data/amenity_dataset.csv

6. To create a database configuration file, run the following commands:

export ADBCLUSTER=my-alloydb-cluster
export ADBINSTANCE=my-alloydb-instance
export PGUSER=postgres
export PGPASSWORD=my-alloydb-pass
export PROJECT_ID=$(gcloud config get-value project)
export REGION=us-central1
export INSTANCE_IP=$(gcloud alloydb instances describe $ADBINSTANCE --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
cd ~/genai-databases-retrieval-app/retrieval_service
cp example-config.yml config.yml
sed -i s/127.0.0.1/$INSTANCE_IP/g config.yml
sed -i s/my-user/$PGUSER/g config.yml
sed -i s/my-password/$PGPASSWORD/g config.yml
sed -i s/my_database/assistantdemo/g config.yml
cat config.yml

7. To populate the database with the sample dataset, run the following commands:

pip install -r requirements.txt
python run_database_init.py

Task 5. Create a service account for the retrieval service

1. In Cloud Shell, to open a new Cloud Shell tab, click Open a new tab (+).

To create a service account and grant it the necessary privileges, in the new tab, run the following commands:

export PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts create retrieval-identity
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:retrieval-identity@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/aiplatform.user"

Task 6. Deploy the retrieval service to Cloud Run

1. To deploy the retrieval service, in the VM SSH Cloud Shell tab, run the following commands:

export REGION=us-central1
cd ~/genai-databases-retrieval-app
gcloud alpha run deploy retrieval-service \
    --source=./retrieval_service/\
    --no-allow-unauthenticated \
    --service-account retrieval-identity \
    --region $REGION \
    --network=default \
    --quiet

2. To verify the service, run the following command:

curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")

Task 7. Register the OAuth consent screen

In this task, you register the OAuth consent screen that is presented to users who are logging in.

1. In the Google Cloud console, select the Navigation menu, and then select APIs & Services > OAuth consent screen.

2. Click Get Started.

3. For App name, enter Cymbal Air.

4. Click User support email, then click your email, and then click Next.

5. For Audience, select External, and then click Next.

6. For Contact information, paste the copied username.

7. Click Next.

8. Click Checkbox to agree the User Data Policy, then click Continue, and then click Create.

Task 8. Create a client ID for the application

In this task, you create a client ID for the application. The application requires a client ID to use Google's OAuth service.

1. In the Google Cloud console, select the Navigation menu, and then select APIs & Services > Credentials.

2. Click + Create Credentials, and then click OAuth client ID.

3. For Application type, select Web application.

4. For Name, enter Cymbal Air. You can generate the JavaScript origin and redirect URI using Cloud Shell.

5. In Cloud Shell, to open a new Cloud Shell tab, click Open a new tab (+).

6. To get the origin and redirect URI, in the new tab, run the following commands:

echo "origin:"; echo "https://8080-$WEB_HOST"; echo "redirect:"; echo "https://8080-$WEB_HOST/login/google"

7. For Authorized JavaScript origins, click + Add URI.

8. Copy the origin URI that was created by the echo command, and then, for URIs 1, paste in the URI.

9. For Authorized redirect URIs, click + Add URI.

10. Copy the redirect URI that was created by the echo command, and then, for URIs 1, paste in the URI.

11. Click Create.

11. To create the environment variable, in the VM SSH Cloud Shell tab, paste the following command without clicking Enter:

export CLIENT_ID=<PASTE THE CLIENT ID>

Task 9. Deploy the sample application

In this task, you run a sample chat application that uses the retrieval service.

Run the application

1. To install the Python requirements for the chat application, in the VM SSH Cloud Shell tab, run the following commands:

cd ~/genai-databases-retrieval-app/llm_demo
pip install -r requirements.txt

2. To specify the base URL of the retrieval service, run the following commands:

export BASE_URL=$(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")
echo $BASE_URL

3. To run the application, run the following command:

python run_app.py

Connect to the VM

1. In Cloud Shell, to open a new Cloud Shell tab, click Open a new tab (+).

2. To create an SSH tunnel to the VM port, in the new tab, run the following command:

gcloud compute ssh app-vm --zone=us-central1-c -- -L 8080:localhost:8081

3. To run the application in the web browser, click Web Preview, and then select Preview on port 8080.

A new tab is opened in the browser, and the application is running. The Cymbal Air application prompts "Welcome to Cymbal Air! How may I assist you?"

4. Enter the following query:

When is the next flight to Dallas?

The application responds with the next flight from SFO to Dallas/Fort Worth.

5. Enter the following query:

Which restaurants are near the departure gate?

The application understands the context, and responds with restaurants near the departure gate in SFO.