Vercel + Astra DB: Get Data into Your GenAI Apps Fast
Carter RabasaHead of Developer Relations
When Vercel announced v3 of their AI SDK last month, it marked a huge step forward in the evolution of generative AI and provides a path for web developers to build generative UI experiences powered by React Server Components. An amazing example of this was the demonstration of a chatbot generating a playable game of Pac-Man at the prompting of the user. There seems to be no limit to the experiences that developers can create, which is why DataStax is excited to announce our Astra DB integration for Vercel.
Now Vercel developers can create Astra DB accounts and provision databases right from within the Vercel experience. Once the database is created and the Vercel project is selected, all of the necessary API keys are copied over to the project and stored securely in the project's environment variables. This makes it even easier for developers to get started building and reduces an entire category of annoying copy/paste/typo errors that are just the worst.
Let's see exactly how this works by cloning Vercel's epic AI SDK v3 demo and connecting it to some actual data so that when you ask it for the price of a stock, you get a real response.
(and I’ll walk you through this on our April 24th livestream, save a spot here!)
Setup
In order to get started, you'll need the following:
- A free Vercel account
- A free Astra DB account
- An OpenAI account
- Node.js installed on your computer
Creating a new Vercel project
After you have created your Vercel account, go to the AI SDK v3 demo and click "Deploy to Vercel.”
You'll be asked to create a Github repo and you can name that repo anything you like. Click "Create."
Next, you'll need to enter your OpenAI API key information:
You can now deploy this app to Vercel. The cloned code with your OpenAI API key gives you a fully functional app! Go to your Dashboard and click "Visit" to take the app for a spin, it should be a perfect copy of what you see on the official demo website.
Now, let's make this app even smarter by connecting it to a database.
Adding the DataStax Astra DB integration
Assuming you're still logged-in to Vercel, go to our official Vercel integration page. Click on "Add Integration".
Make sure to select the Vercel account that you used when you cloned the AI SDK demo. Select the "Specific Projects" radio button and then select the project you created. Click "Install.”
This will spawn a pop-up that you can use to either create a new Astra DB account or (if you scroll down) sign in to an account you already have.
If you are a member of multiple organizations, you'll be asked to select the one you'd like to use.
Next, it's time to select or create a new database!
When you create a database, you'll be asked to give it a name, select a public cloud provider and a region for your database.
Finally, you'll be presented with a confirmation screen. Select the Vercel project that you're linking to and click "Complete Integration."
Once you've completed the installation of the integration, you can confirm that your project has been configured properly by going into your project's settings and reviewing its environment variables. If you see an `ASTRA_DB_API_ENDPOINT` and `ASTRA_DB_APPLICATION_TOKEN`, you're all set!
Loading real stock price data from Astra DB
One thing you'll quickly notice with the app is that the stock price information is made up, or hallucinated.
Let's load some real stock price data into the database we've created.
Log in to your Astra DB account and navigate to the database you created. Click on "Data Explorer".
Create a new collection and Name it "stocks." Disable "vector," as we're not going to store any vector data for this demo.
Now, let's load some data into the collection. Download this CSV of stock price data to your computer. Click "Load Data," select the file containing the CSV data and click "Load Data."
This should just take a few seconds, and then you'll see all the data loaded into your collection. Each row in the CSV has been turned into a document and you can review the JSON structure of your documents right inside of the Data Explorer:
Hacking the AI SDK demo to return real data
Ok, enough click, click, click—let's write some code! We are going to modify the code for the AI SDK v3 demo to retrieve and display real stock price information from our database.
First, clone the Github repository that you created when you created your Vercel project to your local computer. Go into the directory that was created and install the demo's dependencies and the TypeScript client for Astra DB:
npm i @datastax/astra-db-ts
Make a copy of the provided environment variables example file:
cp .env.example .env
Edit that file to include your OpenAI and Astra DB config:
OPENAI_API_KEY=sk-xxx ASTRA_DB_API_ENDPOINT=https://xxx.apps.astra.datastax.com ASTRA_DB_APPLICATION_TOKEN=AstraCS:xxx
Verify that the app works as expected by running `npm run dev` and opening up "http://localhost:3000." You should see the demo web app and if you ask some questions, you should get responses. If there are any issues here, double check your OpenAI API key.
In the file `app/layout.tsx`, update the export at the bottom to change the runtime from "edge" to "nodejs."
export const runtime = 'nodejs';
The main file we're going to edit is `app/action.tsx`. This is where the bulk of the logic for the chatbot is, so go ahead and open it up in your editor.
At the top of the file, import the TypeScript client for Astra DB.
import { AstraDB } from '@datastax/astra-db-ts'Initialize the client using the values that you stored in the `.env` file.
const {
ASTRA_DB_API_ENDPOINT,
ASTRA_DB_APPLICATION_TOKEN
} = process.env
const db = new AstraDB(ASTRA_DB_APPLICATION_TOKEN, ASTRA_DB_API_ENDPOINT)Down around line 124, the prompt for the chatbot is defined. This is where we tell OpenAI what to expect and how we want it to behave. Below the prompt, you'll see an array of functions being defined. This is the heart of generative UI, so let's take a minute to explain what's happening.
OpenAI Functions and Generative UI
The OpenAI Chat Completions API supports function calling. You can read about it in more detail in the OpenAI docs, but essentially you are able to:
- Define a function in your application
- Specify the purpose it serves and the parameters that it expects
- Ask the LLM to call the function under certain conditions
In this demo app example, in the system prompt the LLM is told the following:
"If the user just wants the price, call \`show_stock_price\` to show the price."
Further down you can see where we define the `show_stock_price` and the parameters it expects:
functions: [
{
name: 'show_stock_price',
description:
'Get the current stock price of a given stock or currency.
Use this to show the price to the user.',
parameters: z.object({
symbol: z
.string()
.describe(
'The name or symbol of the stock or currency.
e.g. DOGE/AAPL/USD.',
),
price: z.number().describe('The price of the stock.'),
delta: z.number().describe('The change in price of the stock'),
}),
}, ...]Take a second to stop and realize how amazing this is. You can essentially define arbitrary typesafe JSON APIs that will return data from OpenAI in the format you have requested based on the descriptions you have provided for both the function and its return values.
Querying data from Astra DB
Of course, the small catch here is that OpenAI doesn't actually have data for stock prices, which is why it hallucinates and returns fake data. Let's fix that! Look for where the code for `show_stock_price` is defined in our application:
completion.onFunctionCall(
'show_stock_price',
async ({ symbol, price, delta }) => {
reply.update(
<BotCard>
<StockSkeleton />
</BotCard>,
);
await sleep(1000);
...Remove the `await sleep(1000);` and replace it with the following code:
// fetch actual price from Astra DB if it's in the S&P 500
const collection = await db.collection("stock_prices")
const doc = await collection.findOne({ Symbol: symbol })
console.log("Queried DB, got back ", doc)
if (doc) {
price = parseFloat(doc.Currentprice)
}
else {
await sleep(1000);
}So, what's happening here? Well, when OpenAI calls the `show_stock_price` function, it passes in values for `symbol`, `price` and `delta`. Remember, the `price` and `delta` are just values that the LLM invents and passes to our function.
Now we can use the `symbol` to query our collection in Astra DB to retrieve the associated document. If we get a document back, we override the `price` that's passed-in with the price from our DB. If the stock symbol isn't in our DB, just retain the current behavior.
So, let's take this for a spin! Make sure your files are saved and your dev server is running and go to "http://localhost:3000". Ask the chatbot for the price of MSFT.
It should respond with the value of $404.52, since that's what was in our CSV and that's what we loaded into Astra DB. You did it, you built a chatbot with generative UI capabilities that can return beautiful charts with accurate stock prices!
Wrapping up
Ok, let's review what we learned:
- How to use Vercel to create Astra DB accounts and databases and have those credentials configured for your Vercel projects.
- How to create an Astra DB collection and load CSV data.
- How to use Vercel's AI SDK to retrieve data from Astra DB and incorporate it into the UI elements that it generates
You can find the full source on Github and a deployed example of this app online.
I hope you enjoyed this post! If you have any questions about the code, the experience of building on Astra DB, you can find me on Twitter.
For a live walkthrough of the demo above, save your spot on our April 24th livestream “Building Generative UIs with DataStax and Vercel”
Happy hacking!
More Technology
View All
How to Build a Crystal Image Search App with Vector Search
Knowledge Graphs for RAG without a GraphDB
