Technology•March 7, 2024
Airbyte and DataStax Simplify GenAI and RAG App Development
Announcing a new partnership that offers a DataStax Astra DB data destination in the Airbyte Cloud data pipeline.
pip install ragstack-ai python-dotenv
ASTRA_DB_APPLICATION_TOKEN=AstraCS:blahblah:blahblahYourTokenGoesHere ASTRA_DB_API_ENDPOINT=https://your-endpoint.apps.astra.datastax.com ASTRA_DB_KEYSPACE_NAME=default_keyspace ASTRA_DB_COLLECTION_NAME=airbyte OPENAI_API_KEY=sk-blahblahYourOpenAIKeyFromAbove
import os from dotenv import load_dotenv from astrapy.db import AstraDB from langchain_openai import OpenAIEmbeddings
load_dotenv() model = OpenAIEmbeddings()
db = AstraDB( token=os.environ["ASTRA_DB_APPLICATION_TOKEN"], api_endpoint=os.environ["ASTRA_DB_API_ENDPOINT"], namespace=os.environ["ASTRA_DB_KEYSPACE_NAME"], ) collection = db.collection(os.environ["ASTRA_DB_COLLECTION_NAME"])
if (len(sys.argv) > 1): query = sys.argv[1] else: query = "Kepler" vector = model.embed_query(query)
res = collection.vector_find_one(vector,fields=['title','img','alt']) print(res)
> python airbyte_query.py Query="Kepler" {'_id': 'f3201311-f41c-40ac-be3f-ca1679be8242', 'title': 'Kepler', 'img': 'https://imgs.xkcd.com/comics/kepler.jpg', 'alt': 'Science joke. You should probably just move along.', '$similarity': 0.96341634}
> python airbyte_query.py "orbital mechanics" Query="orbital mechanics" {'_id': '05f8c7ac-1aa4-429f-a9b3-d97f398514f4', 'title': 'Orbital Mechanics', 'img': 'https://imgs.xkcd.com/comics/orbital_mechanics.png', 'alt': "To be fair, my job at NASA was working on robots and didn't actually involve any orbital mechanics. The small positive slope over that period is because it turns out that if you hang around at NASA, you get in a lot of conversations about space.", '$similarity': 0.96553665}
JUMP TO SECTION