Foundation models have been trained on enormous amounts of data to make them excel at human-like, iterative content creation. But they are only as good as the data used during training - ChatGPT can hallucinate (confident-sounding but erroneous output) especially when asked domain-specific questions.

What if you could train ChatGPT in a matter of minutes to answer questions based on your own unstructured data stored in PDFs?

In this session, we will use Dataworkz + AstraDB to build a Retrieval Augmented Generation (RAG) application to ask questions from 1M PDFs available at www.pubmed.gov.

We will showcase:

Extract information from 1M PDFs
Use the embedding model of your choice – OpenAI’s text-embedding-ada-002, all-mpnet-base-v2 from the MTEB benchmark or bring your own
Pre-processing to create text chunks and vector embeddings
Combine metadata with semantic similarity for better information retrieval
Context for a LLM to remove hallucinations
RAG Analytics and how it informs RHLF

Come join us to experience the fastest path to building a Retrieval Augmented Generation (RAG) application – Dataworkz + AstraDB

Nikita Ivanov

Sachin Smotra