Introducing RAGStack: The New Stack for Production Generative AI Applications

There’s never been a better time to be a developer, because there are so many new open source packages and dev tools to implement your generative AI application. Yet ensuring that all these tools work together seamlessly can get in the way of bringing an app to production on time.

Because of this challenge, we are excited to announce the launch of our newest product, RAGStack, a solution designed to simplify the implementation of production-grade retrieval augmented generation (RAG) in AI applications. This out-of-the-box RAG solution stack is designed to help enterprises streamline the process of RAG implementation with an efficient set of tools and techniques.

What is RAGStack?

Implementing RAG can be complex and overwhelming due to the multitude of choices in orchestration frameworks, vector databases, and large language models (LLMs). On top of new software, RAG itself is changing with a multitude of techniques to implement it. There are three typical paths that developers take toward building a stack:

1. Glue together their own stack based on popular developer opinions (HackerNews or Twitter, for example) Because Gen AI is so new and the ecosystem is exploding, taking open source components and gluing them together can lead to incompatibility, and often requires lots of rework in terms of component selection. The result? Developers end up maintaining their own versions of LangChain and other open source software to meet their production needs.
2. Build a DIY stack Instead of using off-the-shelf components (e.g. LangChain), it’s possible to build everything yourself to fit your needs. Unfortunately, only the most well-funded startups and Big Tech can afford to take this path because state-of-art (SOTA) techniques are changing weekly. 
3. Glue together a stack based on production customer patterns This is the safest route because the components and techniques being selected are “production” proven. However, asking this of a developer is nearly impossible because there are so few production gen AI applications—and even fewer that reveal their underlying stack.

The first two paths unfortunately lead to perpetual generative AI prototypes. This is where RAGStack comes in. We’ve assembled a software stack designed for the most proven RAG techniques (e.g. Chain-of-Thought) based on the most widely adopted open source software (as used in DataStax’s production customer’s stack). RAGStack integrates deeply with DataStax Astra DB - the best vector database for production AI apps at scale. It significantly simplifies the process of implementing RAG, reducing the complexity and overwhelming choices that developers face.

To ensure compatibility with LangChain, RAGStack will not fork LangChain; each version of RAGStack will be pinned to a particular release of LangChain. DataStax will strive to push any fixes or modifications for RAGStack’s LangChain back into LangChain’s master within one quarter. To learn more about LangChain and AstraDB in production, see our joint webinar.

“Every company building with generative AI right now is looking for answers about the most effective way to implement RAG within their applications,” said Harrison Chase, CEO, LangChain. “DataStax has recognized a pain point in the market and is working to remedy that problem with the release of RAGStack. Using top-choice technologies, like LangChain and Astra DB among others, Datastax is providing developers with a tested, reliable solution made to simplify working with LLMs.” 

Key RAGStack features

• Curated software components RAGStack includes a selection of the best open-source software for implementing RAG from DataStax and LangChain, reducing the overwhelming choices that developers often face.
• Simple Installation Instead of installing dozens of open source packages into your application, a single install command gets you all the packages you need to build your production-worthy RAG application.
• LangSmith and LangServe compatibility You can use RAGStack with the hosted services from LangChain to add tracing (LangSmith) and hosting (LangServe) to your production GenAI application
• Data abstractions RAGStack provides several abstractions to improve developer productivity and system performance, including orchestration and prompt templates, unstructured data store abstraction, natural language to structured query abstraction, agent memory abstraction, and LLM caching abstraction.
• Advanced RAG techniques RAGStack implements advanced orchestration techniques such as Chain of Thought and Multi-Query RAG on top of Astra DB’s Vector Search  
• Performance, scalability, and cost savings RAGStack is designed to improve response times, scale easily with the increase in data and user base, and lower the cost of LLMs by caching a large percentage of calls and leveraging the inherent parallelism built into Astra DB.
• Compatibility In addition to Astra DB (vector store), RAGStack is designed to be compatible with a variety of other components like agent hosting, model training/inference/fine tuning, LLM monitoring, Streaming/batch data pipelines, and OLTP for structured data.
• Streaming: RAGStack packages LangStream, which combines the best of event-based architectures with the latest Gen AI technologies. With LangStream you can build streaming Gen AI applications in just 10 lines of code.

SkyPoint AI CEO Tisson Mathew said the provider of data services to the senior health industry has built its solution with Astra DB and a host of open source software, including LangChain. 

“With RAGStack, we’ll be able to reduce the pain of maintaining customized open source software,, helping to deliver a more simplified and streamlined healthcare AI solution for our customers,” he said.

Try RAGStack

We’ve built RAGStack with learnings from our customers because all of us would like to see more generative AI applications in production. To learn more about RAGStack and find out how it can simplify the process of getting an AI app to production, visit the RAGStack page.