Prasad Prabhakaran, esynergy

Edwin Jose, esynergy

<p><strong><em>Editor's note: This post originally appeared on esynergy's blog.</em></strong></p>
<p>At <a href="https://esynergy.co.uk/">esynergy</a>, we have leveraged Langchain and <a title="DataStax Astra DB" href="https://www.datastax.com/products/datastax-astra">DataStax Astra DB</a> to develop Sales Copilot &ndash; an AI-powered conversational assistant for sales teams to access customer data and generate informed responses. In this post, we&rsquo;ll dive into the technical architecture and how <a title="What is Langchain" href="https://www.datastax.com/guides/what-is-langchain">Langchain</a> enabled rapid prototyping of a retrieval-augmented chatbot.</p>
<h2>Unveiling the Sales Copilot</h2>
<p>Sales Copilot ingests customer data from SharePoint, encodes it into vectors, <a title="What is a Vector Index" href="https://www.datastax.com/guides/what-is-a-vector-index">indexes</a> in Astra DB for low-latency search, retrieves relevant chunks for a query, and passes them to Claude2 to generate a response.</p>
<p><img style="display: block; margin-left: auto; margin-right: auto;" src="https://cdn.sanity.io/images/bbnkhnhl/production/ad4c4d67fce37017c28a8f3b195b18a9cca7b75a-1990x1592.gif" alt="" width="620" height="496" /></p>
<p>This <a title="What is Retrieval Augmented Generation" href="https://www.datastax.com/guides/what-is-retrieval-augmented-generation">retrieval-augmented</a> setup allows generating contextual responses by conditioning on relevant data. Langchain&rsquo;s modular components made implementing it straightforward.</p>
<p>The application seamlessly integrates with your organization&rsquo;s SharePoint instance, allowing it to ingest customer profiles, communication records, and case studies. This raw data undergoes a meticulous preprocessing stage, where Langchain plays a crucial role. Langchain&rsquo;s capabilities enable the application to efficiently split files into manageable chunks, optimizing them for subsequent processing.</p>
<p>Following this initial step, these chunks are transformed into dense vector representations using AWS Bedrock, a suite of cutting-edge text embedding models. This process allows the application to represent information semantically, facilitating a more nuanced understanding of customer data.</p>
<p>At the heart of the application lies the &ldquo;retrieve-augment&rdquo; architecture. When a user initiates a query through the Streamlit frontend, esynergy, a customized retrieval component, takes center stage. esynergy leverages advanced techniques like query augmentation and MMR (Maximum Marginal Relevance) reranking to ensure the retrieval of the most relevant data points from the <a title="What is a Vector Database" href="https://www.datastax.com/guides/what-is-a-vector-database">vector database</a>, Astra DB.</p>
<p>These retrieved data nuggets then serve as the foundation for response generation. Anthropic&rsquo;s Claude2 <a title="What is a Large Language Model" href="https://www.datastax.com/guides/what-is-a-large-language-model">large language model</a>, integrated through AWS Bedrock, steps in to craft informative responses tailored to the user&rsquo;s query and informed by the retrieved customer data</p>
<h2>Langchain in action</h2>
<h3>Data ingestion</h3>
<p>The `DocumentLoader` extracted SharePoint data into common text formats. The `RecursiveTextSplitter` chunked documents into smaller passages for processing.</p>
<h3>Vector encoding</h3>
<p>AWS Bedrock Titan model encoded chunks into dense vectors capturing semantic meaning. This allowed indexing in Astra DB for efficient similarity search.</p>
<h3>Flexible retrieval</h3>
<p>We built custom retrieval components using Langchain&rsquo;s `Retrievers` abstraction. Techniques like paraphrasing, duplicate removal, and MMR reranking retrieved diverse, relevant passages for a query.</p>
<p>Integrations with LangSmith and LangServe allowed rapidly experimenting with different strategies.</p>
<h3>Response generation</h3>
<p>The Claude2 model, via Bedrock API, generated informed responses conditioned on the retrieved passages. Prompt engineering further improved response coherence.</p>
<h3>Deployment</h3>
<p>The Langserve/ FastAPI web framework exposed the pipeline as an API. Streamlit provided a quick way to build and deploy the frontend.</p>
<h2>Outcomes</h2>
<p>Using Langchain accelerated developing a production-grade conversational assistant. Benefits included:</p>
<ul>
<li>Faster prototyping by building on modular components</li>
<li>Swapping retrieval strategies with LangSmith experiments</li>
<li>Easy integration of Claude2 for natural dialogue</li>
<li>Changing components without impacting others</li>
<li>Quickly validating capabilities before customization</li>
<li>Reduced effort on data preprocessing and ingestion</li>
</ul>
<h2>A glimpse into the benefits</h2>
<p>The Sales Copilot application empowers sales representatives with a plethora of advantages, including:</p>
<ul>
<li><strong>Rapid retrieval of case studies -</strong>&nbsp;Effortlessly access relevant case studies, projects, and customer success stories to bolster product credibility during conversations.</li>
<li><strong>Swift access to project details -</strong>&nbsp;Obtain specifics like timelines, milestones, and deliverables promptly, eliminating the need to scour through multiple systems.</li>
<li><strong>Effortless case study generation -</strong>&nbsp;Leverage the system&rsquo;s ability to auto-generate draft case studies, saving valuable time and streamlining the content creation process.</li>
<li><strong>Enhanced case study quality -</strong>&nbsp;Seek suggestions on improving existing case studies, allowing for the creation of even more compelling narratives.</li>
<li><strong>Streamlined communication drafting -</strong>&nbsp;Generate customized emails, chat messages, and social media posts tailored to individual customers, fostering stronger relationships.</li>
<li><strong>Data-driven report generation -</strong>&nbsp;Gain valuable insights by requesting customized reports or comparisons, empowering informed decision-making.</li>
<li><strong>Comprehensive analytics -</strong>&nbsp;Leverage the system&rsquo;s ability to provide insightful analytics on various aspects, including campaigns, lead conversion rates, and customer sentiment.</li>
</ul>
<p>By acting as an AI assistant, the Sales Copilot application equips sales representatives with the knowledge and tools necessary to elevate their interactions with customers. Data-driven responses not only minimize repetitive tasks but also contribute to fostering more meaningful and productive conversations.</p>
<p>We&rsquo;re excited to see what more we can build by leveraging additional Langchain tools like LangServe and Langsmith. The framework has opened up new possibilities for us to create enterprise AI solutions.</p>

LangChain and DataStax Astra DB Power Esynergy’s Retrieve-Augment Architecture for Sales Copilot

Prasad Prabhakaran, esynergyGenAI practice lead

Edwin Jose, esynergyLead associate, AI

Discover more

Share

Share

Unveiling the Sales Copilot

Langchain in action

Data ingestion

Vector encoding

Flexible retrieval

Response generation

Deployment

Outcomes

A glimpse into the benefits

More Success Stories

How PTC Is Transforming eCommerce with Generative AI, DataStax Astra DB, and Vector Search

LangChain and DataStax Astra DB Power Esynergy’s Retrieve-Augment Architecture for Sales Copilot

Vector Search in Action: How Restworld Improves Relevance and Accuracy in Job Matches

Restworld: Revolutionizing Hospitality Recruitment with Astra DB and Vector Search

One-stop Data API for Production GenAI