Introducing Email AI: How to Slash High Volumes While Improving CSAT Save your Spot →
Quiq

Tag: rag

How Does Data Impact Optimal AI Performance in CX? We Break It Down.

Posted on The Quiq Team

Many customer experience leaders are considering how generative AI might impact their businesses. Naturally, this has led to an explosion of related questions, such as whether it’s worth training a model in-house or working with a conversational AI platform, whether generative AI might hallucinate in harmful ways, and how generative AI can enhance agent performance.

One especially acute source of confusion centers on AI’s data reliance, or the role that data—including your internal data—plays in AI systems. This is understandable, as there remains a great deal of misunderstanding about how large language models are trained and how they can be used to create an accurate, helpful AI assistant.

If you count yourself among the confused, don’t worry. This article will provide a careful look at the relationship between AI and your CX data, equipping you to decide whether you have everything you need to support the use of generative AI, and how to efficiently gather more, if you need to.

Let’s dive in!

What’s the Role of CX Data in Teaching AI?

In our deep dive into large language models, we spent a lot of time covering how public large language models are trained to predict the end of some text. They’ll be shown many sentences with the last word or two omitted (“My order is ___”), and from this, they learn that the last word in is something “missing” or “late.”

The latest CX solutions have done an excellent job leveraging these capabilities, but the current generation of language models still tends to hallucinate (i.e., make up) information.

To get around this, savvy CX directors have begun utilizing a technique known as “retrieval augmented generation,” also known as “RAG.”

With RAG, models are given access to additional data sources that they can use when generating a reply. You could hook an AI assistant up to an order database, for example, which would allow it to accurately answer questions like “Does my order still qualify for a refund?”

RAG also plays an important part in managing language models’ well-known tendency to hallucinate. By drawing on the data contained within an authoritative source, these models become much less likely to fabricate information.

How Do I Know If I Have the Right Data for AI?

CX data tends to fall into two broad categories:

  1. Knowledge, like training manuals and PDFs
  2. Data from internal systems, like issue tickets, chats, call transcripts, etc.

Luckily for CX leaders, there’s usually enough of both lying around to meet an AI assistant’s need for data. Dozens of tools exist for tracking important information – customer profiles, information related to payment and shipping, and the like – and nearly all offer API endpoints that allow them to integrate with your existing technology stack.

What’s more, it’s best if this data looks and feels just like the data your human agents see, so you don’t need to curate a bespoke data repository. All of this is to say that you might already have everything you need for optimal AI performance, even if your sources are scattered or need to be updated.

Processing Data for Generative AI

Data processing work is far from trivial, and outsourcing it to a dedicated set of tools is often the wiser choice. A conversational AI platform built for generative AI should make it easy for you to program instructions for data processing.

That said, you might still need to work on cleaning and formatting the data, which can take some effort.

Understanding the steps involved in preparing data for AI is a big subject, but you’ll almost certainly need to do a mix of the following:

  • Extract: 80% of enterprise data exists in various unstructured formats, such as HTML pages, PDFs, CSV files, and images. This data has to be gathered, and you may have to “clean” it by removing unwanted content and irrelevant sections, just as you would for a human agent.
  • Transform: Your AI assistant will likely support answering various kinds of questions. If you’re using retrieval augmented generation, you may need to create a language “embedding” to answer those questions effectively, or you may need to prepare and enrich your answers so your assistant can find them more effectively.
  • Load: Finally, you will need to “feed” your AI assistant the answers stored in (say) a vector database.

Remember: The GenAI data process isn’t trivial, but it’s also easier than you think, especially if you find the right partner. Quiq’s native “dataset transformation” functionality, for example, facilitates rewriting text, scrubbing unwanted characters, augmenting a dataset (by generating a summary of it), structuring it in new ways, and much more.

What Do I Need to Create Additional Data for AI?

As we said above, your existing data may already be sufficient for optimal AI performance. This isn’t always the case, however, and it’s worth saying a few words about when you will need to create a new resource for a model.

In our experience, the most common data gaps occur when common or important questions are not addressed anywhere in your documentation. Start by creating text about them that a model can use to generate replies, and then work your way out to questions that are less frequent.

One idea our clients use successfully is to ask human agents what questions they see most frequently. Here’s an example of an awesome, simple FAQ from LOOP auto insurance:

When you’re doing this, remember: it’s fine to start small. The quality of your supplementary content is more important than the quantity, and a few sentences in a single paragraph will usually do the trick.

The most important task is to make sure you have a framework to understand what data gaps you have so that you can improve. This could include analyzing previous questions or proactively labeling existing questions you don’t have answers for.

Wrapping Up

There’s no denying the significance of relevant data in AI advancements, but as we’ve hopefully made clear, you probably have most of what you already need—and the process to prepare it for AI is a lot more straightforward than many people think.

If you’re interested in learning more about optimal AI performance and how to achieve it, check out our free e-book addressing the misconceptions surrounding generative AI. Armed with the insights it contains, you can figure out how much AI could impact your contact center, and how to proceed.

Does Quiq Train Models on Your Data? No (And Here’s Why.)

Posted on The Quiq Team

Customer experience directors tend to have a lot of questions about AI, especially as it becomes more and more important to the way modern contact centers function.

These can range from “Will generative AI’s well-known tendency to hallucinate eventually hurt my brand?” to “How are large language models trained in the first place?” along with many others.

Speaking of training, one question that’s often top of mind for prospective users of Quiq’s conversational AI platform is whether we train the LLMs we use with your data. This is a perfectly reasonable question, especially given famous examples of LLMs exposing proprietary data, such as happened at Samsung. Needless to say, if you have sensitive customer information, you absolutely don’t want it getting leaked – and if you’re not clear on what is going on with an LLM, you might not have the confidence you need to use one in your contact center.

The purpose of this piece is to assure you that no, we do not train LLMs with your data. To hammer that point home, we’ll briefly cover how models are trained, then discuss the two ways that Quiq optimizes model behavior: prompt engineering and retrieval augmented generation.

How are Large Language Models Trained?

Part of the confusion stems from the fact that the term ‘training’ means different things to different people. Let’s start by clarifying what this term means, but don’t worry–we’ll go very light on technical details!

First, generative language models work with tokens, which are units of language such as a part of a word (“kitch”), a whole word (“kitchen”), or sometimes small clusters of words (“kitchen sink”). When a model is trained, it’s learning to predict the token that’s most likely to follow a string of prior tokens.

Once a model has seen a great deal of text, for example, it learns that “Mary had a little ____” probably ends with the token “lamb” rather than the token “lightbulb.”

Crucially, this process involves changing the model’s internal weights, i.e. its internal structure. Quiq has various ways of optimizing a model to perform in settings such as contact centers (discussed in the next section), but we do not change any model’s weights.

How Does Quiq Optimize Model Behavior?

There are a few basic ways to influence a model’s output. The two used by Quiq are prompt engineering and retrieval augmented generation (RAG), neither of which does anything whatsoever to modify a model’s weights or its structure.

In the next two sections, we’ll briefly cover each so that you have a bit more context on what’s going on under the hood.

Prompt Engineering

Prompt engineering involves changing how you format the query you feed the model to elicit a slightly different response. Rather than saying, “Write me some social media copy,” for example, you might also include an example outline you want the model to follow.

Quiq uses an approach to prompt engineering called “atomic prompting,” wherein the process of generating an answer to a question is broken down into multiple subtasks. This ensures you’re instructing a Large Language Model in a smaller context with specific, relevant task information, which can help the model perform better.

This is not the same thing as training. If you were to train or fine-tune a model on company-specific data, then the model’s internal structure would change to represent that data, and it might inadvertently reveal it in a future reply. However, including the data in a prompt doesn’t carry that risk because prompt engineering doesn’t change a model’s weights.

Retrieval Augmented Generation (RAG)

RAG refers to giving a language model an information source – such as a database or the Internet – that it can use to improve its output. It has emerged as the most popular technique to control the information the model needs to know when generating answers.

As before, that is not the same thing as training because it does not change the model’s weights.

RAG doesn’t modify the underlying model, but if you connect it to sensitive information and then ask it a question, it may very well reveal something sensitive. RAG is very powerful, but you need to use it with caution. Your AI development platform should provide ways to securely connect to APIs that can help authenticate and retrieve account information, thus allowing you to provide customers with personalized responses.

This is why you still need to think about security when using RAG. Whatever tools or information sources you give your model must meet the strictest security standards and be certified, as appropriate.

Quiq is one such platform, built from the ground-up with data security (encryption in transit) and compliance (SOC 2 certified) in mind. We never store or use data without permission, and we’ve crafted our tools so it’s as easy as possible to utilize RAG on just the information stores you want to plug a model into. Being a security-first company, this extends to our utilization of Large Language Models and agreements with AI providers like Microsoft Open AI.

Wrapping Up on How Quiq Trains LLMs

Hopefully, you now have a much clearer picture of what Quiq does to ensure the models we use are as performant and useful as possible. With them, you can make your customers happier, improve your agents’ performance, and reduce turnover at your contact center.

If you’re interested in exploring some other common misconceptions that CX leaders face when considering incorporating generative AI into their technology stack, check out our ebook on the subject. It contains a great deal of information to help you make the best possible decision!

Request A Demo

Retrieval Augmented Generation – Ultimate Guide

Posted on The Quiq Team

A lot has changed since the advent of large language models a little over a year ago. But, incredibly, there are already many attempts at extending the functionality of the underlying technology.

One broad category of these attempts is known as “tool use”, and consists of augmenting language models by giving them access to things like calculators. Stories of these models failing at simple arithmetic abound, and the basic idea is that we can begin to shore up their weaknesses by connecting them to specific external resources.

Because these models are famously prone to “hallucinating” incorrect information, the technique of retrieval augmented generation (RAG) has been developed to ground model output more effectively. So far, this has shown promise as a way of reducing hallucinations and creating much more trustworthy replies to queries.

In this piece, we’re going to discuss what retrieval augmented generation is, how it works, and how it can make your models even more robust.

Understanding Retrieval Augmented Generation

To begin, let’s get clear on exactly what we’re talking about. The next few sections will overview retrieval augmented generation, break down how it works, and briefly cover its myriad benefits.

What is Retrieval Augmented Generation?

Retrieval augmented generation refers to a large and growing cluster of techniques meant to help large language models ground their output in facts obtained from an external source.

By now, you’re probably aware that language models can do a remarkably good job of generating everything from code to poetry. But, owing to the way they’re trained and the way they operate, they’re also prone to simply fabricating confident-sounding nonsense. If you ask for a bunch of papers about the connection between a supplement and mental performance, for example, you might get a mix of real papers and ones that are completely fictitious.

If you could somehow hook the model up to a database of papers, however, then perhaps that would ameliorate this tendency. That’s where RAG comes in.

We will discuss some specifics in the next section, but in the broadest possible terms, you can think of RAG as having two components: the generative model, and a retrieval system that allows it to augment its outputs with data obtained from an authoritative external source.

The difference between using a foundation model and using a foundation model with RAG has been likened to the difference between taking a closed-book and an open-booked test – the metaphor is an apt one. If you were to poll all your friends about their knowledge of photosynthesis, you’d probably get a pretty big range of replies. Some friends would remember a lot about the process from high school biology, while others would barely even know that it’s related to plants.

Now, imagine what would happen if you gave these same friends a botany textbook and asked them to cite their sources. You’d still get a range of replies, of course, but they’d be far more comprehensive, grounded, and replete with up-to-date details. [1]

How RAG Works

Now that we’ve discussed what RAG is, let’s talk about how it functions. Though there are many subtleties involved, there are only a handful of overall steps.

First, you have to create a source of external data or utilize an existing one. There are already many such external resources, including databases filled with scientific papers, genomics data, time series data on the movements of stock prices, etc., which are often accessible via an API. If there isn’t already a repository containing the information you’ll need, you’ll have to make one. It’s also common to hook generative models up to internal technical documentation, of the kind utilized by e.g. contact center agents.

Then, you’ll have to do a search for relevancy. This involves converting queries into vectors, or numerical representations that capture important semantic information, then matching that representation against the vectorized contents of the external data source. Don’t worry too much if this doesn’t make a lot of sense, the important thing to remember is that this technique is far better than basic keyword matching at turning up documents related to a query.

With that done, you’ll have to augment the original user query with whatever data came up during the relevancy search. In the systems we’ve seen this all occurs silently, behind the scenes, with the user being unaware that any such changes have been made. But, with the additional context, the output generated by the model will likely be much more grounded and sensible. Modern RAG systems are also sometimes built to include citations to the specific documents they drew from, allowing a user to fact-check the output for accuracy.

And finally, you’ll need to think continuously about whether the external data source you’ve tied your model to needs to be updated. It doesn’t do much good to ground a model’s reply if the information it’s using is stale and inaccurate, so this step is important.

The Benefits of RAG

Language models equipped with retrieval augmented generation have many advantages over their more fanciful, non-RAG counterparts. As we’ve alluded to throughout, such RAG models tend to be vastly more accurate. RAG, of course, doesn’t guarantee that a model’s output will be correct. They can still hallucinate, just as one of your friends reading a botany book might misunderstand or misquote a passage. Still, it makes hallucinations far less prevalent and, if the model adds citations, gives you what you need to rectify any errors.

For this same reason, it’s easier to trust a RAG-powered language model, and they’re (usually) easier to use. As we said above a lot of the tricky technical detail is hidden from the end user, so all they see is a better-grounded output complete with a list of documents they can use to check that the output they’ve gotten is right.

Applications of Retrieval Augmented Generation

We’ve said a lot about how awesome RAG is, but what are some of its primary use cases? That will be our focus here, over the next few sections.

Enhancing Question Answering Systems

Perhaps the most obvious way RAG could be used is to supercharge the function of question-answering systems. This is already a very strong use case of generative AI, as attested to by the fact that many people are turning to tools like ChatGPT instead of Google when they want to take a first stab at understanding a new subject.

With RAG, they can get more precise and contextually relevant answers, enabling them to overcome hurdles and progress more quickly.

Of course, this dynamic will also play out in contact centers, which are more often leaning on question-answering systems to either make their agents more effective, or to give customers the resources they need to solve their own problems.

Chatbots and Conversational Agents

Chatbots are another technology that could be substantially upgraded through RAG. Because this is so closely related to the previous section we’ll keep our comments brief; suffice it to say, a chatbot able to ground its replies in internal documentation or a good external database will be much better than one that can’t.

Revolutionizing Content Creation

Because generative models are so, well, generative, they’ve already become staples in the workflows of many creative sorts, such as writers, marketers, etc. A writer might use a generative model to outline a piece, paraphrase their own earlier work, or take the other side of a contentious issue.

This, too, is a place where RAG shines. Whether you’re tinkering with the structure of a new article or trying to build a full-fledged research assistant to master an arcane part of computer science, it can only help to have more factual, grounded output.

Recommendation Systems

Finally, recommendation systems could see a boost from RAG. As you can probably tell from their name, recommendation systems are machine-learning tools that find patterns in a set of preferences and use them to make new recommendations that fit that pattern.

With grounding through RAG, this could become even better. Imagine not only having recommendations, but also specific details about why a particular recommendation was made, to say nothing of recommendations that are tied to a vast set of external resources.

Conclusion

For all the change we’ve already seen from generative AI, RAG has yet more more potential to transform our interaction with AI. With retrieval augmented generation, we could see substantial upgrades in the way we access information and use it to create new things.

If you’re intrigued by the promise of generative AI and the ways in which it could supercharge your contact center, set up a demo of the Quiq platform today!

Request A Demo

Footnotes

[1] This assumes that the book you’re giving them is itself up-to-date, and the same is true with RAG. A generative model is only as good as its data.