Retrieval Augmented Generative AI is an SAT-word way of saying that data is being fetched and injected into the prompt that is sent to the LLM AI.

We do it manually all the time! If you've ever copied some text into ChatGPT for summarization, you are the data-entry cog in a manual RAG process.

It's built into ChatGPT. You can ask ChatGPT to scrape a webpage and if you get lucky, it'll work. You can hook Actions into the GPTs and they can access external APIs, which works significantly better.

Gathering data for RAG is like doing research. You could write an essay without any research, winging it based on your knowledge. Or you could research and use what you find to have a well-informed argument and supporting evidence. Which one is going to be better?

Now, AI has a vast knowledge, but it tends to hallucinate answers when it doesn't know. Loading the knowledge that you want it to use into the prompt is a great way to mitigate the hallucination impact and ensure it uses the knowledge you're supplying.

But, you're better than copying and pasting, that's for interns and "data entry specialists". Copying and pasting is below you, you have more important and interesting work to do.

How can you leverage RAG? Where can you get this data? Let's consider some sources for RAG data:

Public Sources like Webpages

Pages like wikipedia, news stories or blog articles or what you see showing up in Google. They don't require you to be logged in or to have purchased anything. They're just available.

With this, your AI can know about things that just happened, which can be incredibly valuable.

ChatGPT has the ability to scrape a webpage for you. But, it doesn't handle JavaScript and I see it fail more than half the time.

Google's Gemini probably uses a form of RAG here, looking up pages based on your query and using that as part of the context that it will answer with. X's Grok probably does as well.

LangChain has some Document Loaders, so if you're technical, you can use those to scrape pages.

This data may (or may not) eventually get trained into a LLM, but if it's been created after the training, the LLM won't know about it. RAG changes that!

You can search and scrape live or capture scrapes asynchronously and save them so you can look them up internally.

Licensed Data like Books

Data you have access to, but isn't your personal data.

This could be a book that you bought that contains specific data and frameworks that wouldn't have been trained into the base LLM. For example, the LLM might know about marketing in general, but not know the depths about marketing for DTC apparel brands. It may know about history, but not about specific battles as outlined in a book.

Or, it could be a private database you'd like to reference, like a product catalog, so you can recommend (and get affiliate revenue from) products.

Or, a lookup, like a check to see if a domain name is available so you can consider a bunch of domain names.

These licensed data services may have different pricing and data models. Some may be a one-time payment and you'll get all the data you can use however you'd like. Some may charge per-request to their API and not give you full access to the data which may change over time.

There's a growing need for RAG Data Licensing Marketplaces, connecting vendors who have data sets and want to monetize that data with AI processes hungry for relevant data.

Your Private Data

Data about you that you probably wouldn't want an LLM trained on.

If you're running a business, this could be information about your products, vendors and customers so you can ask questions about trends or new solutions. It might give you access to recent marketing campaigns and their performance and creative content that you can use for optimization. It could contain product roadmap plans that you can reference to identify any gaps in your plans.

For your more personal side of things, having access to your emails or notes would unlock personalized to-do lists and summaries.

We already somewhat have this with Snippets in PyroPrompts, but we don't yet have integration into other tools you're using. This is changing soon (as of writing this on 2024-03-18), keep an eye out for RAG integrations coming soon.

Which RAG should you use?

It depends on what you're doing, but you want your AI to have access to all the data it needs to get the job done. All the data you'd want access to if you're manually gathering data to make a decision.

Here's a fun example that incorporates Public, Licensed and Private data:

Let's say that you run a bagel shop in a small community. You might be interested in upcoming events that you could sell or promote at, so you regularly track some local news websites. You might also reference a list of recipes, either your personal ones or ones in a book for ideas for specials and new offerings. You might reference a book on small-business restaurant management for ideas on growing your business. And you might check your reviews to understand any issues every week and check your revenues daily and weekly to understand how your business is growing.

You can incorporate the news and events with recipes to alert you on upcoming events and suggest a special or advertisement you could promote. You can incorporate reviews with recipes to suggest items you should add to your menu or meals you should revisit. You can reference the reviews with the restaurant management book to highlight issue trends and suggest fixes. And ultimately, you can reference your revenues with the business book to grow your business.

AI becomes exponentially more powerful when you incorporate multiple relevant data sources. RAG makes that possible.

How you can get started

ChatGPT has a lot of these already integrated, so if you still pay the $20/month there, you should try these. Their new (about a month old) functionality around mentioning GPTs in a chat can allow you to "tag-in" a GPT that has a news integration or stripe integration built in. Try this, it opens up the GPTs so you can have them interact. Learn more about GPT Mentions

PyroPrompts is beginning to incorporate RAG as well. This week, we are launching our RAG Closed Beta, starting with RSS feeds. Connect to any RSS feed so you can begin incorporating news and blog posts. Based on the feedback in the Closed Beta, additional RAG integrations will be added before we open it up to everyone. If you'd like to join this beta, contact matt@pyroprompts.com before the end of March for access.

Wrapping Up

RAG is how we'll get AI further integrated into our lives. We'll see it everywhere and it'll be transparent, we won't even know or care about it.

It's a tremendous opportunity to provide RAG services. If you have data that people would pay for, build a RAG integration that you can sell. Or, you can build an integration to private data and make that available via RAG.