{
"slug": "RAG-retrieval-augmented-generation-a93e1a3bf531",
"title": "RAG retrieval-augmented generation",
"subtitle": "retrieval-augmented generation",
"excerpt": "retrieval-augmented generation",
"date": "2024-01-31",
"tags": [
"Machine Learning"
],
"readingTime": "3 min",
"url": "https://medium.com/@mobinshaterian/rag-retrieval-augmented-generation-a93e1a3bf531",
"hero": "https://cdn-images-1.medium.com/max/800/1*2fgyWfZ5s-eFqvqkdfyZOw.png",
"content": [
{
"type": "heading",
"level": 2,
"text": "RAG"
},
{
"type": "heading",
"level": 3,
"text": "retrieval-augmented generation"
},
{
"type": "paragraph",
"html": "In this article we can integrate our documents with ChatGPT and search inside our document based on ChatGPT application and get fantastic results."
},
{
"type": "heading",
"level": 2,
"text": "Normal connection to the ChatGPT Via API key"
},
{
"type": "code",
"lang": "bash",
"code": "pip install openaikey = \"sk-Fh*******\"prompt = 'what is the capital of USA?'client = OpenAI(\napi_key = key)chat_completion = client.chat.completions.create( model=\"gpt-3.5-turbo\",\nmessages=[{\"role\": \"user\", \"content\": prompt}])"
},
{
"type": "code",
"lang": "python",
"code": "message=ChatCompletionMessage(content='The capital of the United States is Washington, D.C.',\nrole='assistant'"
},
{
"type": "heading",
"level": 2,
"text": "LangChain Framework"
},
{
"type": "paragraph",
"html": "LangChain has a number of components designed to help build question-answering applications, and RAG applications more generally. To familiarize ourselves with these, we’ll build a simple Q&A application over a text data source. Along the way we’ll go over a typical Q&A architecture, discuss the relevant LangChain components, and highlight additional resources for more advanced Q&A techniques. We’ll also see how LangSmith can help us trace and understand our application. LangSmith will become increasingly helpful as our application grows in complexity."
},
{
"type": "heading",
"level": 3,
"text": "Indexing"
},
{
"type": "list",
"ordered": true,
"items": [
"Load: First we need to load our data. We’ll use DocumentLoaders for this.",
"Split: Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it in to a model, since large chunks are harder to search over and won’t fit in a model’s finite context window.",
"Store: We need somewhere to store and index our splits, so that they can later be searched over. This is often done using a VectorStore and Embeddings model."
]
},
{
"type": "heading",
"level": 2,
"text": "ChatGPT API Pricing"
},
{
"type": "paragraph",
"html": "The ChatGPT API is priced based on the number of tokens used. A token is a unit of text, and the number of tokens in a given piece of text is equal to the number of words plus the number of spaces."
},
{
"type": "paragraph",
"html": "The current pricing for the ChatGPT API is as follows:"
},
{
"type": "list",
"ordered": false,
"items": [
"GPT-3.5-turbo: $0.006 per 1,000 tokens"
]
},
{
"type": "heading",
"level": 2,
"text": "Step 0 — get API key and connect to ChatGPT"
},
{
"type": "code",
"lang": "python",
"code": "%pip install --upgrade --quiet langchain langchain-community langchainhub langchain-openai chromadb\nbs4\nfrom google.colab import userdata\nuserdata.get('key')"
},
{
"type": "paragraph",
"html": "I put my secret key in collab key store."
},
{
"type": "code",
"lang": "python",
"code": "import os\nfrom langchain_community.document_loaders.csv_loader\nimport CSVLoader\nimport bs4\nfrom langchain\nimport hub\nfrom langchain.text_splitter\nimport RecursiveCharacterTextSplitter\nfrom langchain_community.document_loaders\nimport WebBaseLoader\nfrom langchain_community.vectorstores\nimport Chroma\nfrom langchain_core.output_parsers\nimport StrOutputParser\nfrom langchain_core.runnables\nimport RunnablePassthrough\nfrom langchain_openai\nimport ChatOpenAI, OpenAIEmbeddings"
},
{
"type": "heading",
"level": 3,
"text": "Set key"
},
{
"type": "code",
"lang": "python",
"code": "os.environ[\"OPENAI_API_KEY\"] = userdata.get('key')\nos.environ[\"LANGCHAIN_TRACING_V2\"] = \"false\""
},
{
"type": "heading",
"level": 2,
"text": "Step 1 : Read data from CSV"
},
{
"type": "paragraph",
"html": "You need to prepare your data into excel to gathering data in other ways."
},
{
"type": "code",
"lang": "python",
"code": "loader = CSVLoader(file_path=\"./blogs1.csv\")\ndocs = loader.load()"
},
{
"type": "heading",
"level": 2,
"text": "step 2 : split data"
},
{
"type": "code",
"lang": "python",
"code": "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\nsplits =\ntext_splitter.split_documents(docs)"
},
{
"type": "heading",
"level": 2,
"text": "step 3 : get score from ChatGPT"
},
{
"type": "code",
"lang": "python",
"code": "vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())"
},
{
"type": "heading",
"level": 2,
"text": "Step 4 : Retrieve data"
},
{
"type": "code",
"lang": "text",
"code": "# Retrieve and generate using the relevant snippets of the blog.retriever =\nvectorstore.as_retriever()prompt = hub.pull(\"rlm/rag-prompt\")llm =\nChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)"
},
{
"type": "code",
"lang": "python",
"code": "def format_docs(docs):\n return \"\\n\\n\".join(doc.page_content for doc in docs)"
},
{
"type": "code",
"lang": "python",
"code": "rag_chain = ( {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()} | prompt\n| llm | StrOutputParser())"
},
{
"type": "heading",
"level": 2,
"text": "Step 5 : Ask Questions based on your documents."
},
{
"type": "code",
"lang": "python",
"code": "rag_chain.invoke(\"How was the weather in the Netherlands?\")"
},
{
"type": "heading",
"level": 2,
"text": "Find similarity"
},
{
"type": "code",
"lang": "python",
"code": "retriever_2 = vectorstore.as_retriever(search_type=\"similarity\", search_kwargs={\"k\":\n6})\nretrieved_docs = retriever.invoke(\"What is corepass apllicaion?\")"
},
{
"type": "heading",
"level": 2,
"text": "Metadata filtering in the Vector Store"
},
{
"type": "paragraph",
"html": "https://cassio.org/frameworks/langchain/qa-vector-metadata/
https://python.langchain.com/docs/modules/data_connection/retrievers/vectorstore"
},
{
"type": "heading",
"level": 2,
"text": "Prompt"
},
{
"type": "code",
"lang": "text",
"code": "[HumanMessage(content=\"You are an assistant for question-answering tasks. Use the following pieces o\nf retrieved context to answer the question. If you don't know the answer, just say that you don't\nknow. Use three sentences maximum and keep the answer concise.\\nQuestion: filler question \\nContext:\nfiller context \\nAnswer:\")]"
},
{
"type": "heading",
"level": 2,
"text": "How to use CSV files in vector stores with Langchain"
},
{
"type": "paragraph",
"html": "https://how.wtf/how-to-use-csv-files-in-vector-stores-with-langchain.html"
},
{
"type": "heading",
"level": 2,
"text": "Returning sources"
},
{
"type": "code",
"lang": "python",
"code": "from langchain_core.runnables\nimport RunnableParallelrag_chain_from_docs = (\n RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n | prompt\n | llm\n | StrOutputParser())rag_chain_with_source = RunnableParallel(\n {\"context\": retriever, \"question\": RunnablePassthrough()}).assign(answer=rag_chain_from_docs)"
},
{
"type": "code",
"lang": "python",
"code": "res = rag_chain_with_source.invoke(\"How can I excange CTN o botcoin?\")"
},
{
"type": "code",
"lang": "text",
"code": "blogs_data.iloc[res['context'][0].metadata['row']]['URL']"
},
{
"type": "heading",
"level": 2,
"text": "ChatPromptTemplate"
},
{
"type": "heading",
"level": 2,
"text": "Introduction to OpenAI and LangChain"
},
{
"type": "heading",
"level": 2,
"text": "Error This model’s maximum context length is 4097 tokens"
},
{
"type": "code",
"lang": "text",
"code": "This\nmodel's maximum context length is 4097 tokens. However, your messages resulted in 4969 tokens. Please reduce the length of the messages."
},
{
"type": "image",
"src": "https://cdn-images-1.medium.com/max/800/1*2fgyWfZ5s-eFqvqkdfyZOw.png",
"alt": "RAG retrieval-augmented generation",
"caption": "",
"width": 671,
"height": 431
},
{
"type": "paragraph",
"html": "Visit us at DataDrivenInvestor.com"
},
{
"type": "paragraph",
"html": "Subscribe to DDIntel here."
},
{
"type": "paragraph",
"html": "Have a unique story to share? Submit to DDIntel here."
},
{
"type": "paragraph",
"html": "Join our creator ecosystem here."
},
{
"type": "paragraph",
"html": "DDIntel captures the more notable pieces from our main site and our popular DDI Medium publication. Check us out for more insightful work from our community."
},
{
"type": "paragraph",
"html": "DDI Official Telegram Channel: https://t.me/+tafUp6ecEys4YjQ1"
},
{
"type": "paragraph",
"html": "Follow us on LinkedIn, Twitter, YouTube, and Facebook."
}
]
}