Build a RAG with llama

The article describes Retrieval-Augmented Generation (RAG) which helps large language models answer questions based on unseen documents. RAG breaks down documents into sentences, transforms them into numerical representations using a sentence transformer model, and stores them in a database for fast retrieval. When a question is asked, RAG searches the database for relevant sentences and feeds them along with the question to a large language model like Llama for answering.

You can use an RAG workflow to let an LLM answer questions based on documents the model has not seen before.

In this RAG workflow, documents are first broken down into chunks of sentences. They are then transformed into embeddings (a bunch of numbers) using a sentence transformer model. Those embeddings are then stored in a vector database with indexing for fast search and retrieval.

The RAG pipeline is implemented using the LangChain RetrievalQA. It uses the similarity search to search the question against the database. The matching sentences and the question are used as the input to the Llama 2 Chat LLM.

That’s why the LLM can answer questions based on the document: RAG uses vector search to find relevant sentences and includes them in the prompt!

Note that there are many ways to implement RAG. Using vector database is just one of the options.

Gemma google model

Install Ollama

bash

sudo snap install ollama

Ollama List

text

ollama list

Download Ollama2 ans set proxy

text

iF ollama is installed on your machine as a daemon or service, stop it,In most Linux distributions
you can stop the service by executing the following command:sudo systemctl stop ollamathen open a
terminal, and set your proxy information like this: export ALL_PROXY=<your proxy address and port>Be
sure you are in the same Terminal then you can run the ollama using the following command:ollama
serveyou can run the ollama from another terminal (or you can run it as a background process and
then download your LLM using the ollama run llm_name)

bash

sudo systemctl stop ollama

Simple ChatGPT version 3.5

python

import os
from dotenv
import load_dotenvload_dotenv()OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

Ask question

python

from langchain_openai.chat_models
import ChatOpenAImodel = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
model="gpt-3.5-turbo")model.invoke("Why Ai is so important?")AIMessage(content='1. Efficiency: AI can automate repetitive tasks, allowing businesses to operate more efficiently and free up human employees to focus on higher-value tasks.\n\n2. Decision-making: AI can analyze vast amounts of data and provide insights that can help businesses make better decisions and improve their operations.\n\n3. Personalization: AI can analyze customer data and provide personalized recommendations, improving customer experience and driving sales.\n\n4. Innovation: AI can help businesses develop new products and services, and drive innovation through the use of advanced technologies such as machine learning and natural language processing.\n\n5. Competitive advantage: Businesses that leverage AI effectively can gain a competitive edge over their competitors by improving efficiency, decision-making, and customer experience.\n\n6. Scalability: AI can quickly scale to handle large amounts of data and tasks, allowing businesses to grow and expand without worrying about limitations on resources.\n\n7. Risk management: AI can help businesses identify and mitigate risks by analyzing data and predicting potential issues before they occur.\n\n8. Cost savings: AI can help businesses reduce costs by automating tasks, improving efficiency, and reducing the need for human labor.', response_metadata={'finish_reason': 'stop', 'logprobs': None})

Simple code for Llama ask question

python

from langchain_community.llms
import Ollama
from langchain_community.embeddings
import OllamaEmbeddingsMODEL = "llama3"model =
Ollama(model=MODEL)model.invoke("Tell me a funny joke!")

“Here’s one:\n\nWhy couldn’t the bicycle stand up by itself?\n\n(Wait for it…)\n\nBecause it was two-tired!\n\nHope that made you laugh or at least crack a smile! Do you want to hear another one?”

Chain model

python

from langchain_core.output_parsers
import StrOutputParserparser = StrOutputParser()chain = model | parser
chain.invoke("Tell me a joke")

“Why couldn’t the bicycle stand up by itself?\n\nBecause it was two-tired!\n\nHope that made you smile!”

How to load CSVs

python

from langchain_community.document_loaders.csv_loader
import CSVLoaderfile_path = (
    "../../../docs/integrations/document_loaders/example_data/mlb_teams_2012.csv")loader =
    CSVLoader(file_path=file_path)data = loader.load()print(data[0])

Prompt template

python

from langchain.prompts
import PromptTemplatetemplate =
"""Answer the question based on the context below. If you can't answer the question, reply "I
don't know".Context: {context}Question: {question}"""prompt = PromptTemplate.from_template(template)prompt.format(context="Here is some context", question="Here is a question")

Retriever

python

from langchain_community.vectorstores
import Chroma
from langchain_community
import embeddings# 2. Convert documents to Embeddings and store themvectorstore =
Chroma.from_documents(
    documents=data,
    collection_name="rag-chroma",
    embedding=embeddings.ollama.OllamaEmbeddings(model='nomic-embed-text'),)retriever =
    vectorstore.as_retriever()

Chain and use the model

python

chain = prompt | model | parserchain.invoke({"context": "My parents named me Santiago", "question":
"What's your name'?"})

Chain with Source

python

from langchain_core.prompts
import ChatPromptTemplate
from langchain_core.runnables
import RunnablePassthrough
from langchain_core.output_parsers
import StrOutputParser
from langchain_core.runnables
import RunnableParallel
from langchain_core.runnables
import RunnablePassthroughrag_chain = (
    {"context": lambda x: retriever, "question": RunnablePassthrough()} | prompt |
    model)rag_chain_with_source = RunnableParallel(            {"context": retriever, "question":
    RunnablePassthrough()}            ).assign(answer=rag_chain)

nomic-embed-text

nomic-embed-text is a large context length text encoder that surpasses OpenAI text-embedding-ada-002 and text-embedding-3-small performance on short and long context tasks.

Similarity search with score

rag-chat-with-pdf

Ollama Embedding: How to Feed Data to AI for Better Response?

https://mer.vin/2024/02/ollama-embedding/

Embedding models

Build a RAG with llama