{
  "slug": "Build-a-RAG-with-llama-a24e88e1a399",
  "title": "Build a RAG with llama",
  "subtitle": "The article describes Retrieval-Augmented Generation (RAG) which helps large language models answer questions based on unseen documents…",
  "excerpt": "The article describes Retrieval-Augmented Generation (RAG) which helps large language models answer questions based on unseen documents…",
  "date": "2024-06-06",
  "tags": [
    "Design Rag System",
    "Machine Learning"
  ],
  "readingTime": "4 min",
  "url": "https://medium.com/@mobinshaterian/build-a-rag-with-llama-a24e88e1a399",
  "hero": "https://cdn-images-1.medium.com/max/800/1*I0M07vQcobkzqXuslY6EFw.png",
  "content": [
    {
      "type": "heading",
      "level": 2,
      "text": "Build a RAG with llama"
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*I0M07vQcobkzqXuslY6EFw.png",
      "alt": "https://agi-sphere.com/retrieval-augmented-generation-llama2/",
      "caption": "https://agi-sphere.com/retrieval-augmented-generation-llama2/",
      "width": 834,
      "height": 462
    },
    {
      "type": "paragraph",
      "html": "The article describes Retrieval-Augmented Generation (RAG) which helps large language models answer questions based on unseen documents. RAG breaks down documents into sentences, transforms them into numerical representations using a sentence transformer model, and stores them in a database for fast retrieval. When a question is asked, RAG searches the database for relevant sentences and feeds them along with the question to a large language model like Llama for answering."
    },
    {
      "type": "paragraph",
      "html": "You can use an RAG workflow to let an LLM answer questions based on documents the model has not seen before."
    },
    {
      "type": "paragraph",
      "html": "In this RAG workflow, documents are first broken down into chunks of sentences. They are then transformed into embeddings (a bunch of numbers) using a sentence transformer model. Those embeddings are then stored in a vector database with indexing for fast search and retrieval."
    },
    {
      "type": "paragraph",
      "html": "The RAG pipeline is implemented using the LangChain RetrievalQA. It uses the similarity search to search the question against the database. The matching sentences and the question are used as the input to the Llama 2 Chat LLM."
    },
    {
      "type": "paragraph",
      "html": "That’s why the LLM can answer questions based on the document: RAG uses vector search to find relevant sentences and includes them in the prompt!"
    },
    {
      "type": "paragraph",
      "html": "Note that there are many ways to implement RAG. Using vector database is just one of the options."
    },
    {
      "type": "embed",
      "provider": "youtube",
      "url": "https://www.youtube.com/embed/HRvyei7vFSM?feature=oembed"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Gemma google model"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Install Ollama"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "sudo snap install ollama"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Ollama List"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "ollama list"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Download Ollama2 ans set proxy"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "iF ollama is installed on your machine as a daemon or service, stop it,In most Linux distributions\nyou can stop the service by executing the following command:sudo systemctl stop ollamathen open a\nterminal, and set your proxy information like this: export ALL_PROXY=<your proxy address and port>Be\nsure you are in the same Terminal then you can run the ollama using the following command:ollama\nserveyou can run the ollama from another terminal (or you can run it as a background process and\nthen download your LLM using the ollama run llm_name)"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "sudo systemctl stop ollama"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Simple ChatGPT version 3.5"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "import os\nfrom dotenv\nimport load_dotenvload_dotenv()OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\")"
    },
    {
      "type": "heading",
      "level": 3,
      "text": "Ask question"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_openai.chat_models\nimport ChatOpenAImodel = ChatOpenAI(openai_api_key=OPENAI_API_KEY,\nmodel=\"gpt-3.5-turbo\")model.invoke(\"Why Ai is so important?\")AIMessage(content='1. Efficiency: AI can automate repetitive tasks, allowing businesses to operate more efficiently and free up human employees to focus on higher-value tasks.\\n\\n2. Decision-making: AI can analyze vast amounts of data and provide insights that can help businesses make better decisions and improve their operations.\\n\\n3. Personalization: AI can analyze customer data and provide personalized recommendations, improving customer experience and driving sales.\\n\\n4. Innovation: AI can help businesses develop new products and services, and drive innovation through the use of advanced technologies such as machine learning and natural language processing.\\n\\n5. Competitive advantage: Businesses that leverage AI effectively can gain a competitive edge over their competitors by improving efficiency, decision-making, and customer experience.\\n\\n6. Scalability: AI can quickly scale to handle large amounts of data and tasks, allowing businesses to grow and expand without worrying about limitations on resources.\\n\\n7. Risk management: AI can help businesses identify and mitigate risks by analyzing data and predicting potential issues before they occur.\\n\\n8. Cost savings: AI can help businesses reduce costs by automating tasks, improving efficiency, and reducing the need for human labor.', response_metadata={'finish_reason': 'stop', 'logprobs': None})"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Simple code for Llama ask question"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_community.llms\nimport Ollama\nfrom langchain_community.embeddings\nimport OllamaEmbeddingsMODEL = \"llama3\"model =\nOllama(model=MODEL)model.invoke(\"Tell me a funny joke!\")"
    },
    {
      "type": "quote",
      "html": "<em>“Here’s one:\\n\\nWhy couldn’t the bicycle stand up by itself?\\n\\n(Wait for it…)\\n\\nBecause it was two-tired!\\n\\nHope that made you laugh or at least crack a smile! Do you want to hear another one?”</em>"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Chain model"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_core.output_parsers\nimport StrOutputParserparser = StrOutputParser()chain = model | parser\nchain.invoke(\"Tell me a joke\")"
    },
    {
      "type": "quote",
      "html": "“Why couldn’t the bicycle stand up by itself?\\n\\nBecause it was two-tired!\\n\\nHope that made you smile!”"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "How to load CSVs"
    },
    {
      "type": "embed",
      "provider": "youtube",
      "url": "https://www.youtube.com/embed/5e_h3tvRpog?feature=oembed"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_community.document_loaders.csv_loader\nimport CSVLoaderfile_path = (\n    \"../../../docs/integrations/document_loaders/example_data/mlb_teams_2012.csv\")loader =\n    CSVLoader(file_path=file_path)data = loader.load()print(data[0])"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Prompt template"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain.prompts\nimport PromptTemplatetemplate =\n\"\"\"Answer the question based on the context below. If you can't answer the question, reply \"I\ndon't know\".Context: {context}Question: {question}\"\"\"prompt = PromptTemplate.from_template(template)prompt.format(context=\"Here is some context\", question=\"Here is a question\")"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Retriever"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_community.vectorstores\nimport Chroma\nfrom langchain_community\nimport embeddings# 2. Convert documents to Embeddings and store themvectorstore =\nChroma.from_documents(\n    documents=data,\n    collection_name=\"rag-chroma\",\n    embedding=embeddings.ollama.OllamaEmbeddings(model='nomic-embed-text'),)retriever =\n    vectorstore.as_retriever()"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Chain and use the model"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "chain = prompt | model | parserchain.invoke({\"context\": \"My parents named me Santiago\", \"question\":\n\"What's your name'?\"})"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Chain with Source"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "from langchain_core.prompts\nimport ChatPromptTemplate\nfrom langchain_core.runnables\nimport RunnablePassthrough\nfrom langchain_core.output_parsers\nimport StrOutputParser\nfrom langchain_core.runnables\nimport RunnableParallel\nfrom langchain_core.runnables\nimport RunnablePassthroughrag_chain = (\n    {\"context\": lambda x: retriever, \"question\": RunnablePassthrough()} | prompt |\n    model)rag_chain_with_source = RunnableParallel(            {\"context\": retriever, \"question\":\n    RunnablePassthrough()}            ).assign(answer=rag_chain)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "nomic-embed-text"
    },
    {
      "type": "paragraph",
      "html": "nomic-embed-text is a large context length text encoder that surpasses OpenAI text-embedding-ada-002 and text-embedding-3-small performance on short and long context tasks."
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Similarity search with score"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "rag-chat-with-pdf"
    },
    {
      "type": "embed",
      "provider": "youtube",
      "url": "https://www.youtube.com/embed/TMaQt8rN5bE?feature=oembed"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Ollama Embedding: How to Feed Data to AI for Better Response?"
    },
    {
      "type": "embed",
      "provider": "youtube",
      "url": "https://www.youtube.com/embed/jENqvjpkwmw?feature=oembed"
    },
    {
      "type": "paragraph",
      "html": "<a href=\"https://mer.vin/2024/02/ollama-embedding/\" target=\"_blank\" rel=\"noreferrer noopener\">https://mer.vin/2024/02/ollama-embedding/</a>"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Embedding models"
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*YqWoSjARCd3_GWXXVOrS9w.png",
      "alt": "Build a RAG with llama",
      "caption": "",
      "width": 776,
      "height": 224
    }
  ]
}