{
  "slug": "Get-Data-from-medium-and-make-an-AI-model-e2eda73f62cb",
  "title": "Get Data from medium and make an AI model",
  "subtitle": "In this tutorial I am going to get data from medium a make my version AI bot.",
  "excerpt": "In this tutorial I am going to get data from medium a make my version AI bot.",
  "date": "2024-02-23",
  "tags": [
    "Design Rag System",
    "Machine Learning"
  ],
  "readingTime": "1 min",
  "url": "https://medium.com/@mobinshaterian/get-data-from-medium-and-make-an-ai-model-e2eda73f62cb",
  "hero": "https://cdn-images-1.medium.com/max/800/1*VPLYUecubOuFJtCGgNNFKA.jpeg",
  "content": [
    {
      "type": "heading",
      "level": 2,
      "text": "Get Data from medium and make an AI model"
    },
    {
      "type": "paragraph",
      "html": "In this tutorial I am going to get data from medium a make my version AI bot."
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*VPLYUecubOuFJtCGgNNFKA.jpeg",
      "alt": "Get Data from medium and make an AI model",
      "caption": "",
      "width": 640,
      "height": 832
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Export your account data"
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*gRRUdZca8HHbfo9wyQVDEQ.png",
      "alt": "Get Data from medium and make an AI model",
      "caption": "",
      "width": 904,
      "height": 395
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*nftU2-k_n7nJ_WW48qHGlQ.png",
      "alt": "Get Data from medium and make an AI model",
      "caption": "",
      "width": 776,
      "height": 361
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*0AA362-f2DZMoPReW0iFNQ.png",
      "alt": "Get Data from medium and make an AI model",
      "caption": "",
      "width": 534,
      "height": 158
    },
    {
      "type": "paragraph",
      "html": "<strong>Posts exist on “posts” folder.</strong>"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Find the text of the given tag using BeautifulSoup"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "# Importing BeautifulSoup class from the bs4 module from bs4 import BeautifulSoup   # Opening the\nhtml file HTMLFile = open(\"1.html\", \"r\") # Reading the file index = HTMLFile.read() # Creating a\nBeautifulSoup object and specifying the parser Parse = BeautifulSoup(index, 'lxml')\nprint(Parse.title.string)\nprint(Parse.body)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Get content of Medium blog"
    },
    {
      "type": "paragraph",
      "html": "The class might be change in future it depend on medium policy."
    },
    {
      "type": "code",
      "lang": "python",
      "code": "text_element = Parse.find(class_='section-inner sectionLayout--insetColumn')\ntext =\ntext_element.get_text(strip=True)\ntext = text.replace(u'\\xa0', u' ')\ntext = (text.encode('ascii',\n'ignore')).decode(\"utf-8\")"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Get Link of post"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "link_element = Parse.find(class_='p-canonical')\nlink = link_element.get('href')"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Get first image of post"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "images = Parse.findAll('img')\npicture = images[0].get('src')"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "RAQ"
    },
    {
      "type": "paragraph",
      "html": "Now data is ready and can send to the RAG system. I explain it in another article."
    },
    {
      "type": "paragraph",
      "html": "Visit us at <a href=\"https://www.datadriveninvestor.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>DataDrivenInvestor.com</em></a>"
    },
    {
      "type": "paragraph",
      "html": "Subscribe to DDIntel <a href=\"https://www.ddintel.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>here</em></a>."
    },
    {
      "type": "paragraph",
      "html": "Have a unique story to share? Submit to DDIntel <a href=\"https://datadriveninvestor.com/ddintelsubmission\" target=\"_blank\" rel=\"noreferrer noopener\"><em>here</em></a>."
    },
    {
      "type": "paragraph",
      "html": "Join our creator ecosystem <a href=\"https://join.datadriveninvestor.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>here</em></a>."
    },
    {
      "type": "paragraph",
      "html": "<a href=\"https://ddintel.datadriveninvestor.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>DDIntel</em> </a>captures the more notable pieces from our <a href=\"https://www.datadriveninvestor.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>main site</em></a> and our popular <a href=\"https://medium.datadriveninvestor.com/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>DDI Medium publication</em></a>. Check us out for more insightful work from our community."
    },
    {
      "type": "paragraph",
      "html": "DDI Official Telegram Channel: <a href=\"https://t.me/+tafUp6ecEys4YjQ1\" target=\"_blank\" rel=\"noreferrer noopener\">https://t.me/+tafUp6ecEys4YjQ1</a>"
    },
    {
      "type": "paragraph",
      "html": "Follow us on <a href=\"https://www.linkedin.com/company/data-driven-investor\" target=\"_blank\" rel=\"noreferrer noopener\"><em>LinkedIn</em></a>, <a href=\"https://twitter.com/@DDInvestorHQ\" target=\"_blank\" rel=\"noreferrer noopener\"><em>Twitter</em></a>, <a href=\"https://www.youtube.com/c/datadriveninvestor\" target=\"_blank\" rel=\"noreferrer noopener\"><em>YouTube</em></a>, and <a href=\"https://www.facebook.com/datadriveninvestor\" target=\"_blank\" rel=\"noreferrer noopener\"><em>Facebook</em></a>."
    }
  ]
}