Get Data from medium and make an AI model
In this tutorial I am going to get data from medium a make my version AI bot.
Export your account data
Posts exist on “posts” folder.
Find the text of the given tag using BeautifulSoup
python
# Importing BeautifulSoup class from the bs4 module from bs4 import BeautifulSoup # Opening the
html file HTMLFile = open("1.html", "r") # Reading the file index = HTMLFile.read() # Creating a
BeautifulSoup object and specifying the parser Parse = BeautifulSoup(index, 'lxml')
print(Parse.title.string)
print(Parse.body)Get content of Medium blog
The class might be change in future it depend on medium policy.
python
text_element = Parse.find(class_='section-inner sectionLayout--insetColumn')
text =
text_element.get_text(strip=True)
text = text.replace(u'\xa0', u' ')
text = (text.encode('ascii',
'ignore')).decode("utf-8")Get Link of post
python
link_element = Parse.find(class_='p-canonical')
link = link_element.get('href')Get first image of post
python
images = Parse.findAll('img')
picture = images[0].get('src')RAQ
Now data is ready and can send to the RAG system. I explain it in another article.
Visit us at DataDrivenInvestor.com
Subscribe to DDIntel here.
Have a unique story to share? Submit to DDIntel here.
Join our creator ecosystem here.
DDIntel captures the more notable pieces from our main site and our popular DDI Medium publication. Check us out for more insightful work from our community.
DDI Official Telegram Channel: https://t.me/+tafUp6ecEys4YjQ1