{
  "slug": "Read-data-from-JSON-and-normalize-then-write-into-excel-cb7c29d1268b",
  "title": "Read data from JSON and normalize then write into excel",
  "subtitle": "read data from json and write in CSV",
  "excerpt": "read data from json and write in CSV",
  "date": "2024-01-23",
  "tags": [
    "Machine Learning"
  ],
  "readingTime": "2 min",
  "url": "https://medium.com/@mobinshaterian/read-data-from-json-and-normalize-then-write-into-excel-cb7c29d1268b",
  "hero": "https://cdn-images-1.medium.com/max/800/1*ahuQkH6ZJ5bmqvJnpdpFIA.jpeg",
  "content": [
    {
      "type": "heading",
      "level": 2,
      "text": "Read data from JSON and normalize then write into excel"
    },
    {
      "type": "paragraph",
      "html": "read data from JSON and write in CSV"
    },
    {
      "type": "image",
      "src": "https://cdn-images-1.medium.com/max/800/1*ahuQkH6ZJ5bmqvJnpdpFIA.jpeg",
      "alt": "Read data from JSON and normalize then write into excel",
      "caption": "",
      "width": 640,
      "height": 832
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Install Pandas"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "pip install pandas"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Ubuntu Jupyter notebooks"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "pip install jupyterlabjupyter labjupyter server list"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Read data"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "import pandas as pddf = pd.read_json('data.json')"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Show length of data"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "len(df.index)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Give information about data frame"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "df.info()"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Get first element"
    },
    {
      "type": "paragraph",
      "html": "Get first element and access to the index"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "first_element = df.iloc[0]first_element['helpdesk_ticket']"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Classes in python"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "class Person:  def __init__(self, name, age):    self.name = name    self.age = agep1 =\nPerson(\"John\", 36)print(p1.name)print(p1.age)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "What is a Series?"
    },
    {
      "type": "paragraph",
      "html": "A Pandas Series is like a column in a table."
    },
    {
      "type": "paragraph",
      "html": "It is a one-dimensional array holding data of any type."
    },
    {
      "type": "code",
      "lang": "python",
      "code": "import pandas as pda = [1, 7, 2]myvar = pd.Series(a)print(myvar)"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "0    11    72    2dtype: int64"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Similarity"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Python For Loops"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "fruits = [\"apple\", \"banana\", \"cherry\"]\nfor x in fruits:  print(x)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Text Normalization for Natural Language Processing in Python"
    },
    {
      "type": "heading",
      "level": 3,
      "text": "Install package"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "pip install nltk\npip install unidecode\npip install pycontractions\npip install word2number"
    },
    {
      "type": "paragraph",
      "html": "if got error of java version"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "$ sudo apt install openjdk-8-jdk$ sudo update-alternatives --set java\n/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java$ pip install language-check$ pip install\npycontractions"
    },
    {
      "type": "code",
      "lang": "bash",
      "code": "pip install language-tool-python"
    },
    {
      "type": "paragraph",
      "html": "Download package"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "python3\nimport nltknltk.download()punkt"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Tokenization"
    },
    {
      "type": "paragraph",
      "html": "This package will tokenize sentences."
    },
    {
      "type": "code",
      "lang": "python",
      "code": "sentences =\nsent_tokenize(text)\nprint(sentences)['“Everything we’re doing is about going forward,” Phoebe Philo told Vogue in 2009, shortly before showing her debut Resort collection for Céline.', 'Although the label had garnered headlines when it was revived by Michael Kors in the late ’90s, it was Philo who truly brought the till then somewhat somnambulant luxury house to the forefront.', 'Critics credited her with pushing fashion in a new direction, toward a more spare, stripped-down kind of sophistication.', 'What Céline now offered women was, as the magazine put it, “a grown-up and hip way to put themselves together.”']"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Removing stopwords"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "stop_words = set(stopwords.words('english'))\nfiltered = [word for word in\nword_tokenize(test_sentence) if word not in stop_words]\nprint(filtered)['“', 'Everything', 'going',\n'forward', '”', 'Phoebe', 'Philo', 'told', 'Vogue', '2009', 'shortly', 'showing', 'debut', 'Resort',\n'collection', 'Céline', '.']"
    },
    {
      "type": "code",
      "lang": "python",
      "code": ">>> import nltk>>> nltk.download('stopwords')"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Whitespace"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "test_sentence = re.sub(' +',' ', test_sentence)\nprint(test_sentence)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Converting to lowercase"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "test_sentence = test_sentence.lower()\nprint(test_sentence)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Expanding contractions"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "pattern = r'we[\\’\\']re'\nreplacement = 'we are'\ntest_sentence =\nre.sub(pattern,replacement,test_sentence)\nprint(test_sentence)"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Downloader API for gensim"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "import gensim.downloader as apiapi.info()  # return dict with info about available\nmodels/datasetsapi.info(\"text8\")  # return dict with info about \"text8\" dataset"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "import gensim.downloader as apimodel = api.load(\"glove-twitter-25\")  # load glove vectors"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Solve problem install package"
    },
    {
      "type": "code",
      "lang": "text",
      "code": "Check Java installation :    for language-check, Java 8 is expected    For MacOS: Follow these steps\n: https://stackoverflow.com/a/24657630    Install language-checkpip install language-check,\nwon't directly work, you would have to:    Install from a forked & fixed repo for language-check    pip install git+https://github.com/MCFreddie777/language-check.git    Install pyemdpip install pyemd again runs into installation issues in Python 3.7, therefore, follow the following steps:    Ensure you have XCode installed - xcode-select --install    Set export MACOSX_DEPLOYMENT_TARGET=10.9    pip install pyemd, should now work    Now, pip install pycontractions should work!"
    },
    {
      "type": "heading",
      "level": 2,
      "text": "Simple normalization"
    },
    {
      "type": "code",
      "lang": "python",
      "code": "@staticmethod    def simple_normalize(text) :        text = re.sub(' +',' ', text)        text =\ntext.replace('\\n', ' ')        text = text.replace('\\t', ' ')        text = text.replace('\\r', ' ')\ntext = text.replace('\\\"', ' ')        text = text.replace('\\'', ' ')        text = text.lower()\npattern = r'we[\\'\\']re'        replacement = 'we are'        text = re.sub(pattern,replacement,text)\npattern = r'we[\\'\\']ll'        replacement = 'we will'        text =\nre.sub(pattern,replacement,text)         return text"
    }
  ]
}