Конвертировать csv в Json для отправки в API понимания watson-personal с Python - PullRequest
0 голосов
/ 25 июня 2018

Я пытаюсь преобразовать CSV, который содержит некоторые обзоры, которые я извлек.Мне нужно преобразовать в Json в качестве входных данных для понимания личности IBM watson.

CSV (WordFinal.csv) с отзывами выглядит так:

ID,Review,advice,con,date,employeeType,overallStar,position,pro,reviewLink,reviewNo,summary
3,ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ,"Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ," Nov 9, 2017",Former Employee - Operations Manager,3,I worked at Amazon full-time (More than 3 years),ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm,empReview_17818923,Take it with a grain of salt
19, company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ,None, work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ," Oct 29, 2016",Current Employee - Product Manager,4,I have been working at Amazon full-time (More than 3 years), company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm,empReview_12494284,Incredible growth opportunity with downsides

Формат, принятый IBM-Уотсон выглядит следующим образом:

    {
   "contentItems": [
      {
         "content": "Wow, I liked @TheRock before, now I really SEE how special he is. The daughter story was IT for me. So great! #MasterClass",
         "contenttype": "text/plain",
         "created": 1447639154000,
         "id": "666073008692314113",
         "language": "en"
      },
      {
         "content": ".@TheRock how did you Know to listen to your gut and Not go back to football? #Masterclass",
         "contenttype": "text/plain",
         "created": 1447638226000,
         "id": "666069114889179136",
         "language": "en"
      },
      {
         "content": ".@TheRock moving back in with your parents so humbling. \" on the other side of your pain is something good if you can hold on\" #masterclass",
         "contenttype": "text/plain",
         "created": 1447638067000,
         "id": "666068446325665792",
         "language": "en"
      }   
   ]
}

Я пытаюсь сгенерировать вывод, как его принимает IBM-watson, но я не могу понять, как мне решить эту задачу.Код ниже:

import json, csv

csvfile = 'C:\\WordFinal.csv'
jsonfile = 'C:\\OutputJ.json'

fieldnames=['ID','summary']
data= {}
with open(csvfile) as cs:
    #reader = csv.DictReader(cs)
    reader = csv.DictReader(cs, fieldnames)
    for row in reader:
    csvid = row["ID"]
    data[csvid] = row

with open(jsonfile, "w",encoding='utf8') as js:
    js.write(json.dumps(data,indent=2))

И вывод (мне нужны только столбцы ID и сводки из моего csv - первого и последнего столбца):

   {
  "ID": {
    "ID": "ID",
    "summary": "Review",
    "null": [
      "advice",
      "con",
      "date",
      "employeeType",
      "overallStar",
      "position",
      "pro",
      "reviewLink",
      "reviewNo",
      "summary"
    ]
  },
  "3": {
    "ID": "3",
    "summary": "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
    "null": [
      "Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",
      "my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
      " Nov 9, 2017",
      "Former Employee - Operations Manager",
      "3",
      "I worked at Amazon full-time\u00c2\u00a0(More than 3 years)",
      "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm",
      "empReview_17818923",
      "Take it with a grain of salt"
    ]
  },
  "19": {
    "ID": "19",
    "summary": " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
    "null": [
      "None",
      " work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
      " Oct 29, 2016",
      "Current Employee - Product Manager",
      "4",
      "I have been working at Amazon full-time\u00c2\u00a0(More than 3 years)",
      " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm",
      "empReview_12494284",
      "Incredible growth opportunity with downsides"
    ]
  }
}

Любая идея, как я могу получитьправильный формат?

Ответы [ 3 ]

0 голосов
/ 25 июня 2018

Если вам не нужно ничего, кроме summary, просто храните только summary в data

with open(csvfile) as cs:
    reader = csv.DictReader(cs, fieldnames)
    first_row = next(reader)             # skip reading the header line
    for row in reader[:
        csvid = row["ID"]
        data[csvid] = row['summary']     # just store the summary
0 голосов
/ 25 июня 2018

Не указывайте имена полей.Вытащите те, которые вы хотите из полных записей:

with open(csvfile, "r", encoding="utf-8") as infile:
    reader = csv.DictReader(infile)

    with open(jsonfile, "w", encoding="utf-8") as outfile:
        outfile.write(json.dumps([{"ID": row["ID"], "summary": row["summary"]} for row in reader], indent=2))

Дает:

[
  {
    "ID": "3",
    "summary": "Take it with a grain of salt"
  },
  {
    "ID": "19",
    "summary": "Incredible growth opportunity with downsides"
  }
]
0 голосов
/ 25 июня 2018

Так что с пандами мне гораздо проще работать.

Сначала начните с создания фрейма данных, содержащего вашу CSV-информацию (при условии, что ваш файл в хорошем формате).

import pandas as pd
df = pd.read_csv('WordFinal.csv')

Затем вы можете перебирать кадр данных.

records = []
for index, row in df.iterrows(): 
    rec = { 
        row['ID']: { 
            'ID': row['ID'],
            'summary': row['summary']
            # ... etc
        }
    }

    records.append(rec)

Ваш объект записи будет содержать код JSON, который вы можете сохранить.

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...