Сравните json объектов с CSV-файлом - PullRequest
1 голос
/ 24 февраля 2020

Редактировать: Пока мой код находит сравнения. Я работаю над добавлением данных объекта JSON в строку, где происходит сопоставление слов.

Я пытаюсь найти совпадающие слова между моим JSON файлом и моим CSV, затем проверьте, где это слово имеет низкий рейтинг (столбец с десятичными значениями) из CSV.

Если слово имеет низкий рейтинг. Я записываю время слова и индекс слова (отредактировано). Есть ли способ для меня использовать что-то вроде pandas до l oop над всеми моими json объектами и добавлять данные объектов, когда слова совпадают в крайнем правом столбце моего csv?

Редактировать (в соответствии с ответами, приведенными ниже):

row,col = dfSynsets.shape
for value in contents['words']:
    current_word = value['word']
    for csv_row in range(row):
        curr_csv_word = dfSynsets.loc[csv_row][-1]
        if curr_csv_word == current_word:
            print(curr_csv_word)
            print(current_word)

Этот блок кода производит такой вывод:

universe
universe
in
in
apparent
apparent
mention
mention
passing
passing
way
way
even
even
over
over
there
there
total
total
experiment
experiment
most
most
work
work
by
by
low
low
empty
empty
in
in
fill
fill

Вот пример моего json файла

Json Файл:

{
        "transcript": "The universe is bustling with matter and energy. Even in the vast apparent emptiness of intergalactic space, there's one hydrogen atom per cubic meter. That's not the mention a barrage of particles and electromagnetic radiation passing every which way from stars, galaxies, and into black holes. There's even radiation left over from the Big Bang. So is there such thing as a total absence of everything? This isn't just a thought experiment. Empty spaces, or vacuums, are incredibly useful. Inside our homes,  most vacuum cleaners work by using a fan to create a low-pressure relatively empty area that sucks matter in to fill the void. But that's far from empty. There's still plenty of matter bouncing around. Manufacturers rely on more thorough, sealed vacuums for all sorts of purposes. That includes vacuum-packed food that stays fresh longer, and the vacuums inside early light bulbs that protected filaments from degrading. These vacuums are generally created with some version of what a vacuum cleaner does using high-powered pumps that create enough suction to remove as many stray atoms as possible. But the best of these industrial processes tends to leave hundreds of millions of atoms per cubic centimeter of space. That isn't empty enough for scientists who work on experiments, like the Large Hadron Collider, where particle beams need to circulate at close to the speed of light for up to ten hours without hitting any stray atoms. So how do they create a vacuum? The LHC's pipes are made of materials, like stainless steel, that don't release any  of their own molecules and are lined with a special coating to absorb stray gases. Raising the temperature  to 200 degrees Celsius burns off any moisture, and hundreds of vacuum pumps take two weeks to trap enough gas and debris out of the pipes for the collider's incredibly sensitive experiments. Even with all this, the Large Hadron Collider isn't a perfect vacuum. In the emptiest places, there are still about 100,000 particles per cubic centimeter. But let's say an experiment like that could somehow get every last atom out. There's still an unfathomably huge amount of radiation all around us that can pass right through the walls. Every second, about 50 muons from cosmic rays, 10 million neutrinos coming directly from the Big Bang, 30 million photons from the cosmic microwave background, and 300 trillion neutrinos from the Sun pass through your body. It is possible to shield vacuum chambers with substances, including water, that absorb and reflect this radiation, except for neutrinos. Let's say you've somehow removed all of the atoms and blocked all of the radiation. Is the space now totally empty? Actually, no. All space is filled with what  physicists call quantum fields. What we think of as subatomic particles, electrons and photons and their relatives, are actually vibrations in a quantum fabric that extends throughout the universe. And because of a physical law called the Heisenberg Principle, these fields never stop oscillating, even without any particles to set off the ripples. They always have some minimum fluctuation called a vacuum fluctuation. This means they have energy, a huge amount of it. Because Einstein's equations tell us that mass and energy are equivalent, the quantum fluctuations in every cubic meter of space have an energy that corresponds to a mass of about four protons. In other words, the seemingly empty space inside your vacuum would actually weigh a small amount. Quantum fluctuations have existed since the earliest moments of the universe. In the moments after the Big Bang, as the universe expanded, they were amplified and stretched out to cosmic scales. Cosmologists believe that these original quantum fluctuations were the seeds of everything we see today: galaxies and the entire large scale structure of the universe, as well as planets and solar systems. They're also the center of one of the greatest scientific mysteries of our time because according to the current theories, the quantum fluctuations in the vacuum of space ought to have 120 orders of magnitude more energy than we observe. Solving the mystery of that missing energy may entirely rewrite our understanding of physics and the universe. ",
        "words": [
            {
                "alignedWord": "the",
                "end": 6.31,
                "start": 6.17,
                "word": "The"
            },
            {
                "alignedWord": "universe",
                "end": 6.83,
                "start": 6.31,
                "word": "universe"
            },
            {
                "alignedWord": "is",
                "end": 7.05,
                "start": 6.85,
                "word": "is"
            },
            {
                "alignedWord": "bustling",
                "end": 7.4799999999999995,
                "start": 7.05,
                "word": "bustling"
            },
            {
                "alignedWord": "with",
                "end": 7.65,
                "start": 7.48,
                "word": "with"
            },
            {
                "alignedWord": "matter",
                "end": 7.970000000000001,
                "start": 7.65,
                "word": "matter"
            },
            {
                "alignedWord": "and",
                "end": 8.09,
                "start": 7.97,
                "word": "and"
            },
            {
                "alignedWord": "energy",
                "end": 8.579999,
                "start": 8.099999,
                "word": "energy"
            },
            {
                "alignedWord": "even",
                "end": 9.35,
                "start": 9.08,
                "word": "Even"
            },
            {
                "alignedWord": "in",
                "end": 9.439999,
                "start": 9.349999,
                "word": "in"
            },
            {
                "alignedWord": "the",
                "end": 9.53,
                "start": 9.44,
                "word": "the"
            },
            {
                "alignedWord": "vast",
                "end": 9.84,
                "start": 9.53,
                "word": "vast"
            },
            {
                "alignedWord": "apparent",
                "end": 10.17,
                "start": 9.84,
                "word": "apparent"
            },
            {
                "alignedWord": "emptiness",
                "end": 10.67,
                "start": 10.19,
                "word": "emptiness"
            },
            {
                "alignedWord": "of",
                "end": 10.8,
                "start": 10.67,
                "word": "of"
            }
        ]
}

Вот мой CSV-файл

CSV-файл:

     572714       0.0 ['knocked out', 'kayoed', '"KOd"', 'out', 'stunned']                  "KOd"
0      1771194  0.500000                   ['get', '"get under ones skin"']    "get under ones skin"
1       462301  0.125000              ['south-southwest', '"sou-sou-west"']           "sou-sou-west"
2       250898  0.500000                             ['between', '"tween"']                  "tween"
3      2203763  0.400000                             ['thirteenth', '13th']                     13th
4      2202047  0.333333                                   ['first', '1st']                      1st
...        ...       ...                                                ...                      ...
5552   1848465  0.000000  ['move over', 'give way', 'give', 'ease up', '...                    yield
5553   7176243  0.000000            ['concession', 'conceding', 'yielding']                 yielding
5554  14425853  0.000000                                          ['youth']                    youth
5555   8541841  0.250000                      ['zone', 'geographical zone']                     zone
5556   1943718  0.500000  ['soar', 'soar up', 'soar upwards', 'surge', '...                     zoom

Пример желаемого вывода

col1:synset  col2:rating col3:list col4:word col5:json data
9466280 0.5 ['universe', 'existence', 'creation', 'world', 'cosmos', 'macrocosm']   macrocosm  
{
                "alignedWord": "universe",
                "end": 178.109999,
                "start": 177.599999,
                "word": "universe"
            },

Ответы [ 2 ]

0 голосов
/ 24 февраля 2020

В соответствии с вашими вопросами, я констатирую, что вы хотите пройти по файлу JSON, получить значение клавиши «слово» и сравнить это значение с последним столбцом файла CSV. Если оба слова совпадают, выведите равно , в противном случае 'не равно' .

Если это правильно, найдите следующий подход: -

import pandas as pd
df = pd.read_csv(CSV FILE NAME)
row,col = df.shape
for value in contents['words']:
    current_word = value['word']
    for csv_row in range(row):
        curr_csv_word = df.loc[csv_row][-1]
        if curr_csv_word == current_word:
            print("EQUAL")
        else:
            print("NOT EQUAL")

Надеюсь, вы найдете свой ответ.

0 голосов
/ 24 февраля 2020

Сначала определите функцию отображения:

import json
import pandas

def apply_fun (row):
    for value in contents['words']
        if value['word'] in row['word'] :
            return json.dumps(value)
    return ""

Затем добавьте ее в свой фрейм данных:

x = dfSynsets.apply(lambda row : apply_fun(row),axis=1)
dfSynsets.insert(4,'json_ref',x)
...