Замена кадра данных не работает - encoding = 'ISO-8859-1', Python 3.6

Question

Я импортировал файл CSV во фрейм данных

Data = pd.read_csv(filePath, encoding = 'ISO-8859-1', dtype=object)

Я заменяю столбец «Индикатор» некоторыми значениями

DataT['Indicator'] = DataT['Indicator'].str.replace('export(us$ mil)', 'exports (in us$ mil)')
DataT['Indicator'] = DataT['Indicator'].str.replace('import(us$ mil)', 'imports (in us$ mil)')

Но замена не работает из-за проблемы с кодировкой.

Подскажите, пожалуйста, как решить эту проблему?

Файл загружен с: http://wits.worldbank.org/data/public/cp/wits_en_trade_summary_allcountries_allyears.zip

Код для импорта всех файлов CSV: -

for i, file in os.listdir(sourcePath):
    if file.upper().endswith('.CSV'):
    filePath = os.path.join(sourcePath, file)
    Data = pd.read_csv(filePath, encoding = 'ISO-8859-1', dtype=object) 

    Data['FileName'] = file
    DataAll = pd.concat([DataAll, Data], sort=False)

Answer 1

После долгих проб, я попал в приведенное ниже решение, просто импортируйте re module.

Однако вы можете упростить свой код как:

import pandas as pd
import glob
import re
for f in glob('/your_Dir_path/somefiles*.csv'):
    Data = pd.read_csv(f, encoding = 'ISO-8859-1', dtype=object)

Dataset:

>>> Data['Indicator'].head()
0     GDP (current US$ Mil)
1    No. Of Export partners
2    No. Of Export products
3    No. Of Import partners
4    No. Of Import products
Name: Indicator, dtype: object
>>> Data['Indicator'].head(100)
0                     GDP (current US$ Mil)
1                    No. Of Export partners
2                    No. Of Export products
3                    No. Of Import partners
4                    No. Of Import products
5                   No. Of Tariff Agreement
6           Trade Balance (current US$ Mil)
7      Trade (US$ Mil)-Top 5 Export Partner
8      Trade (US$ Mil)-Top 5 Export Partner
9      Trade (US$ Mil)-Top 5 Export Partner
10     Trade (US$ Mil)-Top 5 Export Partner
11     Trade (US$ Mil)-Top 5 Import Partner
12     Trade (US$ Mil)-Top 5 Export Partner
13     Trade (US$ Mil)-Top 5 Import Partner
14     Trade (US$ Mil)-Top 5 Export Partner
15     Trade (US$ Mil)-Top 5 Import Partner
16     Trade (US$ Mil)-Top 5 Export Partner
17     Trade (US$ Mil)-Top 5 Export Partner
18     Trade (US$ Mil)-Top 5 Import Partner

Результат:

>>> Data['Indicator'].str.replace(re.escape("Trade (US$ Mil)"), "IN Trade (US$ Mil)").head(100)
0                       GDP (current US$ Mil)
1                      No. Of Export partners
2                      No. Of Export products
3                      No. Of Import partners
4                      No. Of Import products
5                     No. Of Tariff Agreement
6             Trade Balance (current US$ Mil)
7     IN Trade (US$ Mil)-Top 5 Export Partner
8     IN Trade (US$ Mil)-Top 5 Export Partner
9     IN Trade (US$ Mil)-Top 5 Export Partner
10    IN Trade (US$ Mil)-Top 5 Export Partner
11    IN Trade (US$ Mil)-Top 5 Import Partner
12    IN Trade (US$ Mil)-Top 5 Export Partner
13    IN Trade (US$ Mil)-Top 5 Import Partner
14    IN Trade (US$ Mil)-Top 5 Export Partner
15    IN Trade (US$ Mil)-Top 5 Import Partner
16    IN Trade (US$ Mil)-Top 5 Export Partner
17    IN Trade (US$ Mil)-Top 5 Export Partner
18    IN Trade (US$ Mil)-Top 5 Import Partner
19    IN Trade (US$ Mil)-Top 5 Import Partner
20    IN Trade (US$ Mil)-Top 5 Import Partner
21    IN Trade (US$ Mil)-Top 5 Export Partner
22    IN Trade (US$ Mil)-Top 5 Export Partner
23    IN Trade (US$ Mil)-Top 5 Export Partner
24    IN Trade (US$ Mil)-Top 5 Export Partner
25    IN Trade (US$ Mil)-Top 5 Export Partner
26    IN Trade (US$ Mil)-Top 5 Export Partner
27    IN Trade (US$ Mil)-Top 5 Export Partner
28    IN Trade (US$ Mil)-Top 5 Import Partner
29    IN Trade (US$ Mil)-Top 5 Export Partner
                       ...
70      Partner share(%)-Top 5 Export Partner
71      Partner share(%)-Top 5 Import Partner
72      Partner share(%)-Top 5 Export Partner
73      Partner share(%)-Top 5 Import Partner
74      Partner share(%)-Top 5 Export Partner
75      Partner share(%)-Top 5 Export Partner
76      Partner share(%)-Top 5 Import Partner
77      Partner share(%)-Top 5 Import Partner
78      Partner share(%)-Top 5 Import Partner
79      Partner share(%)-Top 5 Export Partner
80      Partner share(%)-Top 5 Export Partner
81      Partner share(%)-Top 5 Export Partner
82      Partner share(%)-Top 5 Export Partner
83      Partner share(%)-Top 5 Export Partner
84      Partner share(%)-Top 5 Export Partner
85      Partner share(%)-Top 5 Export Partner
86      Partner share(%)-Top 5 Import Partner
87      Partner share(%)-Top 5 Export Partner
88      Partner share(%)-Top 5 Import Partner
89      Partner share(%)-Top 5 Export Partner
90                         Country Growth (%)
91           Duty Free Tariff Lines Share (%)
92                    Export Product share(%)
93                    Export Product share(%)
94                    Export Product share(%)
95                    Export Product share(%)
96                    Export Product share(%)
97                    Export Product share(%)
98                    Export Product share(%)
99                    Export Product share(%)
Name: Indicator, Length: 100, dtype: object

Для вашего примера попробуйте ниже:

import re

DataT['Indicator'] = DataT['Indicator'].str.replace(re.escape('export(us$ mil)'), 'exports (in us$ mil)')
DataT['Indicator'] = DataT['Indicator'].str.replace(re.escape('import(us$ mil)'), 'imports (in us$ mil)')

Answer 2

загружая образец из ваших данных, я заметил, что значения для столбца «Индикатор» не все в нижнем регистре - то есть 'Export(US$ Mil)', а не 'export(us$ mil)'. вам нужно либо использовать правильное значение, либо альтернативно:

DataT['Indicator'] = DataT['Indicator'].str.lower().replace('export(us$ mil)',
                                                            'exports (in us$ mil)')

Вы всегда можете проверить уникальные значения для столбца, используя df[col].unique()