pandas.core.indexing.IndexingError: слишком много индексаторов с NaN в мультииндексе

Question

Рассмотрим следующий фрейм данных

my_df = pd.DataFrame()
my_df.at[0,'tunnel1']=3
my_df.at[1,'tunnel1']=3
my_df.at[1,'tunnel2']=2
my_df.at[2,'tunnel1']=3
my_df.at[2,'tunnel2']=2
my_df.at[3,'tunnel1']=4
my_df.at[3,'tunnel2']=1
my_df.at[3,'tunnel3']=4
my_df.at[4,'tunnel1']=1
my_df.at[4,'tunnel2']=5
my_df.at[4,'tunnel3']=1
my_df.at[5,'tunnel1']=1
my_df.at[5,'tunnel2']=5
my_df.at[5,'tunnel3']=1
my_df.at[5,'tunnel4']=3
my_df.at[6,'tunnel1']=6
my_df.at[6,'tunnel2']=5
my_df.at[6,'tunnel3']=5
my_df.at[6,'tunnel4']=2
my_df['data1']='ham'
my_df['data2']='eggs'
my_df['data3']='coffee'

df выглядит как

   tunnel1  tunnel2  tunnel3  tunnel4 data1 data2   data3
0      3.0      NaN      NaN      NaN   ham  eggs  coffee
1      3.0      2.0      NaN      NaN   ham  eggs  coffee
2      3.0      2.0      NaN      NaN   ham  eggs  coffee
3      4.0      1.0      4.0      NaN   ham  eggs  coffee
4      1.0      5.0      1.0      NaN   ham  eggs  coffee
5      1.0      5.0      1.0      3.0   ham  eggs  coffee
6      6.0      5.0      5.0      2.0   ham  eggs  coffee

Затем установите мультииндекс

my_df = my_df.set_index(['tunnel1', 'tunnel2', 'tunnel3', 'tunnel4'])

Похоже

                               data1 data2   data3
tunnel1 tunnel2 tunnel3 tunnel4                    
3.0     NaN     NaN     NaN       ham  eggs  coffee
        2.0     NaN     NaN       ham  eggs  coffee
                        NaN       ham  eggs  coffee
4.0     1.0     4.0     NaN       ham  eggs  coffee
1.0     5.0     1.0     NaN       ham  eggs  coffee
                        3.0       ham  eggs  coffee
6.0     5.0     5.0     2.0       ham  eggs  coffee

Теперь я хочу нарезать его так, чтобы получить строки для каждой уникальной записи мультииндекса

for configuration in my_df.index.unique():
            mini_df=my_df.loc[configuration] 

pandas.core.indexing.IndexingError: Слишком много индексаторов

Первый индексный слайдер -

configuration
(3.0, nan, nan, nan)

И это, я считаю, вызывает ошибку.

То, что я хочу из моего цикла, это

mini_df

   tunnel1  tunnel2  tunnel3  tunnel4 data1 data2   data3
0      3.0      NaN      NaN      NaN   ham  eggs  coffee

mini_df '

   tunnel1  tunnel2  tunnel3  tunnel4 data1 data2   data3
1      3.0      2.0      NaN      NaN   ham  eggs  coffee
2      3.0      2.0      NaN      NaN   ham  eggs  coffee

mini_df' '

   tunnel1  tunnel2  tunnel3  tunnel4 data1 data2   data3
3      4.0      1.0      4.0      NaN   ham  eggs  coffee

mini_df' ''

   tunnel1  tunnel2  tunnel3  tunnel4 data1 data2   data3
4      1.0      5.0      1.0      NaN   ham  eggs  coffee

Любые предложения о том, что попробовать здесь, пожалуйста? Заранее спасибо за помощь.

Answer 1

Использование DataFrame.xs + Index.get_level_values:

for id1 in my_df.index.get_level_values(0).unique():
    print(my_df.xs(id1))

Вы можете сохранить кадры данных в формате dict:

df_id1={id1:my_df.xs(id1) for id1 in my_df.index.get_level_values(0).unique()}
for key in df_id1:
    print(f'df_id1[{key}]')
    print('-'*50)
    print(df_id1[key])

df_id1[3.0]
--------------------------------------------------
                        data1 data2   data3
tunnel2 tunnel3 tunnel4                    
NaN     NaN     NaN       ham  eggs  coffee
2.0     NaN     NaN       ham  eggs  coffee
                NaN       ham  eggs  coffee
df_id1[4.0]
--------------------------------------------------
                        data1 data2   data3
tunnel2 tunnel3 tunnel4                    
1.0     4.0     NaN       ham  eggs  coffee
df_id1[1.0]
--------------------------------------------------
                        data1 data2   data3
tunnel2 tunnel3 tunnel4                    
5.0     1.0     NaN       ham  eggs  coffee
                3.0       ham  eggs  coffee
df_id1[6.0]
--------------------------------------------------
                        data1 data2   data3
tunnel2 tunnel3 tunnel4                    
5.0     5.0     2.0       ham  eggs  coffee

---

Мы также можем использовать DataFrame.groupby:

for i, group in my_df.groupby(level=0):
#for i, group in my_df.groupby('tunnel1'): #latest versions of pandas
    print(group)

                                data1 data2   data3
tunnel1 tunnel2 tunnel3 tunnel4                    
1.0     5.0     1.0     NaN       ham  eggs  coffee
                        3.0       ham  eggs  coffee
                                data1 data2   data3
tunnel1 tunnel2 tunnel3 tunnel4                    
3.0     NaN     NaN     NaN       ham  eggs  coffee
        2.0     NaN     NaN       ham  eggs  coffee
                        NaN       ham  eggs  coffee
                                data1 data2   data3
tunnel1 tunnel2 tunnel3 tunnel4                    
4.0     1.0     4.0     NaN       ham  eggs  coffee
                                data1 data2   data3
tunnel1 tunnel2 tunnel3 tunnel4                    
6.0     5.0     5.0     2.0       ham  eggs  coffee

Answer 2

Почему бы не попробовать replace или fillna NaN со строкой 'NaN'

my_df = my_df.fillna('NaN').set_index(['tunnel1', 'tunnel2', 'tunnel3', 'tunnel4'])