Динамически объединяйте строки с одним и тем же ключом в одну

Question

У меня есть Dataframe , и я хотел бы создать еще один столбец, который объединяет столбцы, имя которых начинается с одного и того же значения в ответе и QID.

То есть, вот exerpt фрейма данных:

    QID     Category    Text    QType   Question    Answer0     Answer1
0   16  Automotive  Access to car   Single  Do you have access to a car?    I own a car/cars    I own a car/cars
1   16  Automotive  Access to car   Single  Do you have access to a car?    I lease/ have a company car     I lease/have a company car
2   16  Automotive  Access to car   Single  Do you have access to a car?    I have access to a car/cars     I have access to a car/cars
3   16  Automotive  Access to car   Single  Do you have access to a car?    No, I don’t have access to a car/cars   No, I don't have access to a car
4   16  Automotive  Access to car   Single  Do you have access to a car?    Prefer not to say   Prefer not to say
5   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Audi    Audi
6   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Alfa Romeo  Alfa Romeo
7   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    BMW     BMW
8   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Cadillac    Cadillac
9   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chevrolet   Chevrolet
10  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chrysler    Chrysler
11  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Citroen     Citroen
12  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Daihatsu    Daihatsu
13  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Fiat    Fiat
14  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Ford    Ford
15  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Honda   Honda
16  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Hyundai     Hyundai
...

И я хотел бы получить что-то вроде этого:

    QID     Category    Text    QType   Question    Answer0     Answer1     Answer3     Answer4     Answer5     Answer6     Answer7     Answer8     Answer9     Answer10    Answer11     Answer12     ...      
4   16  Automotive  Access to car   Single  Do you have access to a car?    I own a car/cars    I lease/ have a company car     I have access to a car/cars     No, I don’t have access to a car/cars   Prefer not to say       
5   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Audi    Alfa Romeo  BMW     Cadillac    Chevrolet   Chrysler    Citroen     ...

Благодаря Робу Раймонду Я могу объединить given / stati c номер столбцов, имя которых начинается с одного и того же значения в ответе и QID:

df = pd.DataFrame('path/to/file')

# lazy - want first of all attributes except QID and Answer columns
agg = {col:"first" for col in list(df.columns) if col!="QID" and "Answer" not in col}
# get a list of all answers in Answer0 for a QID
agg = {**agg, **{"Answer0":lambda s: list(s)}}

# helper function for row call.  not needed but makes more readable
def ans(r, i):
    return "" if i>=len(r["AnswerT"]) else r["AnswerT"][i]

# split list from aggregation back out into columns using assign
# rename Answer0 to AnserT from aggregation so that it can be referred to.  
# AnswerT drop it when don't want it any more
dfgrouped = df.groupby("QID").agg(agg).reset_index().rename(columns={"Answer0":"AnswerT"}).assign(
    Answer0=lambda dfa: dfa.apply(lambda r: ans(r, 0), axis=1),
    Answer1=lambda dfa: dfa.apply(lambda r: ans(r, 1), axis=1),
    Answer2=lambda dfa: dfa.apply(lambda r: ans(r, 2), axis=1),
    Answer3=lambda dfa: dfa.apply(lambda r: ans(r, 3), axis=1),
    Answer4=lambda dfa: dfa.apply(lambda r: ans(r, 4), axis=1),
    Answer5=lambda dfa: dfa.apply(lambda r: ans(r, 5), axis=1),
    Answer6=lambda dfa: dfa.apply(lambda r: ans(r, 6), axis=1),
).drop("AnswerT", axis=1)

print(dfgrouped.to_string(index=False))

Как я могу объединить динамический c номер столбцов, имена которых начинаются с одного и того же значения в ответах и ​​QID?

Answer 1

1. сгенерировать список ответов, принадлежащих одному и тому же QID
2. развернуть этот AnswerT список, создав новый фрейм данных динамически
3. merge() это обратно с использованием внутреннего соединения

Это динамическое c - столбцы, построенные в кадре данных, полностью основаны на размере списка

data = """    QID     Category    Text    QType   Question    Answer0     Answer1
0   16  Automotive  Access to car   Single  Do you have access to a car?    I own a car/cars    I own a car/cars
1   16  Automotive  Access to car   Single  Do you have access to a car?    I lease/ have a company car     I lease/have a company car
2   16  Automotive  Access to car   Single  Do you have access to a car?    I have access to a car/cars     I have access to a car/cars
3   16  Automotive  Access to car   Single  Do you have access to a car?    No, I don’t have access to a car/cars   No, I don't have access to a car
4   16  Automotive  Access to car   Single  Do you have access to a car?    Prefer not to say   Prefer not to say
5   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Audi    Audi
6   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Alfa Romeo  Alfa Romeo
7   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    BMW     BMW
8   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Cadillac    Cadillac
9   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chevrolet   Chevrolet
10  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chrysler    Chrysler
11  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Citroen     Citroen
12  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Daihatsu    Daihatsu
13  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Fiat    Fiat
14  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Ford    Ford
15  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Honda   Honda
16  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Hyundai     Hyundai"""

a = [[t.strip() for t in re.split("  ",l) if t!=""]  for l in [re.sub("([0-9]+[ ])*(.*)", r"\2", l) for l in data.split("\n")]]

df = pd.DataFrame(data=a[1:], columns=a[0])

# lazy - want first of all attributes except QID and Answer columns
agg = {col:"first" for col in list(df.columns) if col!="QID" and "Answer" not in col}
# get a list of all answers in Answer0 for a QID
agg = {**agg, **{"Answer0":lambda s: list(s)}}

# helper function for row call.  not needed but makes more readable
def ans(r, i):
    return "" if i>=len(r["AnswerT"]) else r["AnswerT"][i]

# group by QID and construct new column AnswerT which is list of answers
dfgrouped = df.groupby("QID").agg(agg).reset_index().rename(columns={"Answer0":"AnswerT"})#.assign(

# build a new dataframe from AnswerT by building up standard list / dict structure to constructor
# merge on QID and finally drop the temporary AnswerT columns
dfgrouped = dfgrouped.merge(
    pd.DataFrame(
        [{**{"QID":r[0]},**{f"Answer{i}":v for i,v in enumerate(r[1])}} 
         for r in dfgrouped[["QID","AnswerT"]].values.tolist()]
    ), on="QID", how="inner").drop(columns="AnswerT")

print(dfgrouped.to_string(index=False))

выход

QID    Category              Text     QType                                          Question           Answer0                      Answer1                      Answer2                                Answer3            Answer4   Answer5  Answer6   Answer7 Answer8 Answer9 Answer10 Answer11
 16  Automotive     Access to car    Single                      Do you have access to a car?  I own a car/cars  I lease/ have a company car  I have access to a car/cars  No, I don’t have access to a car/cars  Prefer not to say       NaN      NaN       NaN     NaN     NaN      NaN      NaN
 17  Automotive  Make of car/cars  Multiple  If you own/lease a car(s), which brand are they?              Audi                   Alfa Romeo                          BMW                               Cadillac          Chevrolet  Chrysler  Citroen  Daihatsu    Fiat    Ford    Honda  Hyundai