Как сравнить 2 CSV файловую базу на нескольких столбцах AWK?

Question

У меня есть 2 CSV-файла с примером формата, как показано ниже, ~ 5000 строк в каждом файле:

файл 1:

    EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110426,Balbon,Susan,Lagat,"abc Equity Ventures, Inc.",Group Internal Audit,Supervisor,I.S. Audit Supervisor,susan.balbon@abc.com
30083,Mendezona,Bingen,Roemer,"abc Equity Ventures, Inc.",Risk Management Office,Vice President,VP - AEV Security,bing.mendezona@abc.test
110773,Casas,Joyce Grace,Bea,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Manager,Tax Counsel,joyce.grace.casas@abc.com
286,Fernandez,Mark Brian,Tato,abc Foundation Inc.,Computer Services Division,Supervisor,Senior Applications Supervisor,mark.fernandez@abc.com
291,Plando,Marilou,Polleros,"abc Equity Ventures, Inc.",Administration,Assistant Vice President,AVP - Risk Management,marilou.plando@abc.test
110813,Gemelo-Abarca,Therese Xyza,Dableo,"abc Equity Ventures, Inc.",Governance & Compliance Team,Manager,Associate General Counsel - Corporate Secretarial and Compliance,therese.xyza.abarca@abc.com
30096,Abay,Joanna Marie,Saluria,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Compliance Officer,joanna.abay@abc.com
110711,Ostan,Margilyn,Salibio,"abc Equity Ventures, Inc.",Accounting,Staff,Senior Accountant 1,margilyn.ostan@abc.com
110732,Fumar-Gonzales,Vanessa Concepcion,Altarejos,"abc Equity Ventures, Inc.",Legal and Corporate Services,Manager,Associate General Counsel - Labor & Litigation,vanessa.gonzales@abc.com

файл 2:

    EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110426,Balbon,Susan,Lagat,"abc Equity Ventures, Inc.",Group Internal Audit,Supervisor,I.S. Audit Supervisor,susan.balbon@abc.com
30083,Mendezona,Bingen,Roemer,"abc Equity Ventures, Inc.",Security,Vice President,VP - AEV Security,jetee.velante@abc.com
110773,Casas,Joyce Grace,Bea,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Counsel,joyce.grace.casas@abc.com
286,Fernandez,Mark Brian,Tato,abc Foundation Inc.,Computer Services Division,Supervisor,Senior Applications Supervisor,mark.fernandez@abc.com
291,Plando,Marilou,Polleros,"abc Equity Ventures, Inc.",Risk Management Office,Assistant Vice President,AVP - Risk Management,marilou.plando@abc.test
110866,Dugan,Belinda,Escultura,"abc Equity Ventures, Inc.",Legal Management,Vice President,Vice President for Legal Services Management,dixie.dugan@abc.test
221,Montehermoso,Gladys,Enoy,"abc Equity Ventures, Inc.",Accounting,Staff,Senior Accountant,gladys.montehermoso@abc.com
30102,Oblianda,Anna Cielo,Salud,"abc Equity Ventures, Inc.",Accounting,Supervisor,Accounting Supervisor,cielo.oblianda@abc.com
110499,Bucol,Charmaine Ann,Rebusa,"abc Equity Ventures, Inc.",Group Internal Audit,Staff,Audit Senior,charmaine.ann.bucol@abc.com

Я хотел бы подать все строки с одинаковым значением в столбцы EMPLOYEE_NUMBER + EMAIL_ADDRESS, но с другими значениями в других столбцах, используя awk.

Мой идеал - объединить 2 CSV-файла на столбцах EMPLOYEE_NUMBER + EMAIL_ADDRESS по вертикали и удалить дублирующую строку с помощью awk. Спасибо

Вывод будет так:

EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110773,Casas,Joyce Grace,Bea,"Aboitiz Equity Ventures, Inc.",Tax Advisory and Compliance,Manager,Tax Counsel,joyce.grace.casas@aboitiz.com
110773,Casas,Joyce Grace,Bea,"Aboitiz Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Counsel,joyce.grace.casas@aboitiz.com

Answer 1

Использование простого awk_script позволяет добиться этого,

awk_script:

NR==FNR{
  if(FNR==1){print}
  a[$1 $2]=$0
  next
}
a[$1 $2]!=$0 && a[$1 $2]!=""{
  print a[$1 $2],$0
}

Команда для выполнения,

awk -F',' -v OFS="\n" -f awk_script file1 file2