исправление производительности: join -t, -j99 file2 file1 | awk -F, -v OFS =, '$ 3 == $ 6 && $ 4 == $ 8 {print $ 2, $ 3, $ 4, $ 5, $ 7, $ 9}'

Question

File1

111,222,560,0.7
111,333,560,0.2
111,444,560,0.1

File2

2017,111,560,0.0537
2018,111,560,0.0296
2019,111,560,0.0624

Выход:

2017,111,560,0.0537,222,0.7
2017,111,560,0.0537,333,0.2
2017,111,560,0.0537,444,0.1
2018,111,560,0.0296,222,0.7
2018,111,560,0.0296,333,0.2
2018,111,560,0.0296,444,0.1
2019,111,560,0.0624,222,0.7
2019,111,560,0.0624,333,0.2
2019,111,560,0.0624,444,0.1

для файла1 у меня есть 31 000 строк для файла2 у меня 1,3 миллиона строк

и это заняло очень много времени для обработки файла. эта команда выполняется в Unix

Может кто-нибудь помочь? заранее спасибо

Answer 1

еще awk/join

$ join -t, <(awk -F, '{print $2"_"$3 FS $0}' f2 | sort) \
           <(awk -F, '{print $1"_"$3 FS $2 FS $4}' f1 | sort) | cut -d, -f2-

2017,111,560,0.0537,222,0.7
2017,111,560,0.0537,333,0.2
2017,111,560,0.0537,444,0.1
2018,111,560,0.0296,222,0.7
2018,111,560,0.0296,333,0.2
2018,111,560,0.0296,444,0.1
2019,111,560,0.0624,222,0.7
2019,111,560,0.0624,333,0.2
2019,111,560,0.0624,444,0.1

Answer 2

Вы можете попробовать это

 join -t, -1 2 -2 1 file2 file1  | awk -F, -v OFS=, '{print $2, $1, $3, $4, $5, $7}'

Результат тот же, но он должен быть быстрее.