bash grep regex как не использовать grep timestamp - PullRequest
0 голосов
/ 05 августа 2020
• 1000 метки времени, хотя я указал начальные конечные буквальные символы.

Не уверен, почему он не работает?

Пример данных:

Jun 22 23:15:09 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:40932 -> 192.168.0.99:80
Jun 22 23:35:46 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:37647 -> 192.168.0.18:80
Jun 25 00:17:41 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:46210 -> 192.168.0.9:80
Jun 25 00:26:30 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:39421 -> 192.168.0.13:80
Jul 31 16:11:52 192.168.0.1 pkg-static: snort reinstalled: 2.9.16 -> 2.9.16
Jul 31 16:11:53 192.168.0.1 snort[89490]: *** Caught Term-Signal
Jul 31 16:11:58 192.168.0.1 snort[90728]: *** Caught Term-Signal
Jul 31 16:12:13 192.168.0.1 php: /etc/rc.packages: Beginning package installation for snort .
Jul 31 16:12:31 192.168.0.1 php: /etc/rc.packages: [Snort] There is a new set of Snort Subscriber rules posted. Downloading snortrules-snapshot-29160.tar.gz...
Jul 31 16:12:36 192.168.0.1 php: /etc/rc.packages: [Snort] There is a new set of Snort OpenAppID detectors posted. Downloading snort-openappid.tar.gz...
Jul 31 16:13:17 192.168.0.1 php: /etc/rc.packages: Successfully installed package: snort.
Jul 31 16:13:17 192.168.0.1 pkg-static: pfSense-pkg-snort upgraded: 3.2.9.13 -> 3.2.9.14_1
Aug  2 10:47:36 192.168.0.1 php-fpm[76321]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  2 10:47:36 192.168.0.1 php-fpm[76321]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  2 10:47:36 192.168.0.1 snort[92683]: Snort Reload: Any change to any output configurations requires a restart.
Aug  2 10:47:59 192.168.0.1 php-fpm[3795]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  2 10:47:59 192.168.0.1 php-fpm[3795]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  2 15:41:03 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:51231 -> 192.168.0.3:445
Aug  3 11:00:08 192.168.0.1 snort[92683]: [1:2030215:2] ET POLICY DNS Query to .onion proxy Domain (onion . ly) [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 192.168.0.2:62288 -> 192.168.0.1:53
Aug  3 11:00:08 192.168.0.1 snort[92683]: [1:2030215:2] ET POLICY DNS Query to .onion proxy Domain (onion . ly) [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 192.168.0.2:62288 -> 192.168.0.1:53
Aug  3 11:00:10 192.168.0.1 snort[92683]: [1:2030216:2] ET POLICY .onion.ly Proxy domain in SNI [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 192.168.0.2:3698 -> 191.168.0.18:443
Aug  3 13:50:24 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 13:50:24 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 13:50:25 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 14:27:36 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025709:2] ET POLICY SMB2 NT Create AndX Request For a DLL File - Possible Lateral Movement [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025699:2] ET POLICY SMB Executable File Transfer [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:15 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:15 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:16 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:49:36 192.168.0.1 php-fpm[349]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  4 10:49:36 192.168.0.1 php-fpm[349]: /snort/snort_alerts.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  4 10:51:38 192.168.0.1 php-fpm[62611]: /snort/snort_rulesets.php: [Snort] Updating rules configuration for: LAN ...
Aug  4 10:51:40 192.168.0.1 php-fpm[62611]: /snort/snort_rulesets.php: [Snort] Enabling any flowbit-required rules for: LAN...
Aug  4 10:51:40 192.168.0.1 php-fpm[62611]: /snort/snort_rulesets.php: [Snort] Building new sid-msg.map file for LAN...
Aug  4 10:51:41 192.168.0.1 php-fpm[62611]: /snort/snort_rulesets.php: [Snort] Snort RELOAD CONFIG for LAN...
Aug  4 23:45:21 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:23 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445

Ожидаемый результат:

Jun 22 23:15:09 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:40932 -> 192.168.0.99:80
Jun 22 23:35:46 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:37647 -> 192.168.0.18:80
Jun 25 00:17:41 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:46210 -> 192.168.0.9:80
Jun 25 00:26:30 192.168.0.1 snort[8791]: [120:28:1] (http_inspect) INVALID CHUNK SIZE OR CHUNK SIZE FOLLOWED BY JUNK CHARACTERS [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.6:39421 -> 192.168.0.13:80
Aug  2 15:41:03 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:51231 -> 192.168.0.3:445
Aug  3 11:00:08 192.168.0.1 snort[92683]: [1:2030215:2] ET POLICY DNS Query to .onion proxy Domain (onion . ly) [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 192.168.0.2:62288 -> 192.168.0.1:53
Aug  3 11:00:08 192.168.0.1 snort[92683]: [1:2030215:2] ET POLICY DNS Query to .onion proxy Domain (onion . ly) [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 192.168.0.2:62288 -> 192.168.0.1:53
Aug  3 11:00:10 192.168.0.1 snort[92683]: [1:2030216:2] ET POLICY .onion.ly Proxy domain in SNI [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 192.168.0.2:3698 -> 191.168.0.18:443
Aug  3 13:50:24 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 13:50:24 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 13:50:25 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  3 14:27:36 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:2746 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025709:2] ET POLICY SMB2 NT Create AndX Request For a DLL File - Possible Lateral Movement [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:14 192.168.0.1 snort[92683]: [1:2025699:2] ET POLICY SMB Executable File Transfer [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:15 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:15 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 10:46:16 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:21 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:22 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445
Aug  4 23:45:23 192.168.0.1 snort[92683]: [1:2025701:2] ET POLICY SMB2 NT Create AndX Request For an Executable File [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.2:6342 -> 192.168.0.3:445

Ответы [ 2 ]

3 голосов
/ 06 августа 2020

Символ + читается в вашем выражении как буквальный +. Используйте параметр -E для соответствия POSIX ERE.

Кроме того, у вас есть бесполезное использование cat, grep также принимает файл в качестве аргумента.

grep -vE '\[[0-9]+:[0-9]+:[0-9]+\]' example.txt
2 голосов
/ 06 августа 2020

Стандартный grep не обрабатывает + как квантор; используйте вместо него \+:

grep -v '\[[0-9]\+:[0-9]\+:[0-9]\+\]' example.txt

Или укажите, что вам нужен вкус регулярного выражения, которое распознает простой + как квантификатор с параметром -E, как в ответе @Ryszard Czech, или с помощью команда egrep вместо grep.

Просто помните о других вещах, которые меняются при переключении разновидностей регулярных выражений. Например, в grep -E или egrep вы формируете группы захвата с обычными круглыми скобками и должны использовать обратную косую черту для соответствия буквальным, в отличие от grep без -E.

Кроме того, это Хорошая идея поместить ваши grep шаблоны - и все, что вы хотите передать команде, буквально без каких-либо изменений оболочки - в одинарных кавычках вместо двойных кавычек. В большинстве случаев это не имеет значения, но иногда имеет значение.

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...