Я нашел много информации об этом, но я не смог получить wget для исключения доменов и расширений файлов.
У меня есть файл .txt, в котором много URL. Я хочу не загружать изображения (jpg, png, gif) определенных доменов, а также избегать загрузки файлов html или ссылок.
Используя следующую команду, я загрузил все в файл file.txt
wget -i file.txt
В файле у меня есть следующие URL
https://feedly.com/
http://img2.rtve.es/v/3195388?w=1600&preview=1435846554460.jpg
https://images.vexels.com/media/users/3/127855/isolated/preview/c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png
https://upload.wikimedia.org/wikipedia/commons/2/2c/Rotating_earth_%28large%29.gif
Чтобы исключить домены, я пытался wget -i file.txt --exclude-domains img2.rtve.es
. Результат без ошибок
wget -i file.txt --exclude-domains img2.rtve.es
--2018-05-18 16:29:54-- https://feedly.com/
Resolving feedly.com... 104.20.60.241, 104.20.59.241
Connecting to feedly.com|104.20.60.241|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’
index.html [ <=> ] 15.45K --.-KB/s in 0.03s
2018-05-18 16:29:55 (616 KB/s) - ‘index.html’ saved [15821]
--2018-05-18 16:29:55-- http://img2.rtve.es/v/3195388?w=1600&preview=1435846554460.jpg
Resolving img2.rtve.es... 8.252.16.124, 8.253.165.245, 8.253.48.245
Connecting to img2.rtve.es|8.252.16.124|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87358 (85K) [image/jpeg]
Saving to: ‘3195388?w=1600&preview=1435846554460.jpg’
3195388?w=1600&prev 100%[===================>] 85.31K 552KB/s in 0.2s
2018-05-18 16:29:56 (552 KB/s) - ‘3195388?w=1600&preview=1435846554460.jpg’ saved [87358/87358]
--2018-05-18 16:29:56-- https://images.vexels.com/media/users/3/127855/isolated/preview/c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png
Resolving images.vexels.com... 177.54.152.45
Connecting to images.vexels.com|177.54.152.45|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9957 (9.7K) [image/png]
Saving to: ‘c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png’
c3f01cf799e4c8714a8 100%[===================>] 9.72K --.-KB/s in 0s
2018-05-18 16:29:56 (69.8 MB/s) - ‘c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png’ saved [9957/9957]
--2018-05-18 16:29:56-- https://upload.wikimedia.org/wikipedia/commons/2/2c/Rotating_earth_%28large%29.gif
Resolving upload.wikimedia.org... 208.80.154.240
Connecting to upload.wikimedia.org|208.80.154.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1429302 (1.4M) [image/gif]
Saving to: ‘Rotating_earth_(large).gif’
Rotating_earth_(lar 100%[===================>] 1.36M 1.00MB/s in 1.4s
2018-05-18 16:29:58 (1.00 MB/s) - ‘Rotating_earth_(large).gif’ saved [1429302/1429302]
FINISHED --2018-05-18 16:29:58--
Total wall clock time: 4.1s
Downloaded: 4 files, 1.5M in 1.5s (978 KB/s)
И исключить расширения wget -i file.txt --reject gif
. Результат без ошибок
MacBook-Pro:test tomillo$ wget -i file.txt --reject gif
--2018-05-18 16:34:28-- https://feedly.com/
Resolving feedly.com... 104.20.59.241, 104.20.60.241
Connecting to feedly.com|104.20.59.241|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’
index.html [ <=> ] 15.45K --.-KB/s in 0.04s
2018-05-18 16:34:30 (429 KB/s) - ‘index.html’ saved [15821]
--2018-05-18 16:34:30-- http://img2.rtve.es/v/3195388?w=1600&preview=1435846554460.jpg
Resolving img2.rtve.es... 8.252.16.124, 8.253.165.245, 8.253.149.117
Connecting to img2.rtve.es|8.252.16.124|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87358 (85K) [image/jpeg]
Saving to: ‘3195388?w=1600&preview=1435846554460.jpg’
3195388?w=1600&prev 100%[===================>] 85.31K 566KB/s in 0.2s
2018-05-18 16:34:30 (566 KB/s) - ‘3195388?w=1600&preview=1435846554460.jpg’ saved [87358/87358]
--2018-05-18 16:34:30-- https://images.vexels.com/media/users/3/127855/isolated/preview/c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png
Resolving images.vexels.com... 177.54.152.175
Connecting to images.vexels.com|177.54.152.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9957 (9.7K) [image/png]
Saving to: ‘c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png’
c3f01cf799e4c8714a8 100%[===================>] 9.72K --.-KB/s in 0s
2018-05-18 16:34:30 (74.2 MB/s) - ‘c3f01cf799e4c8714a815fac05820bea-reloj-despertador-plana-verde-by-vexels.png’ saved [9957/9957]
--2018-05-18 16:34:30-- https://upload.wikimedia.org/wikipedia/commons/2/2c/Rotating_earth_%28large%29.gif
Resolving upload.wikimedia.org... 208.80.154.240
Connecting to upload.wikimedia.org|208.80.154.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1429302 (1.4M) [image/gif]
Saving to: ‘Rotating_earth_(large).gif’
Rotating_earth_(lar 100%[===================>] 1.36M 1024KB/s in 1.4s
2018-05-18 16:34:32 (1024 KB/s) - ‘Rotating_earth_(large).gif’ saved [1429302/1429302]
FINISHED --2018-05-18 16:34:32--
Total wall clock time: 3.9s
Downloaded: 4 files, 1.5M in 1.6s (972 KB/s)
Где проблема?