Я не смог воспроизвести проблему в Ubuntu 16.04 с использованием Python 3.5.
Я изменил скрипт, чтобы закрыть автоматически открытое окно Chrome, используя driver.close()
.
Обновленный код:
from selenium import webdriver
from bs4 import BeautifulSoup
from bs4 import Tag
driver = webdriver.Chrome()
term = "python"
google_url = "https://www.google.com/search?q={}".format(term) + "&num=" + str(5)
driver.get(google_url)
# time.sleep(3)
driver.implicitly_wait(100)
soup = BeautifulSoup(driver.page_source, 'lxml')
result_div = soup.find_all('div', attrs={'class': 'g'})
links = []
titles = []
descriptions = []
for r in result_div:
# Checks if each element is present, else, raise exception
try:
link = r.find('a', href=True)
title = None
title = r.find('h3')
if isinstance(title, Tag):
title = title.get_text()
description = None
description = r.find('span', attrs={'class': 'st'})
if isinstance(description, Tag):
description = description.get_text()
# Check to make sure everything is present before appending
if link != '' and title != '' and description != '':
links.append(link['href'])
titles.append(title)
descriptions.append(description)
# Next loop if one element is not present
except Exception as e:
print(e)
continue
print(links)
print(titles)
print(descriptions)
driver.close()
Установленные пакеты requirements.txt
:
beautifulsoup4==4.7.1
lxml==4.3.4
pkg-resources==0.0.0
selenium==3.141.0
soupsieve==1.9.1
urllib3==1.25.3
Выход:
['https://www.python.org/', 'https://medium.com/@mindfiresolutions.usa/python-7-important-reasons-why-you-should-use-python-5801a98a0d0b', 'https://medium.com/@mindfiresolutions.usa/python-7-important-reasons-why-you-should-use-python-5801a98a0d0b', 'https://www.ics.uci.edu/~pattis/common/handouts/pythoneclipsejava/python.html', 'https://thehelloworldprogram.com/python/why-python-should-be-the-first-programming-language-you-learn/', 'https://www.quora.com/Is-it-possible-to-learn-programming-specially-python-by-my-own-self', 'https://bn.wikipedia.org/wiki/%E0%A6%AA%E0%A6%BE%E0%A6%87%E0%A6%A5%E0%A6%A8_(%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A7%8B%E0%A6%97%E0%A7%8D%E0%A6%B0%E0%A6%BE%E0%A6%AE%E0%A6%BF%E0%A6%82_%E0%A6%AD%E0%A6%BE%E0%A6%B7%E0%A6%BE)', '/search?q=python&num=5&tbm=isch&source=iu&ictx=1&fir=OO5BXHlBkMORMM%253A%252CZIk6oEy_LSc-sM%252C%252Fm%252F05z1_&vet=1&usg=AI4_-kQhGxHuP5STNIQIF-LojlNusowOFg&sa=X&ved=2ahUKEwjB4_vOpujiAhWUe30KHWxxCCoQ_B0wEHoECAEQAw#imgrc=OO5BXHlBkMORMM:']
['Welcome to Python.org', 'Python: 7 Important Reasons Why You Should Use Python - Medium', 'Python: 7 Important Reasons Why You Should Use Python - Medium', 'Python Download and Installation Instructions', 'Why Python Should Be The First Programming Language You Learn ...', 'Is it possible to learn programming specially python by my own ...', 'পাইথন (প্রোগ্রামিং ভাষা) - উইকিপিডিয়া - BN-Wikipedia', 'বিবরণ']
['The official home of the Python Programming Language.', None, None, None, None, None, 'পাইথন (Python) একটি বস্তু-সংশ্লিষ্ট (object-oriented) উচ্চস্তরের প্রোগ্রামিং ... পাইথন একটি বহু-প্যারাডাইম প্রোগ্রামিং ভাষা (ফাংশন-ভিত্তিক, বস্তু-সংশ্লিষ্ট ও\xa0...', None]