Как извлечь текст из результатов поиска duckduckgo с помощью Selenium Python

Question

Я пытаюсь получить описание ссылок на утку из результатов поиска, используя следующее:

results=browser.find_elements_by_xpath("//div[@id='links']/div/div/div[2]")
description=[]
for result in results:
  description.append(result.text)

Я получаю сообщение об ошибке «У списка» нет атрибута «текст». Мне удалось использовать аналогичный метод для получения заголовков результатов поиска, но по какой-то причине я не могу извлечь текст из этого конкретного xpath.

Answer 1

Чтобы извлечь описание ссылок результатов поиска из DuckDuckGo, вам нужно WebDriverWait для видимости всех элементов, расположенных , и вы можете использовать следующее решение:

• Блок кода:

  from selenium import webdriver
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.by import By
  
  options = webdriver.ChromeOptions()
  options.add_argument("start-maximized")
  options.add_argument("disable-infobars")
  options.add_argument("--disable-extensions")
  driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
  driver.get('https://duckduckgo.com/')
  search_box = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.NAME, "q")))
  search_box.send_keys("Selenium")
  search_box.submit()
  elements = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@id='links']/div/div/div[2]")))
  for ele in elements:
      print(ele.text)
  driver.quit()
  

• Консольный вывод:

  What is Selenium? Selenium automates browsers.That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that.
  Selenium is a mineral found in the soil. Selenium naturally appears in water and some foods. While people only need a very small amount, selenium plays a key role in the metabolism.. Why do people ...
  Selenium is a chemical element with symbol Se and atomic number 34. It is a nonmetal (more rarely considered a metalloid) with properties that are intermediate between the elements above and below in the periodic table, sulfur and tellurium, and also has similarities to arsenic.
  Selenium is a trace mineral found naturally in the soil that also appears in certain high-selenium foods, and there are even small amounts in water.. Selenium is an extremely vital mineral for the human body as it increases immunity, takes part in antioxidant activity that defends against free radical damage and inflammation, and plays a key role in maintaining a healthy metabolism.
  Introduction. Selenium is a trace element that is naturally present in many foods, added to others, and available as a dietary supplement. Selenium, which is nutritionally essential for humans, is a constituent of more than two dozen selenoproteins that play critical roles in reproduction, thyroid hormone metabolism, DNA synthesis, and protection from oxidative damage and infection [].
  Selenium is an essential trace mineral that is important for many bodily processes, including cognitive function, a healthy immune system, and fertility in both men and women.
  Your body relies on selenium, an important mineral, for many of its basic functions, from reproduction to fighting infection. The amount of selenium in different foods depends on the amount of ...
  Overview Information Selenium is a mineral. It is taken into the body in water and foods. People use it for medicine. Most of the selenium in the body comes from the diet. The amount of selenium ...
  Selenium WebDriver. The biggest change in Selenium recently has been the inclusion of the WebDriver API. Driving a browser natively as a user would either locally or on a remote machine using the Selenium Server it marks a leap forward in terms of browser automation.
  Downloads. Below is where you can find the latest releases of all the Selenium components. You can also find a list of previous releases, source code, and additional information for Maven users (Maven is a popular Java build tool).
  

Answer 2

Вам не нужно создавать цикл for для пустого списка ... попробуйте использовать этот код:

results=driver.find_elements_by_xpath("//div[@id='links']/div/div/div[2]")

description=[]

for result in results:
    description.append(result.text)

Пример: Чтобы проверить это, я просто набрал 'hmm' в DuckDuckGo, чтобы URLis https://duckduckgo.com/?q=hmm&t=h_&ia=web

from selenium import webdriver

driver=webdriver.Chrome()
driver.get('https://duckduckgo.com/?q=hmm&t=h_&ia=web')

results=driver.find_elements_by_xpath("//div[@id='links']/div/div/div[2]")

description=[]

for result in results:
    description.append(result.text)

print(description[0])
print(' ')
print(description[1])
print(' ')
print(description[2])

Вывод:

HMM для разработки "New-GAUS 2020" ... HMM удерживает PSA-Hyundai Pusan ​​N ... HMM Новые именаVLCC, «Универсальный ... Новогоднее послание 2019 года»;Будущий план HMM;План HMM на 202 год ... HMM подписал официальный контракт ...

Определение Хмм (обычно используется для выражения вдумчивого поглощения, колебаний, сомнений или растерянности.) См. Подробнее.

2 � раньше подчеркивал, что кто-то задал вопрос и ждет ответа. Но скажите Санте правду сейчас, что является самой важной частью для маленького мальчика или девочки?Поле

Результаты поиска: