Ошибка ValueError: Таблицы не найдены

Question

Я пытаюсь получить статистику списка игроков на stats.nba.com и получаю ValueError: No tables found при просмотре URL-адресов игроков. Ошибка иногда возникает на 3-м URL, иногда на 8-м URL, затем на 5-м URL и т. Д ...

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd

driver = webdriver.Chrome()
driver.implicitly_wait(30)

player_boxscores_traditional_url_list = [
    'https://stats.nba.com/player/203500/boxscores-traditional/',
    'https://stats.nba.com/player/1628389/boxscores-traditional/',
    'https://stats.nba.com/player/1629061/boxscores-traditional/',
    'https://stats.nba.com/player/1629152/boxscores-traditional/',
    'https://stats.nba.com/player/200746/boxscores-traditional/',
    'https://stats.nba.com/player/1628959/boxscores-traditional/',
    'https://stats.nba.com/player/1628960/boxscores-traditional/',
    'https://stats.nba.com/player/1628386/boxscores-traditional/',
    'https://stats.nba.com/player/1628443/boxscores-traditional/',
    'https://stats.nba.com/player/202329/boxscores-traditional/',
    'https://stats.nba.com/player/1626147/boxscores-traditional/'
]

player_stats = []

for player_url in player_boxscores_traditional_url_list:
    driver.get(player_url)
    soup = BeautifulSoup(driver.page_source, 'html.parser')
    table = soup.find('table')
    dfs = pd.read_html(str(table))
    df = dfs[0]
    player_stats.append(df)

driver.quit()

Answer 1

Вы должны дождаться загрузки элемента table на странице. Я добавил WebDriverWait в скрипт, так что этот работает для меня:

import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()

ids = [203500, 1628389, 1629061, 1629152, 200746, 1628959, 1628960, 1628386, 1628443, 202329, 1626147]
player_stats = []
for player_id in ids:
    driver.get('https://stats.nba.com/player/{}/boxscores-traditional/'.format(player_id))
    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'nba-stat-table')))
    soup = BeautifulSoup(driver.page_source, 'html.parser')
    table = soup.find('table')
    dfs = pd.read_html(str(table))
    player_stats.append(dfs[0])

driver.quit()