получить таблицу, которая находится в ряду другой видимой таблицы, используя BeautifulSoup

Question

Таблица, которую я ищу, кажется, находится в строке внутри таблицы, и я не могу понять, как ее получить.

URL:

http://www.irishstatutebook.ie/eli/2019/si/497/made/en/print?q=Electricity+Regulation+Act+1999&years=2019

Я изо всех сил пытаюсь научиться использовать BeautifulSoup, но, похоже, нет примеров, которые сталкиваются с этой проблемой!

Таблица, которую я пытаюсь извлечь, является первой таблицей, в которой заголовок первой строки равен "(1) Ref No."

Answer 1

import requests
from simplified_scrapy.simplified_doc import SimplifiedDoc

url = 'http://www.irishstatutebook.ie/eli/2019/si/497/made/en/print?q=Electricity+Regulation+Act+1999&years=2019'
html = requests.get(url).text
doc = SimplifiedDoc(html)
table = doc.getElement('table',attr='class',value='t1')
trs = table.trs
for tr in trs:
  print ([td.text for td in tr.tds])

Результат:

['(1) Ref. No.', '(2) REFIT Power Purchase Agreement Reference', '(3) Supplier', '(4) Electricity generator', '(5) Generation plant Description', '(6) Energy source of electricity', '(7) Capacity accepted into REFIT (MW)']
['1', 'REFIT 1/5/1', 'Bord Gáis Energy Limited', 'Jaroma Windfarm Ltd.', 'Toomyvara, Ballinlough, Co. Tipperary', 'Small Scale Wind Project', '2.550']
['2', 'REFIT 1/5/2', 'Viridian Energy Limited*', 'Reirk Energy Ltd.', 'Dromdeeveen, Ballagh, Co. Limerick', 'Large Scale Wind Project', '10.500']
['3', 'REFIT 1/5/4', 'Bord Gáis Energy Limited', 'Adeery Hydro', 'Killybegs, Co. Donegal', 'Hydro Project', '0.400']
['4', 'REFIT 1/5/5', 'Viridian Energy Limited*', 'Bawnmore Windfarm Ltd', 'Carriganima, Macroom, Co. Cork', 'Large Scale Wind Project', '14.990']
['5', 'REFIT 1/5/6', 'Viridian Energy Limited*', 'Muingnaminnane Windfarms Ltd.', 'Muingnaminnane, Kielduff, Tralee, Co. Kerry', 'Large Scale Wind Project', '15.300']
['6', 'REFIT 1/5/8', 'ESBIE t/a Electric Ireland Ltd', 'Mountainlodge Power Ltd.', 'Mountain Lodge Co. Cavan', 'Large Scale Wind Project', '24.800']
['7', 'REFIT 1/5/9', 'Viridian Energy Limited*', 'Tra Investments Ltd.', 'Ballincollig Hill and Maghanknockane, Tralee, Co. Kerry', 'Large Scale Wind Project', '13.300']
['8', 'REFIT 1/5/20', 'SSE Airtricity Limited', 'Carrig Wind Farm Ltd.', 'Lacka, Carrig, Birr, Co. Offaly', 'Small Scale Wind Project', '2.550']
['9', 'REFIT 1/5/23', 'Viridian Energy Limited*', 'Pallas Windfarm Ltd', 'Pallas, Banemore, Listowel, Co. Kerry', 'Large Scale Wind Project', '37.850']
... and so on.

Примеры SimplifiedDo c можно найти здесь

Answer 2

Вы можете использовать этот скрипт, чтобы получить данные из первой таблицы (и распечатать их на экране):

import requests
from bs4 import BeautifulSoup

url = 'http://www.irishstatutebook.ie/eli/2019/si/497/made/en/print?q=Electricity+Regulation+Act+1999&years=2019'

soup = BeautifulSoup(requests.get(url).content, 'html.parser')

data = []
table = soup.select_one('table.t1') # <-- select first table

for tr in table.select('tr'):
    data.append([*map(lambda td: td.get_text(strip=True), tr.select('td'))])

# print the table:
from textwrap import shorten
for row in data:
    print(('{:<10}' + '{:<33}' * 6).format(*[shorten(data, 33) for data in row]))

Отпечатки:

(1) Ref. No.(2) REFIT Power Purchase [...]   (3) Supplier                     (4) Electricity generator        (5) Generation plant Description (6) Energy source of electricity (7) Capacity accepted into [...] 
1         REFIT 1/5/1                      Bord Gáis Energy Limited         Jaroma Windfarm Ltd.             Toomyvara, Ballinlough, Co. [...]Small Scale Wind Project         2.550                            
2         REFIT 1/5/2                      Viridian Energy Limited*         Reirk Energy Ltd.                Dromdeeveen, Ballagh, Co. [...]  Large Scale Wind Project         10.500                           
3         REFIT 1/5/4                      Bord Gáis Energy Limited         Adeery Hydro                     Killybegs, Co. Donegal           Hydro Project                    0.400                            
4         REFIT 1/5/5                      Viridian Energy Limited*         Bawnmore Windfarm Ltd            Carriganima, Macroom, Co. Cork   Large Scale Wind Project         14.990                           
5         REFIT 1/5/6                      Viridian Energy Limited*         Muingnaminnane Windfarms Ltd.    Muingnaminnane, Kielduff, [...]  Large Scale Wind Project         15.300                           
6         REFIT 1/5/8                      ESBIE t/a Electric Ireland Ltd   Mountainlodge Power Ltd.         Mountain Lodge Co. Cavan         Large Scale Wind Project         24.800                           
7         REFIT 1/5/9                      Viridian Energy Limited*         Tra Investments Ltd.             Ballincollig Hill and [...]      Large Scale Wind Project         13.300                           
8         REFIT 1/5/20                     SSE Airtricity Limited           Carrig Wind Farm Ltd.            Lacka, Carrig, Birr, Co. Offaly  Small Scale Wind Project         2.550                            

...and so on.