Получение всей информации о ячейке с помощью Beautiful Soup / Web Scraping

Question

Я пытаюсь выкинуть из сети эту таблицу статистики оружия из игры Final Fantasy XIII (https://finalfantasy.fandom.com/wiki/Final_Fantasy_XIII_weapons). Как я могу внести в список все содержимое каждой ячейки? Мой код похож на:

summary = soup.find('div',{'class':'mw-content-ltr mw-content-text','id':'mw-content-text'})
tables = summary.find_all('table')
data = []
rows = tables[0].findAll('tr')

for tr in rows:
    cols = tr.findAll('td')

    for td in cols:
        text = td.find(text=True)
        print(text)
        data.append(text)
name = []
rank = []
strenght = []
magic = []
exp = []
acquirement = []
dismantle = []

index = 0

for item in data:
    if 'EXP' in item:
        dismantle.append(data[index+2])
        acquirement.append(data[index+1])
        exp.append(data[index])
        magic.append(data[index-3])
        strenght.append(data[index-4])
        rank.append(data[index-5])
        name.append(data[index-7])
    index += 1

отлично работает для ячеек, в которых есть единственные данные внутри <td>, но для записей, в которых <td имеет несколько тегов, форматирующих содержимое, он захватывает только первый фрагмент данных в ячейке.

Например, в strenght[0] полученное значение равно 15, когда оно должно быть 15 (Min) 115 (Max)

Answer 1

Вы также можете использовать Pandas для получения таблицы, а затем, при необходимости, работать с фреймом данных, чтобы получить нужные вам данные.

import pandas as pd

url = 'https://finalfantasy.fandom.com/wiki/Final_Fantasy_XIII_weapons'

tables = pd.read_html(url)
table = tables[0]

print (table.to_string())

Выход:

print (table.to_string())
                                                    0     1                 2                 3                             4                 5              6                                                  7                                                  8
0                                           NameLevel  Rank          Strength             Magic               Passive Ability   Synthesis Group      EXP for ★                                        Acquirement                                          Dismantle
1                              Blazefire SaberLv.1-26     3   15(Min)115(Max)   15(Min)115(Max)                             —  Physical Defense     24,600 EXP       Buy: Up in Arms(2,000 gil)Initial: Lightning           Level ★: Spark Plug (x13), Tungsten Tube
2   A weapon new to the Sanctum army, this gunblad...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
3                                   FlambergeLv.10-61     5   68(Min)323(Max)   68(Min)323(Max)                             —  Physical Defense    449,310 EXP       Catalyst: Perovskite on Blazefire Saber Lv.★  Level ★: Iridium Plug (x18), Electrolytic Capa...
4   The barrel and frame of this weapon have been ...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
5                               Omega WeaponLv.47-100    11  302(Min)620(Max)  302(Min)620(Max)                        ATB +1       Independent  1,308,252 EXP          Catalyst: Trapezohedron on Flamberge Lv.★  Level ★: Spark Plug (x71), Tesla Turbine (x35)...
6   This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
7                                   Axis BladeLv.1-21     5     8(Min)48(Max)     8(Min)48(Max)            Attack: ATB Charge             Boost     66,750 EXP  Buy: Plautus's Workshop(15,000 gil)Find: Palum...               Level ★: Spark Plug (x7), Perovskite
8   Designed for ease of movement in the field, th...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
9                                   EnkindlerLv.13-41     9   49(Min)133(Max)   49(Min)133(Max)         Attack: ATB Charge II             Boost    287,154 EXP            Catalyst: Adamantite on Axis Blade Lv.★  Level ★: Iridium Plug (x12), Cobaltite, Axis B...
10  Remodeled to suit the mobility needs of specia...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
11                              Omega WeaponLv.39-100    11  132(Min)315(Max)  132(Min)315(Max)  Attack: ATB Charge II,ATB +1       Independent  1,433,256 EXP          Catalyst: Trapezohedron on Enkindler Lv.★  Level ★: Iridium Plug (x51), Mobius Coil (x45)...
12  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
13                               Edged CarbineLv.1-26     3     8(Min)83(Max)   20(Min)170(Max)                             —     Magic Defense     33,000 EXP    Buy: Up in Arms(4,600 gil)Find: Gapra Whitewood          Level ★: Polymer Emulsion (x13), Actuator
14  This gunblade was developed as part of a proje...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
15                              Razor CarbineLv.13-61     5   46(Min)190(Max)  112(Min)448(Max)                             —     Magic Defense    438,360 EXP         Catalyst: Perovskite on Edged Carbine Lv.★  Level ★: Ferroelectric Film (x18), Liquid Crys...
16  A product of research into magically adept bio...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
17                              Omega WeaponLv.47-100    11  104(Min)210(Max)  352(Min)723(Max)                        ATB +1       Independent  1,308,252 EXP      Catalyst: Trapezohedron on Razor Carbine Lv.★  Level ★: Polymer Emulsion (x71), Crystal Oscil...
18  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
19                                   LifesaberLv.1-21     5     8(Min)88(Max)     8(Min)88(Max)                Improved Raise            Low HP     70,850 EXP  Buy: Plautus's Workshop(20,000 gil)Find: Palam...         Level ★: Polymer Emulsion (x7), Perovskite
20  An officer's gunblade. Boosts HP restored to a...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
21                                 PeacemakerLv.16-41     8  100(Min)225(Max)  100(Min)225(Max)             Improved Raise II            Low HP    237,000 EXP             Catalyst: Scarletite on Lifesaber Lv.★  Level ★: Ferroelectric Film (x12), Perovskite,...
22  A commander's gunblade. Boosts HP restored to ...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
23                              Omega WeaponLv.37-100    11  205(Min)520(Max)  205(Min)520(Max)      Improved Raise II,ATB +1       Independent  1,461,537 EXP         Catalyst: Trapezohedron on Peacemaker Lv.★  Level ★: Ferroelectric Film (x51), Superconduc...
24  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
25                                     GladiusLv.1-26     4   25(Min)175(Max)    13(Min)88(Max)                             —  Physical Defense     54,800 EXP      Buy: Up in Arms(7,100 gil)Find: Pulse Vestige              Level ★: Spark Plug (x13), Carburetor
26  The design of this gunblade is based on that o...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
27                             Helter-SkelterLv.19-61     6  154(Min)448(Max)   64(Min)190(Max)                             —  Physical Defense    334,656 EXP                Catalyst: Uraninite on Gladius Lv.★  Level ★: Iridium Plug (x18), Transformer (x12)...
28  Reverse engineered from an ancient pulse weapo...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
29                              Omega WeaponLv.42-100    11  317(Min)723(Max)   94(Min)210(Max)                        ATB +1       Independent  1,388,607 EXP     Catalyst: Trapezohedron on Helter-Skelter Lv.★  Level ★: Spark Plug (x71), Supercharger (x35),...
30  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
31                                     OrganyxLv.1-26     2   37(Min)187(Max)   37(Min)187(Max)                  Leadenstrike          Adamancy     20,100 EXP                  Buy: Gilgamesh, Inc.(280,000 gil)               Level ★: Spark Plug (x13), Millerite
32  Manufactured for anti-Pulse operations, this p...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
33                                 ApocalypseLv.14-61     4  128(Min)410(Max)  128(Min)410(Max)                    Ironstrike          Adamancy    233,872 EXP                Catalyst: Cobaltite on Organyx Lv.★  Level ★: Iridium Plug (x18), Millerite (x2), O...
34  Sacrificing mobility for power, this imposing ...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
35                              Omega WeaponLv.33-100    11  281(Min)750(Max)  281(Min)750(Max)            Ironstrike, ATB +1       Independent  1,514,535 EXP         Catalyst: Trapezohedron on Apocalypse Lv.★  Level ★: Iridium Plug (x61), Credit Chip (x56)...
36  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
37                                 HauteclaireLv.1-26     4   13(Min)138(Max)   13(Min)138(Max)                  Stagger Lock           Gestalt     54,800 EXP  Buy: Gilgamesh, Inc.(20,000 gil)Find: Mah'haba...            Level ★: Ceramic Armor (x13), Cobaltite
38  Merely a replica of an ancient weapon, this gu...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
39                                   DurandalLv.19-61     6  126(Min)378(Max)  126(Min)378(Max)                  Stagger Lock           Gestalt    334,656 EXP            Catalyst: Uraninite on Hauteclaire Lv.★  Level ★: Chobham Armor (x18), Cobaltite (x2), ...
40  A symbol of honor, the ceremonial blade is bes...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
41                              Omega WeaponLv.42-100    11  305(Min)711(Max)  305(Min)711(Max)          Stagger Lock, ATB +1       Independent  1,388,607 EXP           Catalyst: Trapezohedron on Durandal Lv.★  Level ★: Ceramic Armor (x61), Credit Chip (x56...
42  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
43                                   LionheartLv.1-21     6     8(Min)68(Max)     8(Min)68(Max)                 Quick Stagger           Gestalt     63,060 EXP      Buy: Plautus's Workshop(28,000 gil)Find: Eden             Level ★: Ceramic Armor (x7), Uraninite
44  Once the weapon of a l'Cie, this gunblade allo...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
45                              Ultima WeaponLv.15-41     8   83(Min)213(Max)   83(Min)213(Max)                 Quick Stagger           Gestalt    227,448 EXP             Catalyst: Scarletite on Lionheart Lv.★  Level ★: Chobham Armor (x12), Uraninite, Lionh...
46  Used to strike down weakened foes, this gunbla...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN
47                              Omega WeaponLv.35-100    11  183(Min)508(Max)  183(Min)508(Max)          Quick Stagger,ATB +1       Independent  1,488,630 EXP      Catalyst: Trapezohedron on Ultima Weapon Lv.★  Level ★: Chobham Armor (x51), Electrode (x45),...
48  This upgraded gunblade maintains the propertie...   NaN               NaN               NaN                           NaN               NaN            NaN                                                NaN                                                NaN

Answer 2

Вы должны изменить строку:

text = td.find(text=True)

на

text = td.get_text()

, и вы получите весь текст внутри ячейки, например 15 (Min) 115 (Max) вместо 15