Как разобрать в определенном разделе с BeautifulSoup

Question

Я хотел бы проанализировать определенный фрагмент кода HTML с BeautifulSoup.

Я могу получить фрагмент кода HTML, который я хочу проанализировать в section_I_want_to_parse.

MWE:

#!/usr/bin/python3

import requests
from bs4 import BeautifulSoup

url = 'https://www.shopbot.com.au/pp-dji-mavic-mini-fly-more-bundle-price-564515.html'
header = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"}

page = requests.get(url, headers=header)

soup = BeautifulSoup(page.content, 'html.parser')
section_I_want_to_parse = soup.find("section", {"class": "sb-compare-prices", "id": "compare-prices"})
print(section_I_want_to_parse)  # This works - so far so good

# I want to use findAll within the section_I_want_to_parse
# price_list = section_I_want_to_parse.findAll("span", {"class": "sb-discounted-price"})

Я хочу findAll("span", {"class": "sb-discounted-price"}) в пределах section_I_want_to_parse.

Как я могу это сделать?

Answer 1

Решения с использованием упрощения c

import requests
from simplified_scrapy.simplified_doc import SimplifiedDoc 

url = 'https://www.shopbot.com.au/pp-dji-mavic-mini-fly-more-bundle-price-564515.html'
header = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"}

page = requests.get(url, headers=header)

doc = SimplifiedDoc(page.text)
section_I_want_to_parse = doc.getElement("section", attr="id", value="compare-prices")
# print(section_I_want_to_parse)  # This works - so far so good
price_list = section_I_want_to_parse.getElements("span",attr="class",value="sb-discounted-price")

print ([price.text for price in price_list])

результат ：

['$799.00', '$799.00', '$799.00', '$799.00', '$799.00', '$799.00', '$799.00', '$799.00', '$958.80']