Как извлечь несколько ссылок в одном классе в Python?

Question

Я хочу извлечь все ссылки в одном классе div из следующего кода:

<div class='page-numbers clearfix'><span class='current'>
1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>
2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>
3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>
4</a></div>

Я пробовал:

from bs4 import BeautifulSoup

html="<div class='page-numbers clearfix'><span class='current'>1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>4</a></div>
"

soup = BeautifulSoup(html, "html.parser")
for i in soup.find_all('div', {'class': 'page-numbers clearfix'}):
    link= i.find('a', href=True)
    print(link['href'])

Но, похоже, это не работает. Вывод мне нужен:

https://www.example.com/blog/author/abc/page/2/

https://www.example.com/blog/author/abc/page/3/

https://www.example.com/blog/author/abc/page/4/

Answer 1

Возможное (немного более короткое) отклонение от всех других хороших ответов здесь:

for item in soup.select('div', {'class': 'page-numbers clearfix'}):
   for link in item.select('a', href=True):
       print(link['href'])

Answer 2

Вы также должны использовать find_all при поиске тега a.Приведенный ниже код работает нормально.

from bs4 import BeautifulSoup as bs

stra = """
<div class='page-numbers clearfix'><span class='current'>
1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>
2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>
3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>
4</a></div>
"""
soup = bs(stra, 'html.parser')
for i in soup.find_all('div', {'class': 'page-numbers clearfix'}):
    links = i.find_all('a', href=True)
    for link in links:
        print(link['href'])

Вывод:

https://www.example.com/blog/author/abc/page/2/
https://www.example.com/blog/author/abc/page/3/
https://www.example.com/blog/author/abc/page/4/

Answer 3

Это даст вам список ссылок:

from bs4 import BeautifulSoup

html_doc = '''<div class='page-numbers clearfix'><span class='current'>
1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>
2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>
3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>
4</a></div>'''

soup = BeautifulSoup(html_doc, "lxml")

div = soup.find('div', attrs={'class': 'page-numbers clearfix'})
containers = div.find_all('a', attrs={'class': 'inactive'})
links = [c['href'] for c in containers]

links возвращает:

['https://www.example.com/blog/author/abc/page/2/',
 'https://www.example.com/blog/author/abc/page/3/',
 'https://www.example.com/blog/author/abc/page/4/']

Answer 4

Вы можете использовать селектор CSS:

from bs4 import BeautifulSoup

data = '''<div class='page-numbers clearfix'><span class='current'>
1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>
2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>
3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>
4</a></div>'''

soup = BeautifulSoup(data, 'lxml')

for a in soup.select('div.page-numbers.clearfix a[href]'):
    print(a['href'])

Печать:

https://www.example.com/blog/author/abc/page/2/
https://www.example.com/blog/author/abc/page/3/
https://www.example.com/blog/author/abc/page/4/

Answer 5

Попробуйте следующий код.

data='''<div class='page-numbers clearfix'><span class='current'>
1</span><a href='https://www.example.com/blog/author/abc/page/2/' class='inactive'>
2</a><a href='https://www.example.com/blog/author/abc/page/3/' class='inactive'>
3</a><a href='https://www.example.com/blog/author/abc/page/4/' class='inactive'>
4</a></div>'''


soup=BeautifulSoup(data,'html.parser')

item= soup.find('div', class_="page-numbers clearfix")
for item in item.find_all('a', href=True):
    print(item['href'])

Вывод:

https://www.example.com/blog/author/abc/page/2/
https://www.example.com/blog/author/abc/page/3/
https://www.example.com/blog/author/abc/page/4/