Извлечение текста с разрывом строки в BeautifulSoup

Question

Я бы хотел извлечь текст с разрывом строки вместе с тегом "br" с BeautifulSoup.

html = "<td class="s4 softmerge" dir="ltr"><div class="softmerge-inner" style="width: 5524px; left: -1px;">But when he saw many of the Pharisees and Sadducees come to his baptism, he said unto them, <br/>O generation of vipers, who hath warned you to flee from the wrath to come?<br/>Bring forth therefore fruits meet for repentance:<br/>And think not to say within yourselves, We have Abraham to our father: for I say unto you, that God is able of these stones to raise up children unto Abraham.<br/>And now also the axe is laid unto the root of the trees: therefore every tree which bringeth not forth good fruit is hewn down, and cast into the fire.<br/>I indeed baptize you with water unto repentance. but he that cometh after me is mightier than I, whose shoes I am not worthy to bear: he shall baptize you with the Holy Ghost, and with fire:<br/>Whose fan is in his hand, and he will throughly purge his floor, and gather his wheat into the garner; but he will burn up the chaff with unquenchable fire.</div></td>"

Я хочу получить такой результат в строке;

But when he saw many of the Pharisees and Sadducees come to his baptism, he said unto them,
O generation of vipers, who hath warned you to flee from the wrath to come?
Bring forth therefore fruits meet for repentance:
And think not to say within yourselves, We have Abraham to our father: for I say unto you, that God is able of these stones to raise up children unto Abraham.
And now also the axe is laid unto the root of the trees: therefore every tree which bringeth not forth good fruit is hewn down, and cast into the fire.
I indeed baptize you with water unto repentance. but he that cometh after me is mightier than I, whose shoes I am not worthy to bear: he shall baptize you with the Holy Ghost, and with fire:
Whose fan is in his hand, and he will throughly purge his floor, and gather his wheat into the garner; but he will burn up the chaff with unquenchable fire.

Как мне написать код, чтобы получить этот результат?

Answer 1

Есть два способа получить результат

• Соответствует каждой строке внутри тега,
• посмотреть, принадлежит ли оно NavigableString

код

soup = BeautifulSoup(html,"lxml")
for ele in soup.find("div",class_="softmerge-inner"):
    if isinstance(ele,NavigableString):
        print(ele)
print()

result = [ele[1] for ele in re.findall(r"""(<div.*?>|<br.>)(.*?)(?=<\w{1,4}/>|</\w{1,4}>)""",html)]
for e in result:
    print(e)

Answer 2

Вы можете попробовать это

html = '''<p>Hi</p>
<p>how are you </p>
'''
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, 'html5lib')

print(soup.getText())

Answer 3

Извините, если это не то, что вы ищете, но вы можете попробовать replace или regex.

Например, вы можете использовать регулярные выражения, создав фильтр, который находит все теги <br> и заменяет их символами новой строки (\n).

Если вы используете объект BeautifulSoup, я считаю, что вам нужно использовать его атрибут string: html = soupelement.string.

import re
regex = re.compile(r"<br/?>", re.IGNORECASE) # the filter, it finds <br> tags that may or may not have slashes
html = 'blah blah b<br>lah <br/> bl<br/>' 
newtext = re.sub(regex, '\n', html) # replaces matches with the newline
print(newtext)
# Returns 'blah blah b\nlah \n bl\n' !