Доступ к HTML-элементам, которые не имеют тега

Question

Я пытаюсь перебрать html с Beautifulsoup, но похоже, что я не могу добраться до всех элементов.Вот оригинальная ссылка (http://china -market-research.blogspot.com / 2018/10 / Why-kid-market-is-booming-in-china.html )

Ihtml вот так:

<div class="post-body entry-content" id="post-body-2820943256231169701" itemprop="description articleBody">
Why Kid Market is Booming in China  ?<br>
<br>
Very simple , look at this video you will get it.<br>
&nbsp;&nbsp; &nbsp;<iframe allow="autoplay; encrypted-media" allowfullscreen="" frameborder="0" height="573" src="https://www.youtube.com/embed/Fg7jIjmLyWs" width="1019"></iframe>

<br>
<br>
<br>
&nbsp;Birth control: a rule, not a pill
China’s two-child policy is having unintended consequences
Reluctant to pay for multiple maternity leaves, companies are choosing not to hire young women

<br>
THE one-child-per-couple policy was horrific for women in China. Many were subjected to forced sterilisations or abortions. Newborn girls were killed, removed by family-planning officials or abandoned by parents desperate that their one permitted baby be a boy. Women from neighbouring countries suffered, too, as victims of human trafficking; a skewed sex-ratio made it more difficult for young men to find Chinese wives. So the government’s announcement in late 2015 that it was relaxing the policy, after 35 years, was good news. Yet the two-child-per-couple policy that replaced it may bring different kinds of problems. source&nbsp;<a href="https://www.economist.com/china/2018/07/26/chinas-two-child-policy-is-having-unintended-consequences">https://www.economist.com/china/2018/07/26/chinas-two-child-policy-is-having-unintended-consequences</a><br>
<blockquote class="tr_bq">
For a generation the government assured women that “one is enough” and that “late marriage and late childbirth are worthy.” Now state media urge them to marry while still in university and remind them that older mothers are more likely to have babies with birth defects, notes Leta Hong Fincher, an author and academic. Officials are encouraging childbirth because they worry that the fertility rate (the number of children a woman can expect to have during her lifetime) has sunk well below 2.1, the level required to keep the population stable in the long term. They fear a shrinking population will hamper economic growth.</blockquote>
<div style="clear: both;"></div>
</div>

И я пытаюсь охватить каждый элемент, включая те, которые не имеют тегов и представляют собой просто текст.Поэтому, когда я задумываюсь, я должен увидеть этот элемент также: «Политика Китая в отношении двух детей имеет непредвиденные последствия. Неохотно оплачивая многочисленные отпуска по беременности и родам, компании предпочитают не нанимать молодых женщин».Вот мой код:

article_soup = BeautifulSoup(article_html, "html.parser")
find_entry_content = article_soup.find('div',class_="post-body entry-content")
for first_parent_tag in find_entry_content.find_all():
            print(first_parent_tag)

А вот результат из кода выше:

<br/>
<br/>
<br/>
<iframe allow="autoplay; encrypted-media" allowfullscreen="" frameborder="0" height="573" src="https://www.youtube.com/embed/Fg7jIjmLyWs" width="1019"></iframe>
<br/>
<br/>
<br/>
<br/>
<a href="https://www.economist.com/china/2018/07/26/chinas-two-child-policy-is-having-unintended-consequences">https://www.economist.com/china/2018/07/26/chinas-two-child-policy-is-having-unintended-consequences</a>
<br/>
<blockquote class="tr_bq">
For a generation the government assured women that “one is enough” and that “late marriage and late childbirth are worthy.” Now state media urge them to marry while still in university and remind them that older mothers are more likely to have babies with birth defects, notes Leta Hong Fincher, an author and academic. Officials are encouraging childbirth because they worry that the fertility rate (the number of children a woman can expect to have during her lifetime) has sunk well below 2.1, the level required to keep the population stable in the long term. They fear a shrinking population will hamper economic growth.</blockquote>
<div style="clear: both;"></div>

Answer 1

find_all () перебирает все теги / потомки в div.Пока текст, который вы ищете, находится в div, он не находится под тегами tag / child.

<div>
"Some text" # <----- This will be skipped because it isn't a HTML (child) tag in the Div. It's right in the div
"Some more text" # <----- This will also be skipped for the same reason.
<br/>
<iframe allow= .... >
<br/>
<br/>
<a href ....>
<br/>
<blockquote class="tr_bq">
For a generation the government assured women that .... </blockquote>
# ^ This text is found because it's in a blockquote ,which find_all() is looping over'
<div style="clear: both;"></div>

Так что вместо того, чтобы перебирать все теги в div, также смотрите в самом div.Он должен содержать все элементы, не входящие в дочерние элементы div.