Мой код пытается получить только текст статьи из каждого URL-адреса, однако он не может получить каждый p в статье для каждого URL-адреса. Что заставляет его не сканировать их?
import requests from bs4 import BeautifulSoup import json import urllib with open('mn_en.json') as f: d = json.load(f) for k, v in d.items(): if v is not None: #Mongolian soup = BeautifulSoup(requests.get(k).content, 'html.parser') div = soup.findAll('div', attrs={"class":"entry-content"}) for x in div: print(k) print(x.find('p').text)
Вот что я получаю (например, для этого URL https://mn.usembassy.gov/mn/pr-050120-mn/)
АНУ-ын Элчин сайдын яамны Олон нийттэй харилцах алба paspress@usembassy.mn, 7007-6249 2020 оны 5 дугаар сарын 1
Но это должно быть
АНУ-ын Элчин сайдын яамны Олон нийттэй харилцах алба paspress@usembassy.mn, 7007-6249 2020 оны 5 дугаар сарын 1 ХЭВЛЭЛИЙН МЭДЭЭ ШУУРХАЙ Монгол Улсын Сангийн яам, АНУ-ын Олон Улсын Хөгжлийн Агентлагийн (ОУХА) хооронд байгуулсан хоёр талт тусламжийн хэлэлцээрийн хүрээнд 8.3 тэрбум төгрөг (гурван сая ам.доллар)-ийн нэмэлт тусламжийг Монгол Улсын тогтвортой хөгжлийн зорилтын хүрээнд хэрэгжүүлэхээ АНУ-ын Засгийн газар өнөөдөр мэдэгдэв. Уг тусламж нь сайн засаглалыг бэхжүүлэх, иргэний нийгмийн оролцоог нэмэгдүүлэхэд чиглэсэн төслийг үргэлжлүүлэн хэрэгжүүлэх, мөн энэ чиглэлээр шинэ төсөл хөтөлбөрүүдийг хэрэгжүүлэхэд чиглэх аж. ОУХА-аас Монгол Улсын тусгаар тогтнол, ардчиллыг дэмжих чиглэлээр 2004 оноос хойш үзүүлж буй дэмжлэгийн нийт дүнг 342 тэрбум төгрөг (123 сая ам.доллар) болж байгаа билээ.
и так далее с остальной частью статьи.
mn_en. json здесь
{"https://mn.usembassy.gov/mn/2020-naadam-mn/": "https://mn.usembassy.gov/2020-naadam/", "https://mn.usembassy.gov/mn/06272020-presidential-proclamation-mn/": "https://mn.usembassy.gov/062620-presidential-proclamation/", "https://mn.usembassy.gov/mn/pr-060320-mn/": null, "https://mn.usembassy.gov/mn/dv-2021-status-check-mn/": "https://mn.usembassy.gov/dv-2021-status-check/", "https://mn.usembassy.gov/mn/pr-050120-mn/": "https://mn.usembassy.gov/pr-050120/", "https://mn.usembassy.gov/mn/pr-042320-mca-website-mn/": "https://mn.usembassy.gov/pr-042320-mca-website/", "https://mn.usembassy.gov/mn/2020-pr-us-mongolia-cpc-mn/": "https://mn.usembassy.gov/2020-pr-us-mongolia-cpc/", "https://mn.usembassy.gov/mn/lead-2020-in-country-mn/": null, "https://mn.usembassy.gov/mn/press-release-usaid-mar-24-2020-mn/": "https://mn.usembassy.gov/press-release-usaid-mar-24-2020/", "https://mn.usembassy.gov/mn/event-suspension-of-nonimmigrant-and-immigrant-visa-services-due-to-local-covid-19-related-preventative-measures-and-limited-staffing-mn/": "https://mn.usembassy.gov/event-suspension-of-nonimmigrant-and-immigrant-visa-services-due-to-local-covid-19-related-preventative-measures-and-limited-staffing/", "https://mn.usembassy.gov/mn/2020-best-program-pr-mn/": "https://mn.usembassy.gov/2020-best-program-pr/", "https://mn.usembassy.gov/mn/2020-ncov-info-for-visa-mn/": "https://mn.usembassy.gov/2020-ncov-info-for-visa/", "https://mn.usembassy.gov/mn/how-to-wash-hands-properly-mn/": null, "https://mn.usembassy.gov/mn/2019-%d0%be%d0%bd%d1%8b-%d1%88%d0%b8%d0%bd%d1%8d-%d1%82%d3%a9%d1%80%d0%bb%d0%b8%d0%b9%d0%bd-%d0%ba%d0%be%d1%80%d0%be%d0%bd%d0%be%d0%b2%d0%b8%d1%80%d1%83%d1%81%d1%8b%d0%b3-%d1%82%d0%b0%d1%80%d0%b0/": "https://mn.usembassy.gov/proclamation-on-suspension-of-entry-as-immigrants-and-nonimmigrants-of-persons-who-pose-a-risk-of-transmitting-2019-novel-coronavirus/", "https://mn.usembassy.gov/mn/coronavirus-cdc-info-mn/": "https://mn.usembassy.gov/coronavirus-cdc-info/", "https://mn.usembassy.gov/mn/usg-support-sme-mn/": "https://mn.usembassy.gov/usg-support-sme/", "https://mn.usembassy.gov/mn/invitation-to-the-u-s-mongolia-child-protection-compact-civil-society-information-session-mn/": "https://mn.usembassy.gov/invitation-to-the-u-s-mongolia-child-protection-compact-civil-society-information-session/", "https://mn.usembassy.gov/mn/2019-afs-pr-mn/": "https://mn.usembassy.gov/2019-afs-pr/", "https://mn.usembassy.gov/mn/5th-national-alumni-conference/": null, "https://mn.usembassy.gov/mn/2019-mcc-vp-visit-mn/": "https://mn.usembassy.gov/2019-mcc-vp-visit/", "https://mn.usembassy.gov/mn/2019-amb-remarks-sirpa-mn/": "https://mn.usembassy.gov/2019-amb-remarks-sirpa/", "https://mn.usembassy.gov/mn/2019-national-alumni-conference-registration/": null, "https://mn.usembassy.gov/mn/additional-us-government-assistance-to-advance-sustainable-and-inclusive-growth-in-mongolia-mn/": "https://mn.usembassy.gov/additional-us-government-assistance-to-advance-sustainable-and-inclusive-growth-in-mongolia/", "https://mn.usembassy.gov/mn/austr-gresser-gsp-outreach-mn/": "https://mn.usembassy.gov/austr-gresser-gsp-outreach/", "https://mn.usembassy.gov/mn/2019-mcc-us-study-tour-mn/": "https://mn.usembassy.gov/2019-mcc-us-study-tour/", "https://mn.usembassy.gov/mn/nextlevel-2019-mn/": "https://mn.usembassy.gov/nextlevel-2019/", "https://mn.usembassy.gov/mn/2019-pr-cpc-partnership-mn/": "https://mn.usembassy.gov/2019-pr-cpc-partnership/", "https://mn.usembassy.gov/mn/2019-media-statement-secdef-esper-mn/": "https://mn.usembassy.gov/2019-media-statement-secdef-esper/", "https://mn.usembassy.gov/mn/2019-next-level-program-mn/": "https://mn.usembassy.gov/the-next-level-american-hip-hop-program-is-coming-to-mongolia/", "https://mn.usembassy.gov/mn/sec-naadam-greetings-mn/": null, "https://mn.usembassy.gov/mn/%d0%b0%d0%bd%d1%83-%d1%8b%d0%bd-%d1%8d%d0%bd%d1%8d%d1%82%d1%85%d1%8d%d0%b3-%d0%bd%d0%be%d0%bc%d1%85%d0%be%d0%bd-%d0%b4%d0%b0%d0%bb%d0%b0%d0%b9%d0%bd-%d1%86%d1%8d%d1%80%d0%b3%d0%b8%d0%b9%d0%bd-%d0%ba/": "https://mn.usembassy.gov/u-s-indo-pacific-commander-admiral-phil-davidson-makes-visit-to-mongolia/", "https://mn.usembassy.gov/mn/2019-pr-khan-quest-mn/": "https://mn.usembassy.gov/2019-pr-khan-quest/", "https://mn.usembassy.gov/mn/%d0%b0%d0%b1%d0%b5%d1%80%d0%b4%d0%b8%d0%bd-%d0%bc%d0%be%d0%bd%d0%b3%d0%be%d0%bb%d0%b4-%d1%85%d3%a9%d1%82%d3%a9%d0%bb%d0%b1%d3%a9%d1%80%d0%b8%d0%b9%d0%bd-%d1%85%d1%83%d0%b2%d0%b0%d0%b0%d1%80%d1%8c/": "https://mn.usembassy.gov/aberdeen-in-mongolia-program-schedule/", "https://mn.usembassy.gov/mn/u-s-visa-information-for-mongolian-artists-performers-and-entertainers-mn/": "https://mn.usembassy.gov/u-s-visa-information-for-mongolian-artists-performers-and-entertainers/", "https://mn.usembassy.gov/mn/%d0%b0%d0%bd%d1%83-%d1%8b%d0%bd-%d1%8d%d0%bb%d1%87%d0%b8%d0%bd-%d1%81%d0%b0%d0%b9%d0%b4-%d0%bc%d0%b0%d0%b9%d0%ba%d0%bb-%d0%ba%d0%bb%d0%b5%d1%87%d0%b5%d1%81%d0%ba%d0%b8-%d0%b0%d0%bc%d0%b5/": "https://mn.usembassy.gov/u-s-ambassador-michael-klecheski-no-better-way-to-celebrate-the-year-of-youth-than-by-providing-opportunities-to-study-in-america/", "https://mn.usembassy.gov/mn/ama-2019-mn/": null, "https://mn.usembassy.gov/mn/media-note-aberdeen-mn/": "https://mn.usembassy.gov/media-note-aberdeen/", "https://mn.usembassy.gov/mn/2019-h2-visa-update-mn/": "https://mn.usembassy.gov/2019-h2-visa-update/", "https://mn.usembassy.gov/mn/2018-cda-remarks-masa-new-year-mn/": "https://mn.usembassy.gov/2018-cda-remarks-masa-new-year/", "https://mn.usembassy.gov/mn/%d0%be%d0%bb%d0%be%d0%bd-%d2%af%d0%bd%d0%b4%d1%8d%d1%81%d1%82%d0%bd%d0%b8%d0%b9-%d1%86%d0%b0%d0%b3%d0%b0%d0%b0%d1%87%d0%bb%d0%b0%d0%bb%d1%8b%d0%bd-%d0%b2%d0%b8%d0%b7%d0%bd%d0%b8%d0%b9-2020-%d0%be/": null, "https://mn.usembassy.gov/mn/2018-habit-of-environmental-security-dialogue-mn/": "https://mn.usembassy.gov/2018-habit-of-environmental-security-dialogue/", "https://mn.usembassy.gov/mn/2018-mcc-water-compact-signing-mn/": "https://mn.usembassy.gov/2018-mcc-water-compact-signing/", "https://mn.usembassy.gov/mn/2018-roadmap-for-expanded-economic-partnership-mn/": "https://mn.usembassy.gov/2018-roadmap-for-expanded-economic-partnership/", "https://mn.usembassy.gov/mn/2018-joint-statement-pm-khurelsukh-sec-pompeo-mn/": "https://mn.usembassy.gov/2018-joint-statement-pm-khurelsukh-sec-pompeo/", "https://mn.usembassy.gov/mn/2018-vp-pence-pm-khurelsukh-readout-mn/": "https://mn.usembassy.gov/2018-vp-pence-pm-khurelsukh-readout/", "https://mn.usembassy.gov/mn/us-congressional-delegation-official-visit-to-mongolia-mn/": "https://mn.usembassy.gov/us-congressional-delegation-official-visit-to-mongolia/", "https://mn.usembassy.gov/mn/travel-responsibly-phase-2-mn/": null, "https://mn.usembassy.gov/mn/media-note-facebook-live-series-mn/": null, "https://mn.usembassy.gov/mn/joint-statement-2nd-epd-mn/": "https://mn.usembassy.gov/joint-statement-2nd-epd/", "https://mn.usembassy.gov/mn/joint-statement-mongolia-japan-u-s-trilateral-meeting-mn/": "https://mn.usembassy.gov/joint-statement-mongolia-japan-u-s-trilateral-meeting/", "https://mn.usembassy.gov/mn/hrr-2018-pr-mn/": "https://mn.usembassy.gov/hrr-2018-pr/", "https://mn.usembassy.gov/mn/ririe-woodbury-media-note/": null, "https://mn.usembassy.gov/mn/%d1%85%d1%8d%d0%b2%d0%bb%d1%8d%d0%bb-%d0%bc%d1%8d%d0%b4%d1%8d%d1%8d%d0%bb%d0%bb%d0%b8%d0%b9%d0%bd-%d0%b7%d3%a9%d0%b2%d0%bb%d3%a9%d0%bc%d0%b6-%d0%b3%d1%80%d1%8d%d0%bc%d0%bc%d0%b8%d0%b9%d0%bd-%d1%88/": null, "https://mn.usembassy.gov/mn/development-of-the-millennium-challenge-corporation-investment-continues-to-move-forward-mn/": "https://mn.usembassy.gov/development-millennium-challenge-corporation-investment-continues-move-forward/", "https://mn.usembassy.gov/mn/seth-glier-media-note-mn/": null, "https://mn.usembassy.gov/mn/utaagui-mn/": null, "https://mn.usembassy.gov/mn/a-s-schriver_media-note-mn/": "https://mn.usembassy.gov/a-s-schriver_media-note/", "https://mn.usembassy.gov/mn/lunar-greeting-2018-mn/": "https://mn.usembassy.gov/lunar-greeting-2018/", "https://mn.usembassy.gov/mn/h-2-visa-information-in-mongolian/": "https://mn.usembassy.gov/h-2-visa-information/", "https://mn.usembassy.gov/mn/media-note-aas-eap-susan-thornton-travels/": "https://mn.usembassy.gov/media-note-aas-eap-susan-thornton-travels-to-ulaanbaatar/", "https://mn.usembassy.gov/mn/mcc-regional-deputy-vice-president-visits-mongolia-meets-senior-government-officials-mcc-compact-development-progress-mn/": "https://mn.usembassy.gov/mcc-regional-deputy-vice-president-visits-mongolia-meets-senior-government-officials-mcc-compact-development-progress/", "https://mn.usembassy.gov/mn/international-day-elimination-violence-women-mn/": null, "https://mn.usembassy.gov/mn/%d0%b0%d0%bd%d1%83-%d1%8b%d0%bd-%d1%8d%d0%bb%d1%87%d0%b8%d0%bd-%d1%81%d0%b0%d0%b9%d0%b4-%d0%b3%d0%b0%d0%bb%d1%82-%d0%bd%d1%83%d1%82%d0%b0%d0%b3-%d0%b1%d1%83%d1%86%d0%bd%d0%b0/": "https://mn.usembassy.gov/u-s-ambassador-galt-depart-mongolia/", "https://mn.usembassy.gov/mn/%d0%b4%d0%be%d1%80%d0%bd%d0%be%d0%b3%d0%be%d0%b2%d1%8c-%d0%b0%d0%b9%d0%bc%d0%b3%d0%b8%d0%b9%d0%bd-%d0%b7%d0%b0%d0%bc%d1%8b%d0%bd-%d2%af%d2%af%d0%b4-%d1%81%d1%83%d0%bc%d1%8b%d0%bd-%d1%86%d1%8d%d1%86/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-kindergarten-ribbon-cutting-ceremony-zamiin-uud/", "https://mn.usembassy.gov/mn/u-s-ambassador-galt-opens-new-u-s-funded-kindergarten-zamiin-uud-mn/": "https://mn.usembassy.gov/u-s-ambassador-galt-opens-new-u-s-funded-kindergarten-zamiin-uud/", "https://mn.usembassy.gov/mn/media-note-invitation-cover-ribbon-cutting-new-u-s-funded-kindergarten-zamiin-uud-mn/": "https://mn.usembassy.gov/media-note-invitation-cover-ribbon-cutting-new-u-s-funded-kindergarten-zamiin-uud/", "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-responsible-travel-outreach-campaign-launch-event-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-responsible-travel-outreach-campaign-launch-event/", "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-9th-ubiff-opening-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-9th-ubiff-opening%e2%80%a8/", "https://mn.usembassy.gov/mn/press-release-american-film-showcase-joins-9th-annual-ulaanbaatar-international-film-festival-mn/": "https://mn.usembassy.gov/press-release-american-film-showcase-joins-9th-annual-ulaanbaatar-international-film-festival/", "https://mn.usembassy.gov/mn/media-note-chance-interview-american-film-showcase-speakers-stacey-ferreira-tony-gerber-9th-annual-ulaanbaatar-intl-film-festival-mn/": "https://mn.usembassy.gov/media-note-chance-interview-american-film-showcase-speakers-stacey-ferreira-tony-gerber-9th-annual-ulaanbaatar-intl-film-festival/", "https://mn.usembassy.gov/mn/%d0%b1%d0%b0%d0%bb%d0%b0%d0%bd%d1%81-%d0%bc%d1%8d%d0%b9%d0%b6%d0%b8%d0%ba-%d1%81%d1%83%d1%80%d0%b3%d0%b0%d0%bb%d1%82-%d0%b4%d0%b0%d0%b4%d0%bb%d0%b0%d0%b3%d1%8b%d0%bd-%d1%85%d0%b0%d0%b0%d0%bb%d1%82/": "https://mn.usembassy.gov/ambassador-jennifer-zimdahl-galts-remarks-balance-magic-closing-ceremony/", "https://mn.usembassy.gov/mn/%d0%b6%d0%be%d1%80%d0%b6-%d0%bc%d0%b0%d1%80%d1%88%d0%b0%d0%bb%d0%bb%d1%8b%d0%bd-%d1%82%d3%a9%d0%b2%d0%b8%d0%b9%d0%bd-%d1%82%d3%a9%d0%b3%d1%81%d3%a9%d0%b3%d1%87%d0%b4%d0%b8%d0%b9%d0%bd-%d1%83%d1%83/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-george-c-marshall-center-alumni-event/", "https://mn.usembassy.gov/mn/2019-dv-lottery/": null, "https://mn.usembassy.gov/mn/%d2%af%d0%bd%d0%b4%d1%8d%d1%81%d0%bd%d0%b8%d0%b9-%d0%bc%d1%83%d0%b7%d0%b5%d0%b9%d0%bd%d1%83%d1%83%d0%b4%d1%8b%d0%bd-2017-%d0%be%d0%bd%d1%8b-%d0%b1%d0%b0%d0%b3%d0%b0-%d1%85%d1%83%d1%80%d0%b0%d0%bb/": "https://mn.usembassy.gov/ambassador-jennifer-zimdahl-galts-remarks-annual-national-museums-conference-2017/", "https://mn.usembassy.gov/mn/%d0%b8%d0%bd%d0%b2%d0%b5%d1%81%d1%82-%d0%bc%d0%be%d0%bd%d0%b3%d0%be%d0%bb%d0%b8%d0%b0-%d0%b1%d0%b0%d0%b3%d0%b0-%d1%85%d1%83%d1%80%d0%bb%d1%8b%d0%bd-%d1%8d%d0%bb%d1%87%d0%b8%d0%bd-%d1%81%d0%b0%d0%b9/": "https://mn.usembassy.gov/ambassadors-remarks-invest-mongolia-2017-ambassadors-panel/", "https://mn.usembassy.gov/mn/success-story-access-program-alumna-oyunbolor-mn/": "https://mn.usembassy.gov/success-story-access-program-alumna-oyunbolor/", "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-peace-corps-swearing-ceremony-m28-volunteers-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-peace-corps-swearing-ceremony-m28-volunteers/", "https://mn.usembassy.gov/mn/%d1%85%d1%8d%d0%b2%d0%bb%d1%8d%d0%bb%d0%b8%d0%b9%d0%bd-%d0%b7%d3%a9%d0%b2%d0%bb%d3%a9%d0%bc%d0%b6-hackub-%d1%85%d0%b0%d0%ba%d0%b0%d1%82%d0%be%d0%bd-%d0%b1%d2%af%d1%82%d1%8d%d1%8d%d0%bb-%d1%82%d0%b0/": "https://mn.usembassy.gov/media-note-inaugural-hack-ub-hackathon/", "https://mn.usembassy.gov/mn/fourth-annual-american-days-expo-2017-mn/": "https://mn.usembassy.gov/fourth-annual-american-days-expo-2017/", "https://mn.usembassy.gov/mn/lead-mongolia-%d1%85%d3%a9%d1%82%d3%a9%d0%bb%d0%b1%d3%a9%d1%80%d0%b8%d0%b9%d0%bd-%d0%be%d1%80%d0%be%d0%bb%d1%86%d0%be%d0%b3%d1%87%d0%b4%d0%be%d0%b4-%d1%85%d0%b0%d0%bd%d0%b4%d0%b0%d0%b6-%d1%8d%d0%bb/": "https://mn.usembassy.gov/welcoming-remarks-ambassador-jennifer-zimdahl-galt-new-lead-mongolia-fellows/", "https://mn.usembassy.gov/mn/secretary-tillerson-naadam-greeting-mn/": "https://mn.usembassy.gov/secretary-tillerson-naadam-greeting/", "https://mn.usembassy.gov/mn/%d1%82%d1%83%d1%81%d0%b3%d0%b0%d0%b0%d1%80-%d1%82%d0%be%d0%b3%d1%82%d0%bd%d0%be%d0%bb%d1%8b%d0%bd-%d3%a9%d0%b4%d1%80%d0%b8%d0%b9%d0%bd-%d0%b1%d0%b0%d1%8f%d1%80%d1%8b%d0%bd-%d0%b0%d1%80%d0%b3%d0%b0/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-u-s-independence-day-celebration/", "https://mn.usembassy.gov/mn/press-release-project-launch-event-improving-victim-centered-investigations-prosecutions-trafficking-persons-cases-mongolia-mn/": "https://mn.usembassy.gov/press-release-project-launch-event-improving-victim-centered-investigations-prosecutions-trafficking-persons-cases-mongolia/", "https://mn.usembassy.gov/mn/media-note-opening-ceremony-improving-victim-centered-investigations-prosecutions-trafficking-persons-tip-cases-mongolia-project-mn/": "https://mn.usembassy.gov/media-note-opening-ceremony-improving-victim-centered-investigations-prosecutions-trafficking-persons-tip-cases-mongolia-project/", "https://mn.usembassy.gov/mn/%d0%bd%d0%b0%d0%b9%d1%80%d0%b0%d0%bc%d0%b4%d0%b0%d0%bb%d1%8b%d0%bd-%d0%b0%d1%8f%d0%bb%d0%b3%d1%83%d1%83-%d1%82%d2%af%d0%bd%d1%88%d0%bb%d1%8d%d0%bb%d0%b8%d0%b9%d0%bd-30-%d0%b6%d0%b8%d0%bb/": "https://mn.usembassy.gov/friendship-melodies-celebrating-30-years-partnership/", "https://mn.usembassy.gov/mn/%d0%b2%d3%a9%d2%af%d0%ba%d0%bb-%d1%82%d1%80%d0%b0%d1%88-%d1%85%d0%b0%d0%bc%d1%82%d0%bb%d0%b0%d0%b3-%d0%bc%d0%be%d0%bd%d0%b3%d0%be%d0%bb%d0%b4/": "https://mn.usembassy.gov/vocal-trash-mongolia/", "https://mn.usembassy.gov/mn/%d0%b0%d0%b7%d0%b8-%d0%b4%d0%b0%d1%85%d1%8c-%d0%b0%d1%80%d0%b4%d1%87%d0%b8%d0%bb%d0%bb%d1%8b%d0%bd-%d0%b7%d0%b0%d0%bb%d1%83%d1%83-%d0%bc%d0%b0%d0%bd%d0%bb%d0%b0%d0%b9%d0%bb%d0%b0%d0%b3%d1%87%d0%b4/": "https://mn.usembassy.gov/u-s-convenes-summit-asias-young-democracy-leaders-ulaanbaatar/", "https://mn.usembassy.gov/mn/30-%d0%b6%d0%b8%d0%bb%d0%b8%d0%b9%d0%bd-%d0%be%d0%b9%d0%b4-%d0%b7%d0%be%d1%80%d0%b8%d1%83%d0%bb%d1%81%d0%b0%d0%bd-%d1%85%d2%af%d0%bb%d1%8d%d1%8d%d0%bd-%d0%b0%d0%b2%d0%b0%d0%bb%d1%82-%d0%b4%d1%8d%d1%8d/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-30th-anniversary-reception/", "https://mn.usembassy.gov/mn/remarks012717c-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-concert-break-reality-mongolian-state-morin-khuur-ensemble/", "https://mn.usembassy.gov/mn/remarks012717b-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-joint-press-conference-30th-anniversary-photo-exhibit/", "https://mn.usembassy.gov/mn/remarks012717a-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-30th-anniversary-conference-u-s-mongolia-diplomatic-relations/", "https://mn.usembassy.gov/mn/pr122316-mn/": "https://mn.usembassy.gov/u-s-embassy-ministry-foreign-affairs-mongolia-unveil-official-logo-30th-anniversary-u-s-mongolia-diplomatic-relations/", "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-peace-corps-mongolias-25th-anniversary-celebration-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-peace-corps-mongolias-25th-anniversary-celebration/", "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-amcham-july-meeting-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-amcham-july-meeting/", "https://mn.usembassy.gov/mn/screening-live-ub-documentary-mongolia-mn/": "https://mn.usembassy.gov/screening-live-ub-documentary-mongolia/", "https://mn.usembassy.gov/mn/u-s-department-state-launches-paralympians-withoutlimits-social-media-campaign-mn/": "https://mn.usembassy.gov/u-s-department-state-launches-paralympians-withoutlimits-social-media-campaign/", "https://mn.usembassy.gov/mn/remarks-honor-26th-anniversary-americans-disabilities-act-mn/": "https://mn.usembassy.gov/remarks-honor-26th-anniversary-americans-disabilities-act/", "https://mn.usembassy.gov/mn/secretary-kerry-statement-on-mongolian-festival-mn/": null, "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-the240th-anniversary-independence-united-states-america-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-the240th-anniversary-independence-united-states-america/", "https://mn.usembassy.gov/mn/u-s-increases-disaster-risk-reduction-support-lessen-impact-winter-emergency-mongolia-mn/": "https://mn.usembassy.gov/u-s-increases-disaster-risk-reduction-support-lessen-impact-winter-emergency-mongolia/", "https://mn.usembassy.gov/mn/peace-khaan-quest-mongolia-global-peace-operations-initiative-mn/": "https://mn.usembassy.gov/peace-khaan-quest-mongolia-global-peace-operations-initiative/", "https://mn.usembassy.gov/mn/u-s-funded-training-supports-efforts-combat-domestic-violence-mn/": "https://mn.usembassy.gov/u-s-funded-training-supports-efforts-combat-domestic-violence/", "https://mn.usembassy.gov/mn/%d1%82%d3%a9%d1%80%d0%b8%d0%b9%d0%bd-%d0%bd%d0%b0%d1%80%d0%b8%d0%b9%d0%bd-%d0%b1%d0%b8%d1%87%d0%b3%d0%b8%d0%b9%d0%bd-%d0%b4%d0%b0%d1%80%d0%b3%d0%b0-%d0%ba%d0%b5%d1%80%d1%80%d0%b8-2016-%d0%be%d0%bd/": "https://mn.usembassy.gov/secretary-kerry-hosts-2016-trafficking-persons-report-launch-ceremony/", "https://mn.usembassy.gov/mn/u-s-funded-training-advances-efforts-combat-human-trafficking-mn/": "https://mn.usembassy.gov/u-s-funded-training-advances-efforts-combat-human-trafficking/", "https://mn.usembassy.gov/mn/save-date-gist-online-startup-checklist-mr-david-rose-ceo-gust-mn/": "https://mn.usembassy.gov/save-date-gist-online-startup-checklist-mr-david-rose-ceo-gust/", "https://mn.usembassy.gov/mn/2016-%d0%be%d0%bd%d1%8b-%d0%b4%d1%8d%d0%bb%d1%85%d0%b8%d0%b9%d0%bd-%d1%8d%d0%bd%d1%82%d1%80%d0%b5%d0%bf%d1%80%d0%b5%d0%bd%d0%b5%d1%80%d1%83%d1%83%d0%b4%d1%8b%d0%bd-%d1%87%d1%83%d1%83%d0%bb%d0%b3%d0%b0/": "https://mn.usembassy.gov/mongolian-delegates-attend-2016-global-entrepreneurship-summit/", "https://mn.usembassy.gov/mn/visiting-u-s-experts-strengthen-capacity-combat-gender-based-violence-mn/": "https://mn.usembassy.gov/visiting-u-s-experts-strengthen-capacity-combat-gender-based-violence/", "https://mn.usembassy.gov/mn/new-united-states-program-support-democratic-development-mongolia-mn/": "https://mn.usembassy.gov/new-united-states-program-support-democratic-development-mongolia/", "https://mn.usembassy.gov/mn/secretary-kerrys-conversation-young-leaders-mn/": "https://mn.usembassy.gov/secretary-kerrys-conversation-young-leaders/", "https://mn.usembassy.gov/mn/joint-press-availability-mongolian-foreign-minister-purevsuren-lundeg-mn/": "https://mn.usembassy.gov/joint-press-availability-mongolian-foreign-minister-purevsuren-lundeg/", "https://mn.usembassy.gov/mn/secretary-kerrys-remarks-meeting-mongolian-foreign-minister-lundeg-purevsuren-mn/": "https://mn.usembassy.gov/secretary-kerrys-remarks-meeting-mongolian-foreign-minister-lundeg-purevsuren/", "https://mn.usembassy.gov/mn/secretary-kerrys-remarks-staff-families-embassy-mongolia-mn/": "https://mn.usembassy.gov/secretary-kerrys-remarks-staff-families-embassy-mongolia/", "https://mn.usembassy.gov/mn/u-s-elections-2016-mn/": "https://mn.usembassy.gov/u-s-elections-2016/", "https://mn.usembassy.gov/mn/secretary-kerrys-travel-france-mongolia-china-mn/": "https://mn.usembassy.gov/secretary-kerrys-travel-france-mongolia-china/", "https://mn.usembassy.gov/mn/idahot-reinforcing-ideal-everyone-deserves-live-justice-equality-dignity-mn/": "https://mn.usembassy.gov/idahot-reinforcing-ideal-everyone-deserves-live-justice-equality-dignity/", "https://mn.usembassy.gov/mn/op-ed-u-s-secretary-john-kerry-time-treat-corruption-seriousness-deserves-mn/": "https://mn.usembassy.gov/op-ed-u-s-secretary-john-kerry-time-treat-corruption-seriousness-deserves/", "https://mn.usembassy.gov/mn/save-date-international-students-day-may-11-2016-mn/": "https://mn.usembassy.gov/save-date-international-students-day-may-11-2016/", "https://mn.usembassy.gov/mn/air-quality-awareness-week-clean-air-mn/": "https://mn.usembassy.gov/air-quality-awareness-week-clean-air/", "https://mn.usembassy.gov/mn/u-s-department-state-launches-fifth-annual-free-press-campaign-mn/": "https://mn.usembassy.gov/u-s-department-state-launches-fifth-annual-free-press-campaign/", "https://mn.usembassy.gov/mn/womens-political-participation-and-influence-training-mn/": null, "https://mn.usembassy.gov/mn/remarks-ambassador-jennifer-zimdahl-galt-peacekeeping-vehicles-hand-ceremony-mn/": "https://mn.usembassy.gov/remarks-ambassador-jennifer-zimdahl-galt-peacekeeping-vehicles-hand-ceremony/", "https://mn.usembassy.gov/mn/alaska-army-national-guard-soldiers-deploy-afghanistan-alongside-mongolians-mn/": "https://mn.usembassy.gov/alaska-army-national-guard-soldiers-deploy-afghanistan-alongside-mongolians/"}
Обратите внимание, что в каждом div больше 1 <p> тегов. Вы должны использовать findAll ('p') вместо find ('p') . Пример:
<p>
# ... previous code here for x in div: print(k) print('\n\n'.join(map(lambda y: y.text, x.findAll('p'))))