очистка веб-страницы JOINURL и очистка страницы

Question

Я пытаюсь присоединить удаленную ссылку к основному URL сайта (заглушку), а затем передать ее в full_url [].Я не могу заставить его работать, есть идеи?

import pandas as pd
import requests
from bs4 import BeautifulSoup
from urllib import parse
from urllib.parse import urljoin

url = 'http://www.owgr.com/events?pageNo=1&pageSize=400&tour=Eur&year=2019'
stub = 'http://www.owgr.com'
page = requests.get(url, headers=headers)
soup = BeautifulSoup(page.text, 'html.parser')


full_url = []

full_url_elem = soup.find_all(id='ctl5')  

for item in full_url_elem:
    full_url.item.find('a').get('href')
    full_url.append(item(urljoin('stub', 'event_url'))

Answer 1

Здесь есть несколько проблем:

Вы перебираете свой full_url_elem, что хорошо, но что должен делать full_url.item.find('a').get('href')?Избавиться от full_url.

Вам нужно получить тег <a> из item, а затем получить href из этого.Во-вторых, вам нужно исправить то, к чему вы хотите присоединиться.

for item in full_url_elem:
    full_url.item.find('a').get('href')  # <--- remove full_url
    full_url.append(item(urljoin('stub', 'event_url'))  # <--- just use the + to combine 2 strings, and what is event_url? Also, you joining the string `'stub'` and the string `'event_url'` which will return `stubevent_url`, when what you really want is the variable `stub` and variable `event_url`

Вот как это должно выглядеть:

import pandas as pd
import requests
from bs4 import BeautifulSoup
from urllib import parse
from urllib.parse import urljoin


headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36'}

url = 'http://www.owgr.com/events?pageNo=1&pageSize=400&tour=Eur&year=2019'
stub = 'http://www.owgr.com'
page = requests.get(url, headers=headers)
soup = BeautifulSoup(page.text, 'html.parser')


full_url = []

full_url_elem = soup.find_all(id='ctl5')  

for item in full_url_elem:
    event_url = item.find('a').get('href')
    full_url.append(stub + event_url)

Вывод:

print (full_url)
['http://www.owgr.com/en/Events/EventResult.aspx?eventid=7631', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7621', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7611', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7599', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7587', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7562', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7555', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7554', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7538', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7516', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7507', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7493', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7480', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7457', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7440', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7427', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7415', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7403', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7388', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7374', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7369', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7364', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7359', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7346', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7329', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7323', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7314', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7304', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7302', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7297', 'http://www.owgr.com/en/Events/EventResult.aspx?eventid=7290']