Соскребение календаря низких тарифов с сайта авиакомпании.Проблема с необычными заголовками - PullRequest
0 голосов
/ 24 апреля 2019

Я использую Python для сбора данных о ценах от southwest.com. Попытка собрать их Низкие цены в календаре. Пример страницы: https://www.southwest.com/air/low-fare-calendar/select-dates.html?adultPassengersCount=1&currencyCode=USD&departureDate=2019-04-25&destinationAirportCode=MCO&originationAirportCode=MDW&passengerType=ADULT&returnAirportCode=&returnDate=&tripType=oneway

Необходимые данные поступают из почтового запроса на https://www.southwest.com/api/air-booking/v1/air-booking/page/air/low-fare-calendar/select-dates.

Проблема в заголовках, которые начинаются с x -:

x-5ku220jw-a: I_ftfpDjDyifDufjizDcDs7teKDhlBY7UN6xgaX7hf8wp7=ZfV=ZiagmgMX1lgT1Yf8tIziO2PrYhLk6DprKAQChh-kEK2=QDyutU3E-gz6wgBpYQrUf8KlMjN6c0aP_95jAIV9BYr35sb66QM06CZP3IZrgF1XwfgNR0akQIc5tIaOIYagjTf1yAyP3louw57s3iZ_m07huHuaxFJI7lU_IFSvNg91IMb8riZ_RIcUrp7rNlpUffIUfqa8aIrEPkaSxiI=wPSP3k9=c1yvM8ikxhZ6h9t1kjSgmibYJInPU8hPfp-3uezOfeKLZIiQypKSQl14WSN_BPJURUaP1qa9R87EBFb8AqSv-ia7IiB0jlzSZ5IktfFfu2asJBII5lh5ss-XagNiZflSxYf8ZPVCljvSzl36bldUMFhj18zkI07=tgkXuU0UcZ7BPpb8tI3=3jr8-I-5QpfLBKWk3iq0bpK7KIsh7C3Ke1cB7t_S32kXhi_fBYN6VpIfu0-kyi753uzv0I-Q5=UsBYNP3DskijalfqWYP2kf5ly6mDILRqLimi7=AE3=tDcsM8-06fr7zDaLcc9I72zBRfIrLqN6Ch-b8D_f7fNuM__MCgaJLDkfhOBSwk-8yI-lusLPFvrN4fejQR79fsLmxFhrRS7jeEbItf_gBPKYS93w6YNimDrfaXR4uICfPKNP6fZP6ldEpciwKqNu7qMBRm3=mDavQ2szxqr=Cf9Yti-YuEU9c3I7t5WkQDs=LlDID2zSMp-gZI-BSf-0bkfURp3Y5fiRuirrB0aFBqf=SNVjLmBY5jakHlzURuu8cTM5wXKv3gaitlD8M3-vMIcYufiDrE4_fi-RhivW51bi7hY7jkhrhfbUBFh=6IaUbXhWTiu8R0Z_T37SQqW=jMLfA5sEBqrDARzQTqU=Nfkqyia7RTLrMUUgCFJbR2TIhl80I9BSLfaijYM6M5XYrCzvjIN6mU_j3jM-sRNiufLPTjUPm8K16Ua5tfr=b3=pfBacjF46fX7uMCujTsW=j0LS4lK8yWTIKlzQMYNhjfqF8WaPcIbIOsSIyp=8cqy6fvKS5de_Bg7v_IzhtYMf7FI8bluf7kJkZ8UFfqifSiSXBlc7ukXDkFJ=LiKs5pZP_f7jBFzvy5y_rRJ6_qM6Nd7PnL46xItbRu3PMUOEjIbSwgKtsHc9MFz9RI-BepBaLqi7A5zqZFapf0N6PYSiuFtj6iZ6LTsX3pbNSWL=w2USZpbPM3a=RI77fI-DnwiUMeSEcDjm0dyfZPb8Mi6jbE1folpkyf-vNMbUcUV9WfoXu5zEMrrj-pur7jr=MprvMYfjZdp5yqapmYrP3D4lteJ8ZphgMI9Y5W2WhjahM5rPRF-RfIILB2zXrDejnFzStjScMY7RurgpMIbsHDU=CIbYjI9ifsSxFfakif-9RU7sMgrmZ9XihWzkZPA8MI-9cR7sB8bPZYfY90asf5ch0davzIB7ulqrKu-8_PZubgL-ApbujDBfZjf8QDUrRjbfhpbThpJ=3iZ6_lkI1DUPsib8NR2pSfasVDw3uvLTqFBiuFbU7WU=jlD8Blbi7hu7MFq8hj30MmDR1jN6VDyStD4Q1jSP-FpMfgr7k2IjM0f83lkzklKhtFw7qlz=nI=ISqrS1g37flKPAizmAfS4ffvrqehI5fTD3fi=sI_YjtZ6Y2eVtj7=t0SNM9KURiakypa0_ib6-sKOcONujgKQ_dejhi7=4IcPQ5chjiNj3IXIrpbIhUjcZDrBfpas74JiuFV=rjf8UUH=3DsPckzkQIkgMuavhIZrZ21ycDpOag7vNLKXhq7PfDNPQCKuilKRulhSy3ij9DDIZI99cqi9fp7OcFJiB2zfjFAenj_PBq78MMI0ZDLSQ29mmDfrzhZPNFK1t93w3i9WZgIjQDeQBUq80qavQDevID-s7iy6Aj7STFJ63FsY5qTmUPO6MiVjsXBjMPfSmaLlbjaSnp7khFBYSg46ZqrbBT78tf1vmQa5yUyPwE-UR4S4=zN6_TZ669KB4fL=6iy6ZEBSmIa5miefM27LBqNPMDzSQqfS3UAUQhip7gs=322kt2ZPm8XPmixikacSCYMMxpBPQiZitiMjIfcEsp-9Rrz833LYus7k9IakRiVfYiQ0epu7ufLi9DaX5pa0UYiLRDd8jp-9SPsaUDIUR37Sm9ccrDd8_IrPV0JIZImiuTbsagaujULPs9hIjphgjYLi7eUSBezQb2zvjU3=mkzu1Ec=TqfI1m7s3gakhDz5AkUf7I-3BXL0hzKIupak_ffY5qSNfii0t3fj9kbUBL4uwF7kUi_8AjaXhiSgEFurL5svxI3_BF7v1dzPMDfjtTNujFq8Ty37tDc=MlU5Al-N6h7E883SM2I86Cc=r0AUR3I7ug7yRF1kygdzvP1XhgejrdL8Ah7=Ki-hujfirP_7tfuU=oWYfgaYIiaIMfJkifzszkU8PPMjADgpZpc9micIWp46BFM0flQ4Rse0tirZBU-lCWLm3YifrUJshfkXjY7P10_MjkaQMU4cukM_RI-S0lIyBXK5VEKchkZZRFd8TFQXuDZ_rezP3k_ZRlJEfT4utq7XBgf7jUfYmDSy-pabrIz9cFAFRFujhQSNR2ffjjNN=j3EBFBYZq7gwj7j0fL=MII=yhw6BkLY5gs=pgq8Z4mkmtevIjJUBIzk3qM6Q9trSY7uSF7=tIrEBUa8yhIrVYakAOQXSihj5DaR4FJEfU-7sDxg_rKpW9K0rlzvthSi7ptRRULj8WIjj8qURdPIRPT8losP3sLIfhLujp-bjb16bgWYS=A8mDbItmu8kjTISpuL7fqi55ZNcqrjwgavmq=gklkXuDsjti9lZ94D9I-hf9h8Qh3kQJasYqrQaqryRp373iZu7O-khX3YufUjQezmm3L9cFJs7gasuz9YjIZ_jisYugs6cf9kNUMr-DpjNqfjAfssuFAIAIcj3qMctFc7EqLNojTIhCxufBQEfSevQXhTxIz4RgL=eM_1Bh_8tiVM1Uhl7DIIJicELfILfdSXCjB8epcrIjr6mjPFDIsRZpbrBXKS68I7hkMfu3ejsq7kjIdTsCAe9i-vhscEToKJcfII7DdrTFzQL_4Tf8yf75J83JLDjEh8Qp3YjWpsA5zkoic=QFLIjq7E7Spjh8zSZXfEfTIYhjrjtgViKdLiu8swwq7SsNZ6P2Z6ZU3Yfkq8MPbrts9kRlpjLDI5AgUky4KPmfJk2q7=UCAm3DNgEFZ66I-==sqw18pkMfsbjp1zx8BjAMZ6iipfpDyvrJcsLjNNRh1vyi_rRqjkmfS4WSSPf3uaxFyPcjfsjiui3bhXjQWUID9uM61=Ajtjt2aDxFBPMta379s=iqLN=XKSMfushq75Ppu9t0aveDN6MFr1jlbUBqaOcIQkjIchhqf9RuVftLbDcYSptj4QLXh=ti6gA9bYhhauA9KQ382=UizLRIZNf8zRgiuncgLjyFX=iuNZ38s=tFZY7Ib8_DzS-DFDBpcvmSfI7iICaI_fhDsjU2y_LF38mInRKFuf7gIjcfbbBf7gQfBS_gMHvKuLRq9sujUD6lsPzF3SwI3YLh9IjIVj1fz6NFBDiFuI1iL=3qf4jleRtIN6yIN6j83BcIpj9IchAX1uu24XSsM=QiumNsJ-RfQkyI-7jCZu75kqxlhNzKcfhHs7ffN6MM9=jia5wu7pAi-FrIbFfgfgAWaQJfrI4gLrRpZfh6Z6ZgSv3iVFBWrlJIijRg=ZcFKkyYMvyoYFu24iuU15ME3EfY7Yr8z0QF-9D5VQbpKS6pNkpfa0QDrP0hfdx9tv0sKY75z06fLPiFJ8oYf5AIbsPU77ut7zuSa7t=3gEF3=qT7=61P=yrhsBFISmM-8xXKSpD1g6dMlff-RbDz_l2aI7I_9_XKvbi-1NfxjjWsrEfTPtL7LzjMffg1ufdrkEIb69FifjPh8AiwvMWsPc0agrDbbRoU7MhLksDy6x5N_fqLPhUu731_WLdrWdpK0aqaVhpruxlI83FcYYiIBEhP8SvUPmF-_Riue4t-ijSsv3qMfMiN_StL8m2aR5qrYtIx013a=tki83iIf57-kNI3SyYj6VXqLcFhS3jbFR3LjSiTUM3_S=Exj6f4u5FQP0kPky2LyRf98y3s0mvaNBPT8j5JEfeasjjrYujakb9BYwDdeujPfwFAmefIUM=3yIF-JRYaaxp_SL21vBF74BpMjrFh1tUS=z0T8bPQgtls=NDsa4l46N5Q86qfL_FLeVEQXrIXutRz1MphpAFTURUaTJg_gQpakcIuEAF1yBqa7f8yiticYuqNjSh7NBX46EE_lbgsER9JYt=hLRYNuWEJUR0auhjilth9ERU3=SX01w2_yRI6mMi_=QEp=ADzvLhq80gLWxidptsB96p-QVlIrokg1mI3tLEKkRfy6NIFpjDev3Y7rZlk=t2z9fsZ_lU3Yupxf5Hk8sl-g6YLItIaX7jSjtUt0niU=LjBEBBb030ujMsa9Rtr=o9XS3FXwwkL=mpx0yeJkjoJ=je-0csLfgFLgCIch58IfhiuaSfUrR9hDQFhS3Io3uYLy8Nyurp-Xm8Bf1fSvfpLEcDUfRldL3gSgcZyNzIqPZpHWf9JPygaS3IUPZ4Jcfx_fjfIjtIiYtaNP35u=wRJ=hWBYfRB=jiU8hvLFUOsj9Pzp9DIDTMa98YNPVDk8eiKeJlzSDs-w32UL-iz1Q59kQDBYMf4NVXv00jsB0iBjHWKh3ix66D-5AR9zbpJeJOvgtkZ_rDz=UtJ-sgaPSCQrfg9URf-srhNi7E3gQIuCjp7Px4K9Rf7YKq7LRFBSQ3gjSDz=3pZvQlXkZkq53jKupjWYtjaOBFZuBqakbDzPfpJI7UbrBlII7PQUZ93jyhf8tq39jXqIfP46LUBYjqifAXJsMfavTpKkVFtjMU-5zEaHxK6jVI-kCFJk6XMrMoSvm5hP9Qf=jPqj3ichIar=ZI-BYUMrM0a3tFxrbi-ktuh8QYa5Rg_bRpKQB2zXlhSjyFM6hIejYR3pMFK6mqim-DUNRYLFUi377pWIuIzkQkqYJXqrBhumME-v1=ZijCcst2JERieyJiaS_iIDVqSXjrKLoFZ696Mjmg2=_ib8rptj9LejZiI8bkMRtKK3rYf=wXJ=Bl4uJFsQRpK_cgLQ6kxffqzERD_D3qLItfKSAlU=V2ITxUK1k5IwiscpJfkp7qUERIL_cFXrRq7XBkz0siuMbhkvQUJixisEajISkp-OYjL_VdLIjq=IjhffA298jnqejsPhf27o4fLQk8RsuD9QQizpMHc=3iBY7pbEcIzDxMyu=p3j9FR1rgNFWqYcOlK66XWjii9EpI9s4lkIHpaX5Fq5Bk4PmqN65R4N4KN6Ak34fpcEM9tfugzPAI3Y7jTUoYTuht7zsIcLcdxf7F3BBXJ=6kIaNp_fJq7XKUSv3DMjmgNPZEhSTiLULgav=XKvjj7jZDzOI0NutX1uuhT8-D1ge8UD70LIDFa4fgSvyF986fA67H-ORiuDCMakQXq8bqffViUjQ94PxpN6t0akQkaJBi7P9kxru3=VaUzS39tjBKeft3a0SqLPjizRh91vQpZ_Ohb8tfhPfF9=3RaSefilRzaEhYavwkrjJYacjqffIqa9rpZNBDsSQqL8LwpjcDIjQiaPTif8tqaOMkXfu8K6329IBFX8BpAUBXKVufU=tFh8Mf3jcIyP4grS6FKiZIV81UbiWyz=3l9IficLjC16MFZNR93WsfzSjf7=fkSX3rK6Zf2=QUBE7RlgVh_=tkiWjYrPZFNih2e5tiLPKXJPMWsvykfiI3SvMiSjTe_Yjlpshgs7Bib=ZlUPtUKXIF-=_6vySKFwLIuj1s4fjiN6HDf8DeXpt9J1bIoW9KIzLt_E7MJIuIr=8qaFcV-kL2zQ6FzCKhLctgbP6q98tUrSQF-1VlKXrDLMfhasCXhS9WzOR63=VFzvxg7v1UL6Vf_PLqfjZdurTFt6fDzRhlcLLrcRrkFvZ
x-5ku220jw-b: 3k9rur
x-5ku220jw-c: A__57lBqAQAACgDSJrJqFbzAqmJVRyggwSLzliwMgtTT0dRA3TtiKWkLmSk8AawUHfD6K-8VrJMAAOfvAAAAAA==
x-5ku220jw-d: 0
x-5ku220jw-uniquestatekey: A-gE71BqAQAAbcLSHXYa-TmPShEFZzL7WRKJL2s2ZrEiksXMK8pm-YimkgmLAawUHfGuclcCwH8AABszAAAAAA==

Я не знаю, откуда они приходят и как их получить, но код не работает без них (или одного из них). Я уверен в этом, потому что могу получить ответ, когда отправляю запросы на другие URL-адреса с сайта, где не нужно использовать эти заголовки.

Итак, вопрос в следующем: 1. Что это за заголовки? 2. Как (если это возможно) получить значения для этих заголовков?

Мой код:


import requests


data_url = 'https://www.southwest.com/api/air-booking/v1/air-booking/page/air/low-fare-calendar/select-dates'
payload = {
            "adultPassengersCount": "1",
            "currencyCode": "USD",
            "departureDate": "2019-04-25",
            "destinationAirportCode": "MCO",
            "originationAirportCode": "MDW",
            "passengerType": "ADULT",
            "returnAirportCode": "",
            "returnDate": "",
            "tripType": "oneway",
            "application": "air-low-fare-calendar",
            "site": "southwest"
            }
headers = {
            'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36',
            'x-api-key': 'l7xx944d175ea25f4b9c903a583ea82a1c4c'
            }


r = requests.post(data_url, json=payload, headers=headers)
print(r.text)
...