Я пытался получить html-содержимое этой страницы 'https://webcache.gmc -uk.org / gmclrmp_enu / start.swe ' с помощью Robobrowser.вот код:
from robobrowser import RoboBrowser
landing_page_url = 'https://webcache.gmc-uk.org/gmclrmp_enu/start.swe'
browser = RoboBrowser(user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36')
browser.open(landing_page_url)
print(str(browser.parsed))
что я получаю из этого:
<html><body><form action="/gmclrmp_enu/start.swe" method="GET" name="RedirectForHost"><input name="SWECmd" type="hidden" value="Login"/>
<input name="SWEBHWND" type="hidden" value=""/>
<input name="_sn" type="hidden" value="THyz1L6X6J4NsE50rdbFd1FK6G5QhyWSLpaqOIT2U2IqKVWLjsZa7o3plt.FZpEoxV0LT2-0ttd3yjCGxI6DduqQ25v-mQZ12T61jKnDwJzm6kNqqH5UM3KVzeg1wr1U6.s1nxPge7cyz67EZiLw3IMIFn2ZnEaaK8AznGyFU1bjE7k7xj-KEJKs.QI9vpXZ9f4euEgOmmA_"/>
<input name="SRN" type="hidden" value=""/>
<input name="SWEHo" type="hidden" value=""/>
<input name="SWETS" type="hidden" value="1543090428"/>
</form><script language="javascript">var formObj = document.forms["RedirectForHost"];formObj.SWEHo.value=top.location.hostname;formObj.submit();</script></body></html>
то, что должно дать мне, выглядит примерно так:
<HTML lang="en"><head><title>List of Registered Medical Practitioners | Doctor Search</title><script>(function(h,o,t,j,a,r){
h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
h._hjSettings={hjid:782750,hjsv:6};
a=o.getElementsByTagName('head')[0];
r=o.createElement('script');r.async=1;
r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
a.appendChild(r);
})(window,document,'https://static.hotjar.com/c/hotjar-','.js?sv=');</script><script>history.forward();
var sUserAgent = navigator.userAgent;
if (sUserAgent.indexOf("MSIE") > 0)
{
document.write('<link href="files/gmclrmpie.css" rel="stylesheet" type="text/css">');
document.execCommand("BackgroundImageCache",false,true);
}
else
{
document.write('<link href="files/gmclrmp.css" rel="stylesheet" type="text/css">');
}
</script>
<script language="javascript">
if (window.addEventListener)
{
window.addEventListener('load', attachFormSubmit, false);
}
else if (window.attachEvent)
{
window.attachEvent('onload', attachFormSubmit);
}
var oldSWESubmitForm;
function attachFormSubmit()
{
oldSWESubmitForm = SWESubmitForm;
SWESubmitForm = AlwaysFireBeforeSWESubmitForm;
}
function AlwaysFireBeforeSWESubmitForm(varA,varB,varC,varD)
{
var LoadingBox = document.getElementById("Loading");
LoadingBox.style.display = "block";
SWESubmitForm = oldSWESubmitForm;
SWESubmitForm(varA,varB,varC,varD);
}</script>
<script language="javascript" src="23048/scripts/swecommon.js"></script>
<script language="javascript" src="23048/scripts/swemessages_enu.js"></script>
<script language="javascript" src="23048/scripts/swecmn_li.js"></script>
<script language="javascript" src="srf1538672264_444/bscripts/all/controls_applet_gmc_web_health_provider_search_applet.js"></script>
</head><body bgcolor="#ffffff" leftmargin="0" topmargin="0" marginwidth="0" marginheight="0"><div id="Loading" style="width:300px;height:30px;position:absolute;left:330px;top:275px;display:none;text-align:left;vertical-align:text-center;padding:5px;padding-top:15px;margin:0px;background-color:white;font-color:black;filter:alpha(opacity=90);-moz-opacity: .9;opacity: .9;border-style:solid;border-color:lightgrey;"><img id=animatedLoading src="images/loading.gif" border=0 valign=center align=right padding=0> Loading...</div><table border="0" cellpadding="0" cellspacing="0" width="960"><tr><td class="skiptocontentlink"><a href="#PageContent" id="skiptoContent" tabindex="1" title="Skip to the page content" name="skiptoContent">Skip to content</a></td></tr><tr><td><table border="0" cellpadding="6" cellspacing="0" width="960"><tr><td align="left" valign="top"><img src="images/structure/strapline_dark_blue.png" align="top" title="Working with doctors Working for patients" alt="Working with doctors Working for patients" border="0"></td><td rowspan="2" align="right"><img src="images/GMClogo.gif" alt="General Medical Council" title="General Medical Council" border="0"></td></tr><tr><td class="LRMPHeaderHome"><h1>List of Registered Medical Practitioners</h1></td></tr></table></td></tr></table><script language="javascript">top.document.title = "List of Registered Medical Practitioners | Doctor Search";</script><table width="960" cellpadding="0" cellspacing="0" border="0"><tr><td><a name="SWEApplet2" id="SWEApplet2" href="#SWEApplet2" title="" tabindex="0"><span style= "height:1px; width:1px; position:absolute; overflow:hidden; top=10px; visibility:hidden;">Ignore Link</span></a>
<table border="0" cellpadding="0" cellspacing="0" width="960" class="GMCNavTable"><tr><td><div class="GMCNavMenu"><div class="GMCNavMenuItem"><a href='/GMCLRMP_enu/help/help.html#search_page' title="Opens in a new window" target="_blank" tabindex="3" tabindex=1997 id='s_2_1_0_0'>Help</a></div></div></td></tr></table></td></tr><tr><div id="PageContent"><td><a name="SWEApplet3" id="SWEApplet3" href="#SWEApplet3" title="" tabindex="0"><span style= "height:1px; width:1px; position:absolute; overflow:hidden; top=10px; visibility:hidden;">Ignore Link</span></a>
<form name="SWEForm3_0" method="post" action="/gmclrmp_enu/start.swe" >
<table border="0" cellpadding="8" width="960"><tr><td class="home"><h2>Doctor Search</h2></td></tr><tr><td class="tablebody"><p>The List of Registered Medical Practitioners lets you check the registration status of any doctor who is registered with us or who has been registered at any time since 20 October 2005.<br><br></p><p>You can also find information about doctors' fitness to practise history from the same date. This includes sanctions that were applied before 20 October 2005 but which were still active at that time. For example, it does show if a doctor was suspended from the register for six months from 18 October 2005.<br><br></p>
<p>It does not include sanctions from before 20 October 2005 that were not active at that time. For example, it would not show if a doctor was suspended from the register for six months between 1 January 2005 and 1 July 2005.<br><br></p>
<p>Please <a href="http://www.gmc-uk.org/about/contacts.asp" tabindex=3 target="_blank" title="opens in a new window">contact us</a> if you need information from before this time period.<br><br></p>
<p>You can find more information on what is or is not available online in our <a href="/GMCLRMP_enu/help/help.html#explanation" tabindex=3 target="_blank" title="opens in a new window">helptext</a>.<br><br></p>
<p>If you need help searching for a particular doctor, read our <a href="/GMCLRMP_enu/help/help.html#searching" tabindex=3 target="_blank" title="opens in a new window">helptext about doctors' names, reference numbers and other search advice.</a><br><br>
</td>
</tr>
<tr><td class="error"></td></tr>
</table>
<table class="tablebody" border="0" cellpadding="0" cellspacing="4" align="centre">
<ul>
<tr>
<td><li class="formlist"><label for="gmcrefnumber"><a href='/GMCLRMP_enu/help/help.html#doctor_id' title="Opens in a new window" target="_blank" tabindex="3">GMC Reference Number</a></label></li></td><td><nobr><input type="text" name='s_3_1_5_0' value='' id="gmcrefnumber" tabindex="3" style="width:140" id='s_3_1_5_0' tabindex=2997 maxlength="7"></nobr></td></tr><tr><td><li class="formlist"><label for="givenname"><a href='/GMCLRMP_enu/help/help.html#given_name' title="Opens in a new window" target="_blank" tabindex="3">Given Name</a></label></li></td><td><nobr><input type="text" name='s_3_1_3_0' value='' id="givenname" tabindex="3" style="width:140" id='s_3_1_3_0' tabindex=2997 maxlength="255"></nobr></td></tr><tr><td><li class="formlist"><label for="surname"><a href='/GMCLRMP_enu/help/help.html#surname' title="Opens in a new window" target="_blank" tabindex="3">Surname</a></label></li></td><td><nobr><input type="text" name='s_3_1_9_0' value='' id="surname" tabindex="3" style="width:140" id='s_3_1_9_0' tabindex=2997 maxlength="255"></nobr></td></tr><tr><td><li class="formlist"><label for="soundslikeflag"><a href='/GMCLRMP_enu/help/help.html#sounds_like' title="Opens in a new window" target="_blank" tabindex="-1">Sounds Like</a></label></li></td><td><nobr><input type="checkbox" name='60' value='Y' title = 'Sounds Like' id="soundslikeflag" tabindex="3" id='s_60_cb' tabindex=2997 ><input type="hidden" name='s_3_1_6_0' value='60' ></nobr></td></tr><tr><td><li class="formlist"><label for="gpflag"><a href='/GMCLRMP_enu/help/help.html#GPsOnly' title="Opens in a new window" target="_blank" tabindex="-1">Only doctors on the GP Register</a></label></li></td><td><nobr><input type="checkbox" name='40' value='Y' title = 'GP Flag' id="gpflag" tabindex="3" id='s_40_cb' tabindex=2997 ><input type="hidden" name='s_3_1_4_0' value='40' ></nobr></td></tr><tr><td><li class="formlist"><label for="gender"><a href='/GMCLRMP_enu/help/help.html#gender' title="Opens in a new window" target="_blank" tabindex="3">Gender</a></label></li></td><td><nobr><select name="s_3_1_7_0" id="gender" tabindex="3" style="width:80" id='s_3_1_7_0' tabindex=2997 >
<option value="" >-Select-
<option value="Man" >Man
<option value="Woman" >Woman
</select>
</nobr></td></tr></ul></table><table><tr><td><img src="images/spacer.gif" height="5" width="475" alt=""></td><td><span class="minibuttonOn"><a href='JavaScript:SWESubmitForm(document.SWEForm3_0,s_0,"s_3_1_10_0","VRId-1")' onclick='Edit__0__Control__ExecuteQuery__onclick(null, "s_3_1_10_0")' tabindex="3" tabindex=2997 id='s_3_1_10_0'>Search</a></span></td></tr></table><table cellpadding="8" width="100%"><tr><td class="tablebody"><a href='JavaScript:SWESubmitForm(document.SWEForm3_0,s_1,"s_3_1_8_0","VRId-1")' tabindex="3" tabindex=2997 id='s_3_1_8_0'>Search for more than one doctor using their GMC reference numbers</a></td></tr></table><input type="hidden" name='SWEFo' value='SWEForm3_0' >
<input type="hidden" name='SWEField' value='' >
<input type="hidden" name='SWENeedContext' value='true' >
<input type="hidden" name='SWENoHttpRedir' value='false' >
<input type="hidden" name='W' value='t' >
<input type="hidden" name='SWECmd' value='InvokeMethod' >
<input type="hidden" name='SWEMethod' value='Refresh' >
<input type="hidden" name='SWERowIds' value='' >
<input type="hidden" name='SWESP' value='false' >
<input type="hidden" name='SWEVI' value='' >
<input type="hidden" name='SWESPNR' value='' >
<input type="hidden" name='SWEPOC' value='' >
<input type="hidden" name='SWESPNH' value='' >
<input type="hidden" name='SWEH' value='' >
<input type="hidden" name='SWETargetView' value='' >
<input type="hidden" name='SWEDIC' value='false' >
<input type="hidden" name='_sn' value='Ap2CHEtE-yIqJQDq-nR9LuJy6SLpJkaa.5TMwNhmLfqKUAvZLcG5C1zupRv8yaRpWVQJRc3gTHvHNpaP9ui446RbQmhiuAXh3A8ZUoWYfRgoe2BOi5gquh2R69JD7eF5SdWGZ36RAP4n4itQANKWReoSAczi-uL3782qzwpiU4V5DxfZ1SpfNIZaf2FqHsifn3u.9wjUzkA_' >
<input type="hidden" name='SWEReqRowId' value='0' >
<input type="hidden" name='SWEView' value='GMC WEB Doctor Search' >
<input type="hidden" name='SWEC' value='2' >
<input type="hidden" name='SWERowId' value='VRId-1' >
<input type="hidden" name='SWETVI' value='' >
<input type="hidden" name='SWEW' value='' >
<input type="hidden" name='SWEBID' value='-1' >
<input type="hidden" name='SWEM' value='' >
<input type="hidden" name='SRN' value='' >
<input type="hidden" name='SWESPa' value='' >
<input type="hidden" name='SWETS' value='' >
<input type="hidden" name='SWEContainer' value='' >
<input type="hidden" name='SWEWN' value='' >
<input type="hidden" name='SWEKeepContext' value='0' >
<input type="hidden" name='SWEApplet' value='GMC WEB Health Provider Search Applet' >
<input type="hidden" name='SWETA' value='' >
</form>
</td></div></tr><tr><td><a name="SWEApplet1" id="SWEApplet1" href="#SWEApplet1" title="" tabindex="0"><span style= "height:1px; width:1px; position:absolute; overflow:hidden; top=10px; visibility:hidden;">Ignore Link</span></a>
<form name="SWEForm1_0" method="post" action="/gmclrmp_enu/start.swe" >
<table border="0" cellpadding="0" cellspacing="0" width="960"><tr><td><div class="siteTools"><ul><li class="browsealoud"><a href='http://www.gmc-uk.org/accessibility/browsealoud.asp' title="Opens in a new window" target="_blank" tabindex="3" tabindex=3997 id='s_1_1_0_0'>Browsealoud</a></li></ul></div></td></tr></table><input type="hidden" name='SWEField' value='' >
<input type="hidden" name='SWEFo' value='SWEForm1_0' >
<input type="hidden" name='SWENeedContext' value='true' >
<input type="hidden" name='SWESP' value='false' >
<input type="hidden" name='SWERowIds' value='' >
<input type="hidden" name='SWEMethod' value='Refresh' >
<input type="hidden" name='SWECmd' value='InvokeMethod' >
<input type="hidden" name='W' value='t' >
<input type="hidden" name='SWENoHttpRedir' value='false' >
<input type="hidden" name='SWEVI' value='' >
<input type="hidden" name='SWEPOC' value='' >
<input type="hidden" name='SWESPNR' value='' >
<input type="hidden" name='SWETargetView' value='' >
<input type="hidden" name='SWESPNH' value='' >
<input type="hidden" name='SWEH' value='' >
<input type="hidden" name='SWEDIC' value='false' >
<input type="hidden" name='SWEReqRowId' value='0' >
<input type="hidden" name='_sn' value='Ap2CHEtE-yIqJQDq-nR9LuJy6SLpJkaa.5TMwNhmLfqKUAvZLcG5C1zupRv8yaRpWVQJRc3gTHvHNpaP9ui446RbQmhiuAXh3A8ZUoWYfRgoe2BOi5gquh2R69JD7eF5SdWGZ36RAP4n4itQANKWReoSAczi-uL3782qzwpiU4V5DxfZ1SpfNIZaf2FqHsifn3u.9wjUzkA_' >
<input type="hidden" name='SWEView' value='GMC WEB Doctor Search' >
<input type="hidden" name='SWETVI' value='' >
<input type="hidden" name='SWERowId' value='1-CPQ4W3' >
<input type="hidden" name='SWEC' value='2' >
<input type="hidden" name='SWEM' value='' >
<input type="hidden" name='SWEBID' value='-1' >
<input type="hidden" name='SWEW' value='' >
<input type="hidden" name='SWESPa' value='' >
<input type="hidden" name='SRN' value='' >
<input type="hidden" name='SWEContainer' value='' >
<input type="hidden" name='SWETS' value='' >
<input type="hidden" name='SWETA' value='' >
<input type="hidden" name='SWEApplet' value='GMC LRMP Browse Aloud Form Applet' >
<input type="hidden" name='SWEWN' value='' >
</form>
<script>
top._swescript.SWESyncCheck(2, "You cannot continue to work on this page because the state of this page cannot be restored on the server.");
var s_0 = {action:"/gmclrmp_enu/start.swe#SWEApplet3",SWECmd:"InvokeMethod",SWEMethod:"NewQuerySearch",SWEView:"GMC WEB Doctor Search",SWEApplet:"GMC WEB Health Provider Search Applet",SWEReqRowId:"1",SWESP:"false",SWENeedContext:"true",SWEDIC:"false"};
var s_1 = {action:"/gmclrmp_enu/start.swe",target:"_sweview",SWECmd:"GotoView",SWEMethod:"GotoView",SWEView:"GMC WEB Doctor Multiple Search",SWEApplet:"GMC WEB Health Provider Search Applet",SWEReqRowId:"0",SWESP:"false",SWENeedContext:"true",SWEKeepContext:"0",SWEDIC:"false"};
function SWEDoRefresh(anchor) { location.replace('/gmclrmp_enu/start.swe?SWECmd=Refresh&SWEVI=&_sn=Ap2CHEtE-yIqJQDq-nR9LuJy6SLpJkaa.5TMwNhmLfqKUAvZLcG5C1zupRv8yaRpWVQJRc3gTHvHNpaP9ui446RbQmhiuAXh3A8ZUoWYfRgoe2BOi5gquh2R69JD7eF5SdWGZ36RAP4n4itQANKWReoSAczi-uL3782qzwpiU4V5DxfZ1SpfNIZaf2FqHsifn3u.9wjUzkA_&SWEView=GMC+WEB+Doctor+Search&SWEC=2&SRN=' + anchor); }
if (opener == null && top == window) { var SWEPopupWin = null; var SWEJannaPopupWin = null; }
if (typeof(Top().SWEHtmlPopupName) == 'undefined' || Top().SWEHtmlPopupName == null) Top().SWEHtmlPopupName = '_swe1543090536';
if (typeof(top._samePage) != 'undefined' && top._samePage!="") top._samePage = "";
Top().SWECount = 2;
if(typeof(Top().SWEServerCount)=='undefined' || Top().SWEServerCount<Top().SWECount) {Top().SWEServerCount = Top().SWECount;}
g_bInitialized = true;
if (top&&top._sweclient&&top._sweclient._sweviewbar) { top._sweclient._sweviewbar.location.replace('/gmclrmp_enu/start.swe?SWECmd=GetCachedFrame&_sn=Ap2CHEtE-yIqJQDq-nR9LuJy6SLpJkaa.5TMwNhmLfqKUAvZLcG5C1zupRv8yaRpWVQJRc3gTHvHNpaP9ui446RbQmhiuAXh3A8ZUoWYfRgoe2BOi5gquh2R69JD7eF5SdWGZ36RAP4n4itQANKWReoSAczi-uL3782qzwpiU4V5DxfZ1SpfNIZaf2FqHsifn3u.9wjUzkA_&SWEC=2&SWEFrame=top._sweclient._sweviewbar&SRN='); }
var ctrlLookupMap = new Array();
function FindControl (appletName, controlName) { return (ctrlLookupMap [appletName + "." + controlName]); }
var ctrlCS = new Array();if (Top()._swescript!=null && typeof(top._swescript)!='undefined') Top()._swescript.ctrlCS = ctrlCS;
else Top().ctrlCS = ctrlCS;
ctrlLookupMap["GMC WEB Health Provider Search Applet.GMC Person UId"] = "s_3_1_5_0";
ctrlLookupMap["GMC WEB Health Provider Search Applet.First Name"] = "s_3_1_3_0";
ctrlLookupMap["GMC WEB Health Provider Search Applet.Last Name"] = "s_3_1_9_0";
ctrlLookupMap["GMC WEB Health Provider Search Applet.GMC Sounds Like Flag"] = "s_3_1_6_0";
ctrlLookupMap["GMC WEB Health Provider Search Applet.GMC GP Flag"] = "s_3_1_4_0";
ctrlLookupMap["GMC WEB Health Provider Search Applet.Gender"] = "s_3_1_7_0";
</script>
<script for=window event=onunload>
ClearTimer();
if (opener == null) { SWEClosePopup(); SWECloseJannaPopup(); }
</script>
<script for=window event=onload>
StartTimer(540000,1,0,10,0);
</script>
</td></tr></table><table border="0" cellpadding="0" cellspacing="0" width="960"><tr><td><hr></td></tr><tr><td><div class="footerLeft"><p>The GMC is a registered charity in England and Wales (1089278) and Scotland (SC037750).<br/>© Copyright General Medical Council 2010. All rights reserved</p></div><h2 class="hide">Disclaimer and Privacy Statement</h2><div class="footerRight"><ul><li class="first"><a href="http://www.gmc-uk.org/disclaimer.asp" title="Opens in a new window" tabindex="98" target="_blank">Disclaimer</a></li><li><a href="http://www.gmc-uk.org/privacy_policy.asp" title="Opens in a new window" tabindex="99" target="_blank">Privacy Statement</a></li></ul></div></td></tr></table>
Итак, я имею в виду, что на самом деле я не могу попасть на целевую страницу.Может кто-нибудь помочь мне туда добраться?Спасибо!