Переключите pageLoadStrategy с нетерпеливого на обычный для существующего драйвера - PullRequest
Новая
       33

Переключите pageLoadStrategy с нетерпеливого на обычный для существующего драйвера

0 голосов
/ 24 января 2020

Я установил pageLoadStrategy как eager, но теперь метод, который я написал для закрытия всплывающих окон веб-сайта, не работает (он работает, когда pageLoadStrategy равно normal). Итак, Я хочу знать, как изменить pageLoadStrategy с eager на normal, закрыть всплывающие окна, а затем изменить его на eager Я хочу использовать eager так, как хочу Мой код для работы также на низкоскоростных соединениях.

вот содержимое страницы, взятой driver.page_source, когда для pageLoadStrategy установлено значение eager: 

 <head>
  <meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/>
  <meta charset="utf-8"/>
  <title>
   Amizone
  </title>
  <meta content="overview &amp; stats" name="description"/>
  <meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0" name="viewport"/>
  <!-- bootstrap & fontawesome -->
  <link href="/Content/bootstrap.min.css" rel="stylesheet"/>
  <link href="/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/>
  <link href="/Content/color.css" rel="stylesheet"/>
  <!-- page specific plugin styles -->
  <!-- text fonts -->
  <link href="https://fonts.googleapis.com/css?family=Poppins:300,400,500,600,700" rel="stylesheet" type="text/css"/>
  <!-- ace styles -->
  <link href="/Content/ace.min.css" rel="stylesheet"/>
  <link href="/Content/Dashboard.css" rel="stylesheet"/>
  <link href="/assets/bootstrap-datepicker.min.css" rel="stylesheet"/>
  <!-- end css for this page-->
  <script src="/Scripts/jquery.2.1.1.min.js">
  </script>
 </head>
</html>

это часть содержимого страницы (которое я хочу) в тот же момент, когда pageLoadStrategy установлен на normal: 

 <head>
  <meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/>
  <meta charset="utf-8"/>
  <title>
   Amizone
  </title>
  <meta content="overview &amp; stats" name="description"/>
  <meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0" name="viewport"/>
  <!-- bootstrap & fontawesome -->
  <link href="/Content/bootstrap.min.css" rel="stylesheet"/>
  <link href="/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/>
  <link href="/Content/color.css" rel="stylesheet"/>
  <!-- page specific plugin styles -->
  <!-- text fonts -->
  <link href="https://fonts.googleapis.com/css?family=Poppins:300,400,500,600,700" rel="stylesheet" type="text/css"/>
  <!-- ace styles -->
  <link href="/Content/ace.min.css" rel="stylesheet"/>
  <link href="/Content/Dashboard.css" rel="stylesheet"/>
  <link href="/assets/bootstrap-datepicker.min.css" rel="stylesheet"/>
  <!-- end css for this page-->
  <script src="/Scripts/jquery.2.1.1.min.js">
  </script>
  <script src="/Scripts/jquery.unobtrusive-ajax.min.js">
  </script>
  <script src="/Scripts/ace-extra.min.js">
  </script>
  <script src="/Scripts/bootstrap.min.js">
  </script>
  <script src="/Scripts/bootbox.min.js">
  </script>
  <!-- page calender plugin scripts -->
  <!--start  ace scripts -->
  <script src="/Scripts/ace-elements.min.js">
  </script>
  <script src="/Scripts/ace.min.js">
  </script>
  <script src="/Scripts/jquery.easypiechart.min.js">
  </script>
  <script src="/assets/animate-plus.min.js">
  </script>
  <script src="/assets/owl.carousel.js">
  </script>
  <script src="/assets/bootstrap-datepicker.min.js">
  </script>
  <script src="/Scripts/form-wizard.js">
  </script>
  <script src="/Scripts/validator.js">
  </script>
  <link href="/Content/color.css" rel="stylesheet"/>
  <script src="/Scripts/jquery.colorbox.min.js">
  </script>
  <link href="/Content/colorbox.min.css" rel="stylesheet"/>
  <!-- include the style -->
  <link href="/Content/alertifyjs/alertify.min.css" rel="stylesheet"/>
  <!-- include a theme -->
  <link href="/Content/alertifyjs/themes/default.min.css" rel="stylesheet"/>
  <script src="/Scripts/alertify.js">
  </script>
  <link href="/Content/main.css" rel="stylesheet"/>
  <style>
   #myDiv {
            visibility: hidden;
            opacity: 0;
        }
  </style>
  <script>
   alertify.set('notifier', 'position', 'top-right');
  </script>
  <script>
   var baseurl = "";
  </script>
  <script>
   var myVar;
        function myFunction() {
            $("#lodingDiv").css("display", "block");
            myVar = setTimeout(showPage, 300);
        }
        function showPage() {
            $("#lodingDiv").css("display", "none");
            $("#myDiv").css("visibility", "visible");
            $("#myDiv").css("opacity", "1");
        }
  </script>
  <script charset="UTF-8" src="https://www.gstatic.com/charts/46.1/loader.js" type="text/javascript">
  </script>
  <link href="https://www.gstatic.com/charts/46.1/css/core/tooltip.css" id="load-css-0" rel="stylesheet" type="text/css"/>
  <link href="https://www.gstatic.com/charts/46.1/css/util/util.css" id="load-css-1" rel="stylesheet" type="text/css"/>
  <script charset="UTF-8" src="https://www.gstatic.com/charts/46.1/js/jsapi_compiled_format_module.js" type="text/javascript">
  </script>
  <script charset="UTF-8" src="https://www.gstatic.com/charts/46.1/js/jsapi_compiled_default_module.js" type="text/javascript">
  </script>
  <script charset="UTF-8" src="https://www.gstatic.com/charts/46.1/js/jsapi_compiled_ui_module.js" type="text/javascript">
  </script>
  <script charset="UTF-8" src="https://www.gstatic.com/charts/46.1/js/jsapi_compiled_corechart_module.js" type="text/javascript">
  </script>
 </head>
 <body class="no-skin modal-open" oncontextmenu="return false;" onload="myFunction()" style="padding-right: 15px;">
  <div id="lodingDiv" style="display: block;">
   <div id="loader">
   </div>
  </div>
  <script>
   function beginReadNotificationNav(stype, iNoticeId) {
        var hfvalue = $('#Hf' + stype + '_' + iNoticeId).val();
        var count = $('#PendHome' + stype + 's').text();
        var actualCount = parseInt($.trim(count));
        if (hfvalue == "False" && actualCount > 0) {
            $('#PendHome' + stype + 's').text(actualCount - 1);
            $('#PendHome1' + stype + 's').text(actualCount - 1);
            $('#Pend' + stype + 's').text(actualCount - 1);
            $('#Hf' + stype + '_' + iNoticeId).val('True');
        }
    }
  </script>
  <script>
   function imageExists(url, callback) {
    var img = new Image();
    img.onload = function() { callback(true); };
    img.onerror = function() { callback(false); };
    img.src = url;
  }


  function validateImageURL()
    {

      var imageUrl = 'https://amizone.net/amizone/Images/Signatures/7071804_P.png';

      imageExists(imageUrl, function(exists) {
          //Show the result
         // alert('Fileexists=' + exists);
          var html='';  
                if (exists)
          {
                    html= '<img class="nav-user-photo" src="https://amizone.net/amizone/Images/Signatures/7071804_P.png"/>';
             }
              else
              {
                    html = '<img class="nav-user-photo" src="../Images/blankphoto.png"  />';
              }
         // alert(html);
          $('#userphoto').append(html);

      });

  }

    validateImageURL();
  </script>
  <style>
   .margin-left-20{margin-left:20px}
    .top-notice li {
        line-height: 20px;
        overflow: hidden;
        text-overflow: ellipsis;
        white-space: nowrap;
    }

    .Nav-Note-Title {
        font-size: 13px;
        color: #0eb2a4;
    }

    .Nav-Note-Date {
        font-size: 12px;
        color: #18960a;
    }

    .Nav-Note-View a {
        font-size: 11px;
        color: #fff;
        text-decoration: none;
        padding: 2px 10px;
        border-radius: 10px;
    }
    @media(min-width:767px) {

        .modal-close-btn-home {
        display:none;
        }
    }
  </style>
  <div class="navbar navbar-default ace-save-state navbar-fixed-top" id="navbar">
   <script type="text/javascript">
    try { ace.settings.check('navbar', 'fixed') } catch (e) { }
   </script>
   <div class="navbar-container" id="navbar-container">
    <button class="navbar-toggle menu-toggler pull-left" data-target="#sidebar" id="menu-toggler" type="button">
     <span class="sr-only">
      Toggle sidebar
     </span>
     <span class="icon-bar">
     </span>
     <span class="icon-bar">
     </span>
     <span class="icon-bar">
     </span>
    </button>
    <div class="navbar-header pull-left">
     <a class="navbar-brand" href="/Home/">
      <img src="/images/amizone-logo-inner.png"/>
     </a>
    </div>
    <div class="pull-left">
     <h4 class="align-middle white margin-left-20">
      Amity University Uttar Pradesh, Noida
     </h4>
    </div>
    <div class="navbar-buttons pull-right" role="navigation">
     <ul class="nav ace-nav">
      <li class="grey dropdown-modal">
       <a aria-expanded="false" class="dropdown-toggle" data-toggle="dropdown" href="#">
        <i class="ace-icon fa fa-bell">
        </i>
        <span class="badge badge-grey" id="PendHomeNotifications">
         10
        </span>
       </a>
       <ul class="dropdown-menu-right dropdown-navbar dropdown-menu dropdown-caret dropdown-close">
        <li class="dropdown-header">
         <i class="ace-icon fa fa-check">
         </i>
         <span id="PendHome1Notifications">
          10
         </span>
         Notices To Read
         <button aria-label="Close" class="close modal-close-btn-home" data-dismiss="modal" type="button">
          <span aria-hidden="true">
           ×
          </span>
         </button>
        </li>
        <li class="dropdown-content ace-scroll" style="position: relative;">
         <div class="scroll-track" style="display: none;">
          <div class="scroll-bar">
          </div>
         </div>
         <div class="scroll-content" style="max-height: 200px;">
          <div class="scroll-content" style="">
           <ul class="dropdown-menu dropdown-navbar top-notice">
            <li>
             <input id="HfNotification_6699" type="hidden" value="False"/>
             <span class="Nav-Note-Title">
              NOTICE FOR ALL -- LOST &amp; FOUND
             </span>
             <div class="clearfix">
              <span class="pull-left Nav-Note-Date">
               13 Jan 2020
              </span>
              <span class="pull-right Nav-Note-View">
               <a class="bg-b-blue" data-ajax="true" data-ajax-begin="beginReadNotificationNav('Notification','6699');" data-ajax-loading="#lodingDiv" data-ajax-method="GET" data-ajax-mode="replace" data-ajax-success=" $('#FormModal').modal('show');" data-ajax-update="#DivForm" href="/Home/NoticeDescription/6787102C-D0F6-4AF2-8E58-4D6F9FF3D4C8?Type=2" id="6787102C-D0F6-4AF2-8E58-4D6F9FF3D4C8" rel="0">
                View
               </a>
              </span>
             </div>
            </li>
            <li>
             <input id="HfNotification_6700" type="hidden" value="False"/>
             <span class="Nav-Note-Title">
              COMPLETE BAN ON E-CIGARETTES
             </span>
             <div class="clearfix">
              <span class="pull-left Nav-Note-Date">
               10 Jan 2020
              </span>
              <span class="pull-right Nav-Note-View">
               <a class="bg-b-blue" data-ajax="true" data-ajax-begin="beginReadNotificationNav('Notification','6700');" data-ajax-loading="#lodingDiv" data-ajax-method="GET" data-ajax-mode="replace" data-ajax-success=" $('#FormModal').modal('show');" data-ajax-update="#DivForm" href="/Home/NoticeDescription/F81A5BB8-68BE-4B92-8DD0-2A76F1880197?Type=2" id="F81A5BB8-68BE-4B92-8DD0-2A76F1880197" rel="0">
                View
               </a>
              </span>
             </div>
            </li>
            <li>```

this is the python automation code:

#import sys
import time
import db
from bs4 import BeautifulSoup
from selenium.common import exceptions
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
#from selenium.webdriver.support import expected_conditions as EC

start_time = time.time()    #stores time at which program starts
#while(True):

#***setting up chrome driver***
caps = DesiredCapabilities().CHROME
caps["pageLoadStrategy"] = "normal"  #complete
#caps["pageLoadStrategy"] = "eager"  #interactive
#caps["pageLoadStrategy"] = "none"
chromedriver = "/usr/share/chromedriver/chromedriver"

driver = webdriver.Chrome(desired_capabilities=caps,executable_path=chromedriver)
driver.set_window_size(800, 1000)
# driver.set_network_conditions(
#     offline=False,
#     latency=5,  # additional latency (ms)
#     download_throughput=500 * 1024,  # maximal throughput
#     upload_throughput=500 * 1024)  # maximal throughput
#driver.maximize_window()
#wait = WebDriverWait(driver, 10)
wait = driver.implicitly_wait(10)
url = "https://student.amizone.net"
driver.get(url) #getting amizone.net

#***write page content to a file and return page soup***
def page_content_to_file(*argsv):
    wait
    content = driver.page_source
    #print(content)
    page_soup = BeautifulSoup(content, "html.parser")
    page_soup_text = BeautifulSoup.prettify(page_soup)
    if(len(argsv) > 1):
        raise NameError('page_content_to_file cannot take more than 2 arguments')
    if(len(argsv) == 1):
        filename = argsv[0]
        with open(filename, "w") as file:
            file.write(page_soup_text)
            print("wrote to file {}".format(filename))
    return page_soup

#***function that enters login credentials***
def login(username, password):
    try:
    #type | name=_UserName
        driver.find_element(By.NAME, "_UserName").send_keys(username)
    #type | name=_Password
        driver.find_element(By.NAME, "_Password").send_keys(password)
    #click | css=#loginform .login100-form-btn |  
        driver.find_element(By.CSS_SELECTOR, "#loginform .login100-form-btn").click()
    except:
        print("couldn't complete login")

#***function to close popups***
def close_popups():
    #try:
    page_soup = page_content_to_file("popup.html")
    #   getting names of divs having class 'modal fade in'
        # driver.implicitly_wait(10)
        # content = driver.page_source
        # page_soup = BeautifulSoup(content,"html.parser")
    popup_divs = page_soup.find_all('div', {"class":"modal fade in"})
#print(popup_divs)
    popups_name = []
    for div in popup_divs:
        popups_name.append(div['id'])
        print(popups_name)
    if(len(popups_name) == 0):
        print("no popups found popups_name length=0")
    else:
        print("starting")
    #   clicking to close pop-ups
        for name in reversed(popups_name):
            xpath = "//div[@id='" + name + "']//button[@class='close']"
            print(xpath)
            driver.find_element(By.XPATH, xpath).click()
        print("clicks complete")
            #extra code
                #wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#ModalPopAmityHostel button.btn"))).click()
                #driver.execute_script("arguments[0].click()", driver.find_element_by_css_selector("#StudentSatisfactionPop button.btn"))
                #click | id=ModalPopAmityHostel |  
                #driver.find_element(By.XPATH, "//div[@id='ModalPopAmityHostel']//button[@class='close']").click()
                #click | id=StudentSatisfactionPop | 
                #driver.find_element(By.XPATH, "//div[@id='StudentSatisfactionPop']//button[@class='close']").click()
    # except:
        # print("error occured while closing popups")
    return 0

#----------FUNCTION CALLS----------
login("username", "password") #login
close_popups() #close all popups
#----------------------------------

#***go to next/prev date in myClasses***
page_soup = page_content_to_file("amizone.html")
date_prev_next = ""
while(date_prev_next != "end"):
    date_prev_next = input("type prev/next:")
    try:
        if(date_prev_next == "next"):
            #click | css=.fc-icon-right-single-arrow |
            driver.find_element(By.CSS_SELECTOR, ".fc-icon-right-single-arrow").click()
            print(driver.find_element(By.XPATH, "//*[@id='calendar']/div[1]/div[3]/h2").text)
        if(date_prev_next == "prev"):
            # click | css=.fc-prev-button |
            driver.find_element(By.CSS_SELECTOR, ".fc-prev-button").click()
            print(driver.find_element(By.XPATH, "//*[@id='calendar']/div[1]/div[3]/h2").text)
    except exceptions.NoSuchElementException as e:
        print(e, "unable to click. Something may be blocking the element")

#***clicking on the hamburger button and choosing timetable***
def menu_click(option):
    menu_toggler_xpath = "//*[@id='menu-toggler']" #xpath of the menu toggler(hamburger button)
    #   clicking on the hamburger button on top left corner
    driver.find_element(By.XPATH, menu_toggler_xpath).click()
    if(option == "timetable"):
        time_table_navbar_xpath = "//*[@id='10']" #xpath of the timetable option in menu
        #   clicking on the timetable button in the menu
        driver.find_element(By.XPATH, time_table_navbar_xpath).click()
        print("clicked on timetable.")
        wait

menu_click("timetable")
#***scraping timetable***
# print(driver.find_element(By.CLASS_NAME, "tab-content").text)
#NOTE: no need to click on weekdays because all info is in the webpage
#NOTE: just clicking on the tt loads only the current day's tt. if then you click once on any day webpage shows tt of whole week.(why?)
#for i in range(1,8): #iterating over all weekdays 1-7
    #weekday_xpath = "//*[@id='myTab3']/li[" + str(i) + "]/a" #concatenating string to make xpath for each weekday
#clicking on a day to get whole week's tt
page_soup = page_content_to_file("01.html")
weekday_xpath = "//*[@id='myTab3']/li[1]/a" #xpath of day no.1 of the week in the timetable at that time.
driver.find_element(By.XPATH, weekday_xpath).click()

#   get info about classes and attendance marked from myclasses
#TODO: for a course check whether green or blue dot is shown

#extra code
    #date = driver.find_element(By.XPATH, "//*[@id='calendar']/div[1]/div[3]/h2").text #to get the date of myClasses
    #print(calendar_date_element.text)
    #driver.find_element(By.ID, "ModalPopAmityHostel").click()
    #time.sleep(5)
    #driver.implicitly_wait(5000)
    #driver.find_element(By.ID, "StudentSatisfactionPop").click()
    #url = driver.current_url
    #driver.find_element(By.CLASS_NAME, "close").click()
    #driver.implicitly_wait(5000)
period_data = [] #list to make sql statement
#scraping timetable data
page_soup = page_content_to_file()
divs_class_tab_pane = page_soup.findAll("div", {"class":"tab-pane"})  #finds and makes a list all the <div class="tab-pane in active" id="[day]">
for day_div in divs_class_tab_pane:    #selects each day's div from divs_class_tab_pane list
    print()
    day = day_div["id"].strip()    #gets the id attribute of div tag e.g <div class="tab-pane in active" id="Sunday"> returns the day
    print(day)
    try:
        #find all <div class="thumbnail timetable-box"> elements which contains p tags of details of a class
        div_thumbnail_timetable_box = day_div.findAll("div", {"class":"thumbnail timetable-box"})
        if(len(div_thumbnail_timetable_box) == 0):
            print("no classes alloted yet")
    except:
        print("no classes today")
    #selecting element one at a time from div_thumbnail_timetable_box
    for ttbox in div_thumbnail_timetable_box:   
        period_data.append(day) #appending day to list
        print()
        #get text from <p class="class-time"> the class time
        class_time = ttbox.find('p', {"class":"class-time"}).text.strip()
        period_data.append(class_time) #appending class time to list
        print(class_time)
        #get text from <p class="course-code"> the course code
        course_code = ttbox.find('p', {"class":"course-code"}).text.strip()
        period_data.append(course_code) #appending course_code to list
        print(course_code)
        #get text from <p class="course-teacher"> the course teacher
        course_teacher = ttbox.find('p', {"class":"course-teacher"}).text.strip()
        period_data.append(course_teacher) #appending course_teacher to list
        print(course_teacher)
        class_location = ttbox.find('p', {"class":"class-loc"}).text.strip()
        period_data.append(class_location)
        print(class_location)
        print(period_data)

        #connecting to database #TODO: exception handliling required here
        mydb = db.establish_con("localhost", "manik", "sweetbread","amizone")
        script = "','".join(period_data)
        period_data.clear()
        query = "INSERT INTO amizone.tt_data(`day`,`time`,course,teacher, class_loc) VALUES ('" + script + "');"
        #running MySQL query in the database
        mycursor = db.run_sql(mydb, query)
        #mycursor = db.run_sql(mydb, "SELECT * FROM amizone.tt_data;")
        mydb.commit()
#driver.quit()
print("execution time: %ss" % (round(time.time() - start_time, 5)))
#    time.sleep(10)```

1 Ответ

0 голосов
/ 25 января 2020

Вы должны рассмотреть несколько вещей следующим образом:

  • Когда вы устанавливаете pageLoadStrategy как eager до DesiredCapabilities() эта конфигурация запекается в исполняемый файл chromedriver и сохраняется до времени жизни экземпляра WebDriver . Таким образом, вы не можете изменить pageLoadStrategy ни с eager на normal, ни наоборот, пока выполняется тест В процессе .

  • Вы можете найти пару соответствующих обсуждений в:

  • Сравнение Page Source , полученного с помощью варианта WebDriver с использованием обоих pageLoadStrategies you будет наблюдать:

    • Источник страницы с pageLoadStrategy как eager не содержит jQuery и AJAX cals: 

      <script src="/Scripts/jquery.2.1.1.min.js">
</script>
<script src="/Scripts/jquery.unobtrusive-ajax.min.js">
</script>

    • Следовательно, существует более высокая вероятность того, что вариант WebDriver будет пытаться взаимодействовать, например, click() , send_keys() даже до того, как JavaScript зарегистрирует различные HTML события DOM например, обработчики событий, для элементов с HTML документ, который в итоге приведет к click() сбоям .

Это точная причина всплывающего окна ... not закрываясь ... .

Решение

Из источника страницы с pageLoadStrategy как нетерпеливым , это Совершенно очевидно, что элементы на веб-странице - это элементы с поддержкой JavaScript, поэтому лучше использовать pageLoadStrategy как normal при выполнении теста.

...