Ошибка при попытке запросить страницу входа через scrapy shell - PullRequest
Новая
       12

Ошибка при попытке запросить страницу входа через scrapy shell

0 голосов
/ 13 декабря 2018

Я все еще изучаю скрап.Но я проверяю свои звонки через scrapy shell, чтобы узнать, как взаимодействовать с формой входа на seek.com

, если вы визуально набираете seeking.com/login

, вы получаете логинформа, но делает запрос на нее, не показывает никаких данных формы, но после сетевых данных я заметил, что когда вы нажимаете кнопку «Войти», она перенаправляет на: 

https://www.seeking.com/v3/auth/login

и имеет формуполезная нагрузка для входа в систему: 

Request URL: https://www.seeking.com/v3/auth/login
Request Method: POST
Status Code: 200 

referer: https://www.seeking.com/login

{"captcha":"","email":"MYEMAIL.com","password":"MY_PASSWORD","is_rememberme":0,"locale":"en_US","auth_type":"bearer_token","date":"2018-12-13T09:56:22.957Z"}

, поэтому я думаю начать с оболочки, выполнить запрос POST с данными формы, но я не вхожу в систему и не знаю, как двигаться дальше и войти.

вот HTML для seeking.com/login:

========= 

    <html>
    <head>
    <base href="/">
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1, 
    maximum-scale=1, user-scalable=0">
    <meta name="apple-mobile-web-app-title" content="Seeking">
    <meta name="apple-mobile-web-app-status-bar-style" content="white">
    <meta name="apple-mobile-web-app-capable" content="yes" />


    <!-- sentry -->
    <script src="https://cdn.ravenjs.com/3.17.0/raven.min.js" crossorigin="anonymous"></script>
    <script>
      var env = 'production';
      var app_version = '3.57';
      if(window.location.host === "www.seeking.com") {
        env = "production";
      } else if(window.location.hostname === "local.seeking.com") {
        env = "local";
      }

      if(env === "production") {
        Raven.config('https://ff698d81c0a74ade8e38295b5491f546@sentry.infostreamgroup.com/11', {
          environment: env,
          release: app_version,
          collectWindowErrors: false,
          whitelistUrls: [ /https?:\/\/(([a-zA-Z0-9-]*)\.)?seeking\.com/g]
        }).install()

        var getState = localStorage === undefined ? "" : localStorage.getItem("react-state");
        var state = JSON.parse(getState);
        if(state && state.user) {
          Raven.setUserContext({
            username: state.user.username,
            id: state.user.user_id
          });
        } else {
          Raven.setUserContext();
        }

      }
    </script>

      <script>
      var SITE_VERSION = '3.59';
      var APP_CSS = "/css/app.css?v=3.59.1"
      var APP_SIGN_UP_CSS = "/css/signup/app_signup.css?v=3.59.1"
    </script>
    <title>Seeking | Dating For Successful & Attractive | Seeking.com</title>
    <meta name="description" content="Login to your Seeking.com account and find relationships on your terms.">

    <!-- Semantic UI CSS -->
    <link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/semantic-ui/2.2.12/semantic.min.css"></link>

    <!-- Custom CSS -->
    <link id="app-css" rel="stylesheet" type="text/css" href="/css/app.css?v=3.59.1">

    <!-- Favicon -->
    <link rel='shortcut icon' type='image/x-icon' href='//d1k2wjau60zau8.cloudfront.net/static/favicon32.png'>

    <!-- Fonts -->
    <link href='//images.seeking.com/assets/css/SourceSansPro-400-300-600.css' rel='stylesheet' type='text/css'>

    <link rel="apple-touch-icon" href="images/icons/app/logo.png">
    <link rel="apple-touch-icon" sizes="152x152" href="images/icons/app/logo-152x152.png">
    <link rel="apple-touch-icon" sizes="180x180" href="images/icons/app/logo-180x180.png">
    <link rel="apple-touch-icon" sizes="167x167" href="images/icons/app/logo-167x167.png">


    <!-- iPhone X (1125px x 2436px) -->
    <link rel="apple-touch-startup-image" media="(device-width: 375px) and (device-height: 812px) and (-webkit-device-pixel-ratio: 3)" href="images/icons/app/apple_splash_1125.png">

    <!-- iPhone 8, 7, 6s, 6 (750px x 1334px) -->
    <link rel="apple-touch-startup-image" media="(device-width: 375px) and (device-height: 667px) and (-webkit-device-pixel-ratio: 2)" href="images/icons/app/apple_splash_750.png">

    <link href="images/icons/app/apple_splash_xr.png" media="(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 2)" rel="apple-touch-startup-image" />

    <link href="images/icons/app/iphoneplus_splash.png" media="(device-width: 414px) and (device-height: 736px) and (-webkit-device-pixel-ratio: 3)" rel="apple-touch-startup-image" />

    <link href="images/icons/app/iphonexsmax_splash.png" media="(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 3)" rel="apple-touch-startup-image" />

    <!-- iPhone 5 (640px x 1136px) -->
    <link rel="apple-touch-startup-image" media="(device-width: 320px) and (device-height: 568px) and (-webkit-device-pixel-ratio: 2)" href="images/icons/app/apple_splash_640 × 1136.png">

    <link href="images/icons/app/ipad_splash.png" media="(device-width: 768px) and (device-height: 1024px) and (-webkit-device-pixel-ratio: 2)" rel="apple-touch-startup-image" />
    <link href="images/icons/app/ipadpro1_splash.png" media="(device-width: 834px) and (device-height: 1112px) and (-webkit-device-pixel-ratio: 2)" rel="apple-touch-startup-image" />
    <link href="images/icons/app/ipadpro3_splash.png" media="(device-width: 834px) and (device-height: 1194px) and (-webkit-device-pixel-ratio: 2)" rel="apple-touch-startup-image" />
    <link href="images/icons/app/ipadpro2_splash.png" media="(device-width: 1024px) and (device-height: 1366px) and (-webkit-device-pixel-ratio: 2)" rel="apple-touch-startup-image" />

    <!-- Temporary inline styling until replaced with actual SCSS stying -->
    <style>
        .ProfileInfoCard {
            pointer-events: auto !important;
        }

        ::-webkit-scrollbar-track {
            display: none;
        }

        html {
            overflow-x: visible;
        }

        body {
            font-size: inherit;
        }

        a, a:hover {
            color: inherit;
        }

        a.btn--action:hover {
            color: #fff;
        }

        .dropdown {
            display: inline-block;
        }

        .dropdown__content {
            display: none;
            position: absolute;
            top: 60px;
            right: 25px;
            box-shadow: 0 2px 8px rgba(0, 0, 0, 0.2);
        }

        .dropdown--active .dropdown__content {
            display: block;
            z-index: 10001;
        }
    </style>

    <script>
      dataLayer = [];
    </script>
    <script type="text/javascript">
      var hasoffers_domain = "https://reflexmedia.go2cloud.org";
    </script>
    <script type="text/javascript" src="https://media.go2app.org/assets/js/dl.js"></script>

            <!-- Google Tag Manager -->
        <script>(function (w, d, s, l, i) {
            w[l] = w[l] || [];
            w[l].push({
              'gtm.start':
                new Date().getTime(), event: 'gtm.js'
            });
            var f = d.getElementsByTagName(s)[0],
              j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : '';
            j.async = true;
            j.src =
              'https://www.googletagmanager.com/gtm.js?id=' + i + dl;
            f.parentNode.insertBefore(j, f);
          })(window, document, 'script', 'dataLayer', 'GTM-K2VSLZ');</script>
        <!-- End Google Tag Manager -->

    <script>
                sessionStorage.removeItem('join_inputValues');
            </script>
</head>
<body>

<div id="reactApp"></div>

<script type="text/javascript" src="js/bundle/i18n/i18n-979b1c60ad1b6f6ab516.js" charset="utf-8" onerror="Raven.captureException(new ReferenceError('missing i18n js : ' + src))"></script>
<script type="text/javascript" src="js/dll/dll.vendor-744864960cb9830c9faf.js" charset="utf-8" onerror="Raven.captureException(new ReferenceError('missing dll vendor js : ' + src))"></script>
<script type="text/javascript" src="js/bundle/production/bundle-8f9d74342bd267198f48.js" charset="utf-8" onerror="Raven.captureException(new ReferenceError('missing bundle js : ' + src))"></script>

    <!-- Google Tag Manager (noscript) -->
    <noscript>
        <iframe src="https://www.googletagmanager.com/ns.html?id=GTM-K2VSLZ" height="0" width="0"
                style="display:none;visibility:hidden"></iframe>
    </noscript>
    <!-- End Google Tag Manager (noscript) -->

</body>
</html>

===================

вот моя последовательность скорлупы 

In [1]: from scrapy.http import FormRequest

In [2]: frmdata = {"captcha":"","email":"MYEMAIL","password":"MYPASS","is_rememberme":0,"locale":"en_US","auth_type":"bearer_token","date":"2018-12-13T09:56:22.957Z"}


In [3]: url = "https://www.seeking.com/v3/auth/login"

In [4]: r = FormRequest(url, formdata=frmdata)

====== *

Я получаю эту ошибку после того, какпозвоните FormRequest: 

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/scrapy/http/request/form.py", line 31, in __init__
    querystr = _urlencode(items, self.encoding)
  File "/usr/local/lib/python2.7/site-packages/scrapy/http/request/form.py", line 67, in _urlencode
    for v in (vs if is_listlike(vs) else [vs])]
  File "/usr/local/lib/python2.7/site-packages/scrapy/utils/python.py", line 119, in to_bytes
    'object, got %s' % type(text).__name__)
TypeError: to_bytes must receive a unicode, str or bytes object, got int

==================

что я делаю не так?Хотелось бы получить вклад, спасибо.

1 Ответ

0 голосов
/ 13 декабря 2018

В formdata вы передаете is_rememberme как целое число (0), попробуйте поместить его в виде строки: 

frmdata = {"captcha":"","email":"MYEMAIL","password":"MYPASS","is_rememberme":"0","locale":"en_US","auth_type":"bearer_token","date":"2018-12-13T09:56:22.957Z"}
Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.

Нет похожих вопросов

...