Веб-парсинг веб-страницы Jetbrains через python - PullRequest
0 голосов
/ 03 августа 2020

Я хочу получить данные из моей учетной записи администратора Jetbrains для управления лицензиями. Мой код следующий:

import requests
import pyotp
from lxml import html
from bs4 import BeautifulSoup as soup


USERNAME = "username"
PASSWORD = "password"
APP = "Jetbrains IntelliJ"

LOGIN_URL = "https://account.jetbrains.com/login"
URL = "https://account.jetbrains.com/organization/254678"
mfa = pyotp.TOTP('S8I9ASC67C4H9HQE')
token = mfa.now()
#print(token)


def main():
    session_requests = requests.session()

    # Create payload
    payload = {
        "username": USERNAME, 
        "password": PASSWORD, 
        "secondFactor": token
    }

    # Perform login
    result = session_requests.post(LOGIN_URL, data = payload, headers = dict(referer = LOGIN_URL))

    # Scrape url
    result = session_requests.get(URL, headers = dict(referer = URL))
    print(result.text)

if __name__ == '__main__':
    main()

Когда я печатаю токен, ошибок нет. Но когда я пытаюсь распечатать результат, ошибок по-прежнему нет, я не могу войти в консоль администратора. Откроется страница входа в систему.

Результат:

<!DOCTYPE html>

<html>
<head>
<title>JetBrains Account</title>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1.0" name="viewport"/>
<meta content="origin-when-cross-origin" name="referrer"/>
<link href="https://resources.jetbrains.com/storage/ui/fonts/fonts.css" rel="stylesheet" type="text/css"/>
<link href="/appresources/jquery.dataTables.css?v=c2ef715" rel="stylesheet" type="text/css"/>
<link href="/appresources/markside.css?v=a15765bc" rel="stylesheet" type="text/css"/>
<link href="/appresources/bootstrap-full.less?v=ef882a8b" rel="stylesheet" type="text/css"/>
<link href="/appresources/datetimepicker.css?v=5df2ea75" rel="stylesheet" type="text/css"/>
<link href="/appresources/jasny-bootstrap.css?v=f55b5bde" rel="stylesheet" type="text/css"/>
<link href="/appresources/font-awesome.css?v=de015a3" rel="stylesheet" type="text/css"/>
<link href="/appresources/tipsy.css?v=980f2268" rel="stylesheet" type="text/css"/>
<link href="/appresources/chosen.css?v=82ebffd4" rel="stylesheet" type="text/css"/>
<link href="/appresources/credit-card.less?v=9fede289" rel="stylesheet" type="text/css"/>
<link href="/appresources/jquery.growl.css?v=26245b98" rel="stylesheet" type="text/css"/>
<link href="/appresources/bootstrap-multiselect.css?v=49de55a1" rel="stylesheet" type="text/css"/>
<link href="/dynamic/default.css?v=a4af43ce" rel="stylesheet" type="text/css"/>
<link href="/static/favicon.ico" rel="shortcut icon" type="image/x-icon"/>
<script id="ze-snippet" src="https://static.zdassets.com/ekr/snippet.js?key=06c7e4c3-a30c-4343-bdc7-7c95ac2a6110" type="text/javascript"></script>
<link href="https://account.jetbrains.com/login" rel="canonical"/><meta class="local" content="JetBrains, creator of the best Java IDE - IntelliJ IDEA - is a technology-leading software vendor specializing in the creation of intelligent development tools." name="description"/> </head>
<body class="jba-agreement" onunload=";">
<div class="jba-agreement-panel">
<div class="container">
<div class="row">
<div class="col-sm-12">By using the JetBrains Account website, you agree to the JetBrains Account Agreement.<span class="unit-container-sm"></span><a href="/profile/privacyPolicy?backUrl=https%3A%2F%2Faccount.jetbrains.com%2Flogin">Review now</a><span class="unit-container-sm"></span><a class="ajax-action" data-method="POST" data-url="/profile/postponePrivacyPolicy?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" href="#">Remind me later</a></div>
</div>
</div>
</div>
<div class="navbar navbar-default navbar-fixed-top navbar-inverse">
<div class="container">
<div class="navbar-header"><button class="navbar-toggle" data-target=".navbar-menu-collapse" data-toggle="collapse" type="button"><span class="sr-only">Toggle navigation</span><i class="icon icon-bar"></i><i class="icon icon-bar"></i><i class="icon icon-bar"></i></button><a class="logo" href="https://www.jetbrains.com">JetBrains</a></div>
<div class="collapse navbar-collapse navbar-menu-collapse">
<ul class="nav navbar-nav"></ul>
<ul class="nav navbar-nav navbar-actions">
<li><a href="https://sales.jetbrains.com/hc/categories/200934669-JetBrains-Account-Help"><span class="j-icon-help-medium"></span>Help</a></li>
</ul>
</div>
</div>
</div>
<div class="footer-wrap">
<div class="header"></div>
<div class="container">
<div class="js-concurrent-resource-modification-alert" style="display:none;">
<div class="alert alert-danger"><a aria-label="Close" class="close">×</a>Please <a class="js-reload-page-link">reload current page</a> because it's content or state has been changed outside.</div>
</div>
<div class="row">
<div class="col-sm-12">
<div class="container start-screen">
<div class="row">
<div class="col-sm-12 col-md-11 col-lg-10">
<div class="row">
<div class="col-sm-6 col-lg-5">
<h1>Welcome to JetBrains Account</h1>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-purchases-feature pull-left"></i>
<p class="lead">
<strong>Access your purchases</strong>
<br/>
                             and view your order history
                          </p>
</div>
</div>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-alert-feature pull-left"></i>
<p class="lead">
<strong>Identify expired and outdated licenses,</strong>
<br/>
                             order new licenses and upgrades
                          </p>
</div>
</div>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-manage-feature pull-left"></i>
<p class="lead">
<strong>Manage your company licenses</strong>
<br/>
                             and distribute them to end users
                          </p>
</div>
</div>
</div>
<div class="col-sm-4 col-sm-offset-1">
<div class="well panel-highlighted">
<form action="/authorize?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" class="js-auth-dialog-form" method="post">
<p class="js-auth-dialog-div-errors text-danger"></p>
<h5>Sign in with existing account</h5>
<div class="form-group js-auth-dialog-input-username"><input autofocus="" class="form-control" id="username" name="username" placeholder="Email address or Username" style="width:100%" type="text"/></div>
<div class="form-group js-auth-dialog-input-password"><input class="form-control" name="password" placeholder="Password" style="width:100%" type="password"/></div>
<div class="form-group js-auth-dialog-input-2fa" style="display:none;">
<p class="small">You have enabled two-factor authentication, so you will also need to enter the 6‑digit verification code from your mobile authenticator app, or use a recovery code.</p>
<input class="form-control" name="secondFactor" placeholder="One-time password" style="width:100%" type="text"/>
</div>
<div class="form-group clearfix">
<div class="pull-left"><button class="btn btn-primary btn-lg js-auth-dialog-btn-signin login-submit-btn" onclick="return false;">Sign In</button><span class="unit-container-xs"></span></div>
<div class="js-auth-dialog-div-reset-pswd"><a class="btn btn-lg btn-link" href="/forgot-password">Forgot password?</a></div>
</div>
</form>
</div>
<div class="well panel-highlighted">
<form action="/signup-request?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" method="post">
<h5>Not registered yet?</h5>
<h3>Create JetBrains Account</h3>
<div class="form-group"><input class="form-control" id="email" name="email" oninput="try { if($(this).val().trim().match(new RegExp(this.getAttribute('pattern')))) { this.setCustomValidity(''); } else { this.setCustomValidity('Please enter a valid email address'); } } catch(e) {}" pattern="^[a-zA-Z0-9!#$%&amp;'*+/=?^_`{|}~-]+(?:\.[a-zA-Z0-9!#$%&amp;'*+/=?^_`{|}~-]+)*@(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])?\.)+[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]$" placeholder="Your email address" style="width:100%" type="text"/></div>
<div class="form-group clearfix"><button class="btn btn-primary btn-lg eml-submit-btn" onclick="$('#email').val(function(_, x) { return x.trim() })">Sign Up</button></div>
</form>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="footer-push"></div>
</div>
<div class="footer">
<div class="container">
<div class="small">Copyright © 2000–2020 JetBrains s.r.o. | <a href="https://sales.jetbrains.com/hc/categories/200934669-JetBrains-Account-Help" target="_blank">Help</a> | <a href="mailto:sales@jetbrains.com?subject=JetBrains%20Account%20Support">Support</a> | <a href="https://www.jetbrains.com/company/privacy.html" target="_blank">JetBrains Privacy Policy</a> | <a href="//www.jetbrains.com/legal/agreements/jetbrains_account.html">JetBrains Account Agreement</a> | Build #2020.08-2134</div>
</div>
</div>
<script src="/appresources/requirejs-config.js?v=d2d44ead" type="text/javascript"></script>
<script src="/appresources/require.js?v=cc5cbeb6" type="text/javascript"></script>
<script type="text/javascript">
    requirejs.onError = function(error) {
        require(['jetprofile'], function (JetProfile) {
            console.error(error.message, error.stack || error.originalError.target.src);
            window.onerror(error.message, error.stack || error.originalError.target.src);
        });
    };
    require(['/appresources/main.js']);</script>
</body>
</html>

Можно ли изменить какой-либо метод для получения доступа?

...