Запрос на Youtube от Beautifulsoup4 не вернул содержание тела - PullRequest
Новая
       14

Запрос на Youtube от Beautifulsoup4 не вернул содержание тела

0 голосов
/ 04 мая 2020

Я пытался получить содержимое плейлиста Youtube. Но единственное, что я получил, это его заголовок. Я пытался много искать, но ни один из них не помог мне. Единственная подсказка заключалась в том, что было что-то, связанное с безопасностью, и Youtube не мог предоставить мне свой контент html. Во всяком случае, это то, что я делал до сих пор: 

import requests
import csv
from bs4 import BeautifulSoup
url = 'https://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8'
page = requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
print(soup)

и вот что я получаю: 

<!DOCTYPE html>
<html data-cast-api-enabled="true" lang="vi-VN">
 <head>
  <style name="www-roboto">
   @font-face{font-family:'Roboto';font-style:italic;font-weight:400;src:local('Roboto Italic'),local('Roboto-Italic'),url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xHIzc.ttf)format('truetype');}@font-face{font-family:'Roboto';font-style:normal;font-weight:400;src:local('Roboto Regular'),local('Roboto-Regular'),url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu7WxP.ttf)format('truetype');}@font-face{font-family:'Roboto';font-style:normal;font-weight:500;src:local('Roboto Medium'),local('Roboto-Medium'),url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fCxc9.ttf)format('truetype');}@font-face{font-family:'Roboto';font-style:italic;font-weight:500;src:local('Roboto Medium Italic'),local('Roboto-MediumItalic'),url(//fonts.gstatic.com/s/roboto/v18/KFOjCnqEu92Fr1Mu51S7ACc1CsE.ttf)format('truetype');}
  </style>
  <script name="www-roboto">
   if (document.fonts && document.fonts.load) {document.fonts.load("400 10pt Roboto", "T");document.fonts.load("500 10pt Roboto", "T");}
  </script>
  <script>
   var ytcsi = {gt: function(n) {n = (n || '') + 'data_';return ytcsi[n] || (ytcsi[n] = {tick: {},info: {}});},now: (window.performance && window.performance.timing &&window.performance.now && window.performance.timing.navigationStart) ?function() {return window.performance.timing.navigationStart +window.performance.now();} :function() {return (new Date()).getTime();},tick: function(l, t, n) {ticks = ytcsi.gt(n).tick;var v = t || ytcsi.now();if (ticks[l]) {ticks['_' + l] = (ticks['_' + l] || [ticks[l]]);ticks['_' + l].push(v);}ticks[l] = v;},info: function(k, v, n) {ytcsi.gt(n).info[k] = v;},setStart: function(s, t, n) {ytcsi.info('yt_sts', s, n);ytcsi.tick('_start', t, n);}};(function(w, d) {ytcsi.setStart('dhs', w.performance ? w.performance.timing.responseStart : null);var isPrerender = (d.visibilityState || d.webkitVisibilityState) == 'prerender';var vName = (!d.visibilityState && d.webkitVisibilityState)? 'webkitvisibilitychange' : 'visibilitychange';if (isPrerender) {ytcsi.info('prerender', 1);var startTick = function() {ytcsi.setStart('dhs');d.removeEventListener(vName, startTick);};d.addEventListener(vName, startTick, false);}if (d.addEventListener) {d.addEventListener(vName, function() {ytcsi.tick('vc');}, false);}function isGecko() {if (!w.navigator || !w.navigator.userAgent) {return false;}var ua = w.navigator.userAgent;return ua.indexOf('Gecko') > 0 &&ua.toLowerCase().indexOf('webkit') < 0 &&ua.indexOf('Edge') < 0 &&ua.indexOf('Trident') < 0 &&ua.indexOf('MSIE') < 0;}if (isGecko()) {var isHidden = (d.visibilityState || d.webkitVisibilityState) == 'hidden';if (isHidden) {ytcsi.tick('vc');}}var slt = function(el, t) {setTimeout(function() {var n = ytcsi.now();el.loadTime = n;if (el.slt) {el.slt();}}, t);};w.__ytRIL = function(el) 
{if (!el.getAttribute('data-thumb')) {if (w.requestAnimationFrame) {w.requestAnimationFrame(function() {slt(el, 0);});} else {slt(el, 16);}}};})(window, document);
  </script>
  <script>
   var ytcfg = {d: function() {return (window.yt && yt.config_) || ytcfg.data_ || (ytcfg.data_ = {});},get: function(k, o) {return (k in ytcfg.d()) ? ytcfg.d()[k] : o;},set: function() {var a = arguments;if (a.length > 1) {ytcfg.d()[a[0]] = a[1];} else {for (var k in a[0]) {ytcfg.d()[k] = a[0][k];}}}};
  </script>
  <script>
   ytcfg.set("ROOT_VE_TYPE", 5754);ytcfg.set("EVENT_ID", "avavXrLULonSqAG7mryYAg");
  </script>
  <script>
   window.yterr = window.yterr || true;
  </script>
  <script>
   (function(){/*

 Copyright The Closure Library Authors.
 SPDX-License-Identifier: Apache-2.0
*/
var a={a:"content-snap-width-1",b:"content-snap-width-2",f:"content-snap-width-3"};function f(){var c=[],b;for(b in a)c.push(a[b]);return c}
function h(c){var b=f().concat(["guide-pinned","show-guide"]),d=b.length,g=[];c.replace(/\S+/g,function(e){for(var k=0;k<d;k++)if(e==b[k])return;g.push(e)});
return g}
;function l(c,b,d){var g=document.getElementsByTagName("html")[0],e=h(g.className);c&&1251<=(window.innerWidth||document.documentElement.clientWidth)&&(e.push("guide-pinned"),b&&e.push("show-guide"));d&&(d=(window.innerWidth||document.documentElement.clientWidth)-21-50,1251<=(window.innerWidth||document.documentElement.clientWidth)&&c&&b&&(d-=230),e.push(1262<=d?"content-snap-width-3":1056<=d?"content-snap-width-2":"content-snap-width-1"));g.className=e.join(" ")}
var m=["yt","www","masthead","sizing","runBeforeBodyIsReady"],n=this||self;m[0]in n||"undefined"==typeof n.execScript||n.execScript("var "+m[0]);for(var p;m.length&&(p=m.shift());)m.length||void 0===l?n[p]&&n[p]!==Object.prototype[p]?n=n[p]:n=n[p]={}:n[p]=l;}).call(this);

      try {window.ytbuffer = {};ytbuffer.handleClick = function(e) {var element = e.target || e.srcElement;while (element.parentElement) {if (/(^| )yt-can-buffer( |$)/.test(element.className)) {window.ytbuffer = {bufferedClick: e};element.className += ' yt-is-buffered';break;}element = element.parentElement;}};if (document.addEventListener) {document.addEventListener('click', ytbuffer.handleClick);} else {document.attachEvent('onclick', ytbuffer.handleClick);}} catch(e) {}

    yt.www.masthead.sizing.runBeforeBodyIsReady(true,true,true);
  </script>
  <script name="scheduler/scheduler" src="/yts/jsbin/scheduler-vfl08UJUN/scheduler.js" type="text/javascript">
  </script>
  <link href="/yts/cssbin/www-core-vflsWry6p.css" name="www-core" rel="stylesheet"/>
  <link href="/s/player/64dddad9/www-player.css" name="player/www-player" rel="stylesheet"/>
  <link href="/yts/cssbin/www-pageframe-vfl2-hqXw.css" name="www-pageframe" rel="stylesheet"/>
  <link href="/yts/cssbin/www-guide-vflc-Schd.css" name="www-guide" rel="stylesheet"/>
  <title>
   Âm nhạc - YouTube
  </title>
  <link href="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="canonical"/>
  <meta content="YouTube" property="og:site_name"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" property="og:url"/>
  <meta content="Âm nhạc - YouTube" property="og:title"/>
  <meta content="" property="og:description"/>
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEWCKgBEF5IWvKriqkDCQgBFQAAiEIYAQ==&amp;rs=AOn4CLD3BirgIBzWnlS6MKrgePejx1p44A" property="og:image"/>  
  <meta content="168" property="og:image:width"/>
  <meta content="94" property="og:image:height"/>
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEWCMQBEG5IWvKriqkDCQgBFQAAiEIYAQ==&amp;rs=AOn4CLBVf0gyjrwYcohzkCzQgR-xLrKa2A" property="og:image"/>  
  <meta content="196" property="og:image:width"/>
  <meta content="110" property="og:image:height"/>
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEXCPYBEIoBSFryq4qpAwkIARUAAIhCGAE=&amp;rs=AOn4CLASO9cTSzVvkZUXTirNgvratCdSUg" property="og:image"/>  
  <meta content="246" property="og:image:width"/>
  <meta content="138" property="og:image:height"/>
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEXCNACELwBSFryq4qpAwkIARUAAIhCGAE=&amp;rs=AOn4CLBtIbpLAKY8G-Q_tDYxIqj8uAPpXQ" property="og:image"/>  
  <meta content="336" property="og:image:width"/>
  <meta content="188" property="og:image:height"/>
  <meta app-argument="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" content="app-id=544007664" name="apple-itunes-app"/>
  <meta content="544007664" property="al:ios:app_store_id"/>
  <meta content="YouTube" property="al:ios:app_name"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8&amp;feature=applinks" property="al:ios:url"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8&amp;feature=applinks" property="al:android:url"/>
  <meta content="YouTube" property="al:android:app_name"/>
  <meta content="com.google.android.youtube" property="al:android:package"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8&amp;feature=applinks" property="al:web:url"/>
  <meta content="true" property="al:web:should_fallback"/>
  <meta content="website" property="og:type"/>
  <meta content="87741124305" property="fb:app_id"/>
  <meta content="18+" property="og:restrictions:age"/>
  <meta content="summary" name="twitter:card"/>
  <meta content="@YouTube" name="twitter:site"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" name="twitter:url"/>
  <meta content="Âm nhạc - YouTube" name="twitter:title"/>
  <meta content="" name="twitter:description"/>
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEWCKgBEF5IWvKriqkDCQgBFQAAiEIYAQ==&amp;rs=AOn4CLD3BirgIBzWnlS6MKrgePejx1p44A" name="twitter:image"/> 
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEWCMQBEG5IWvKriqkDCQgBFQAAiEIYAQ==&amp;rs=AOn4CLBVf0gyjrwYcohzkCzQgR-xLrKa2A" name="twitter:image"/> 
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEXCPYBEIoBSFryq4qpAwkIARUAAIhCGAE=&amp;rs=AOn4CLASO9cTSzVvkZUXTirNgvratCdSUg" name="twitter:image"/> 
  <meta content="https://i.ytimg.com/vi/DyDfgMOUjCI/hqdefault.jpg?sqp=-oaymwEXCNACELwBSFryq4qpAwkIARUAAIhCGAE=&amp;rs=AOn4CLBtIbpLAKY8G-Q_tDYxIqj8uAPpXQ" name="twitter:image"/> 
  <meta content="YouTube" name="twitter:app:name:iphone"/>
  <meta content="544007664" name="twitter:app:id:iphone"/>
  <meta content="YouTube" name="twitter:app:name:ipad"/>
  <meta content="544007664" name="twitter:app:id:ipad"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8&amp;feature=twitter-deep-link" name="twitter:app:url:ipad"/>
  <meta content="YouTube" name="twitter:app:name:googleplay"/>
  <meta content="com.google.android.youtube" name="twitter:app:id:googleplay"/>
  <meta content="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8&amp;feature=twitter-deep-link" name="twitter:app:url:googleplay"/>
  <meta content="Âm nhạc - YouTube" name="title"/>
  <meta name="nosnippet"/>
  <link href="http://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="alternate"/>
  <link href="android-app://com.google.android.youtube/http/youtube.comhttp://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="alternate"/>
  <link href="ios-app://544007664/http/youtube.comhttp://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="alternate"/>
  <link href="/manifest.json" rel="manifest"/>
  <link href="https://www.youtube.com/opensearch?locale=vi_VN" rel="search" title="Tìm kiếm Video trên YouTube" type="application/opensearchdescription+xml"/>
  <link href="https://s.ytimg.com/yts/img/favicon-vfl8qSV2F.ico" rel="shortcut icon" type="image/x-icon"/>
  <link href="/yts/img/favicon_32-vflOogEID.png" rel="icon" sizes="32x32"/>
  <link href="/yts/img/favicon_48-vflVjB_Qk.png" rel="icon" sizes="48x48"/>
  <link href="/yts/img/favicon_96-vflW9Ec0w.png" rel="icon" sizes="96x96"/>
  <link href="/yts/img/favicon_144-vfliLAfaB.png" rel="icon" sizes="144x144"/>
  <meta content="#ff0000" name="theme-color"/>
  <link href="android-app://com.google.android.youtube/http/www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="alternate"/>
  <link href="ios-app://544007664/vnd.youtube/www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8" rel="alternate"/>
  <link href="/yts/cssbin/www-home-c4-vflSOyhuz.css" name="www-home-c4" rel="stylesheet"/>
  <style>
   .exp-invert-logo .hats-logo {background: no-repeat url(/yts/img/ringo/hitchhiker/logo_mini_gr

Спасибо, ребята

1 Ответ

0 голосов
/ 04 мая 2020

Это то, что вы ищете? 

import requests
from bs4 import BeautifulSoup
s = requests.Session()
r = s.get('https://www.youtube.com/playlist?list=OLAK5uy_n5owy0fZQh1rzWt8QZk3iVvi8gjL7lVh8')
soup = BeautifulSoup(r.text, 'lxml')

Вам лучше использовать селен, когда вы очищаете данные c. Нравится комментарии на YouTube.

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.

Нет похожих вопросов

...