Я пытаюсь использовать следующий код для удаления веб-сайта javascript. В браузере, когда веб-сайт открыт, он загружает то, что я пытаюсь получить, внутрь <div class=results>. Моя проблема в том, что при запуске этого кода в блокноте Jupyter этот div кажется пустым.
from requests_html import AsyncHTMLSession
session = AsyncHTMLSession()
r = await session.get('https://thumbtack.github.io/abba/demo/abba.html#Baseline=13855%2C91428&Variation+1=13242%2C90703&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true')
await r.html.arender()
print(r.content)
#test of code
это разрешает наличие пустого div:
<div class="results"><div>
Для приведенного выше URL-адреса в браузере, используя inspect тот же URL, тот же самый div разрешает в div с содержимым,
<div class="results" style="display: block;"> <table> <thead> <tr> <th></th> <th>Successes</th> <th>Total</th> <th colspan="2">Success Rate</th> <th>p-value</th> <th>Improvement</th> </tr> </thead> <tbody class="result-table"> <tr class="result-row"> <th class="bucket-name">Baseline</th> <td class="yes">15,453</td> <td class="total">101,901</td> <td class="conversion-numeric"> <div class="interval"> <div class="error"><span class="lower-bound">15%</span> – <span class="upper-bound">15%</span></div> <span class="base">(<span class="base-value">15%</span>)</span> </div></td> <td class="conversion-visual" style="display: inline-block;"><svg font-size="10px" font-family="sans-serif" fill="none" stroke="none" stroke-width="1.5" width="220" height="15" data-ol-has-click-handler=""><g transform="translate(10,0)"><rect x="114.94823723343774" width="85.05176276656226" height="15" fill="rgb(184,184,184)"></rect><rect x="114.94823723343774" width="1e-10" height="15" fill="rgb(180,38,71)"></rect><rect x="200" width="1e-10" height="15" fill="rgb(38,180,60)"></rect></g><g transform="translate(10,0)"><line shape-rendering="crispEdges" x1="-1" y1="4" x2="-1" y2="11" stroke="rgb(68,68,68)" stroke-width="1"></line><line shape-rendering="crispEdges" x1="200" y1="4" x2="200" y2="11" stroke="rgb(68,68,68)" stroke-width="1"></line></g><g transform="translate(10,0)"><text pointer-events="none" x="-3" dy=".35em" transform="translate(-1,7.5)" fill="rgb(68,68,68)" text-anchor="end" style="font: 12px Arial;">-</text><text pointer-events="none" x="3" dy=".35em" transform="translate(200,7.5)" fill="rgb(68,68,68)" style="font: 12px Arial;">+</text></g><g transform="translate(10,0)"><line shape-rendering="crispEdges" x1="0" y1="7.5" x2="200" y2="7.5" stroke="rgb(68,68,68)" stroke-width="1"></line></g></svg></td> <td class="p-value">—</td> <td class="improvement">—</td> </tr> <tr class="result-row"> <th class="bucket-name">Variation 1</th> <td class="yes">14,690</td> <td class="total">100,845</td> <td class="conversion-numeric"> <div class="interval"> <div class="error"><span class="lower-bound">14%</span> – <span class="upper-bound">15%</span></div> <span class="base">(<span class="base-value">15%</span>)</span> </div></td> <td class="conversion-visual" style="display: inline-block;"><svg font-size="10px" font-family="sans-serif" fill="none" stroke="none" stroke-width="1.5" width="220" height="15" data-ol-has-click-handler=""><g transform="translate(10,0)"><rect width="84.08874241310063" height="15" fill="rgb(184,184,184)"></rect><rect width="84.08874241310063" height="15" fill="rgb(180,38,71)"></rect><rect x="200" width="1e-10" height="15" fill="rgb(38,180,60)"></rect></g><g transform="translate(10,0)"><line shape-rendering="crispEdges" x1="-1" y1="4" x2="-1" y2="11" stroke="rgb(68,68,68)" stroke-width="1"></line><line shape-rendering="crispEdges" x1="200" y1="4" x2="200" y2="11" stroke="rgb(68,68,68)" stroke-width="1"></line></g><g transform="translate(10,0)"><text pointer-events="none" x="-3" dy=".35em" transform="translate(-1,7.5)" fill="rgb(68,68,68)" text-anchor="end" style="font: 12px Arial;">-</text><text pointer-events="none" x="3" dy=".35em" transform="translate(200,7.5)" fill="rgb(68,68,68)" style="font: 12px Arial;">+</text></g><g transform="translate(10,0)"><line shape-rendering="crispEdges" x1="0" y1="7.5" x2="200" y2="7.5" stroke="rgb(68,68,68)" stroke-width="1"></line></g></svg></td> <td class="p-value">0.0002</td> <td class="improvement"> <div class="interval"> <div class="error"><span class="lower-bound">-6%</span> – <span class="upper-bound">-1.9%</span></div> <span class="base">(<span class="base-value">-3.9%</span>)</span> </div></td> </tr></tbody> </table></div>
Редактировать: в соответствии с документацией к библиотеке questions_html, он может отображать JavaScript. https://pypi.org/project/requests-html/