Как удалить указанные c слов из прокручиваемой веб-таблицы?

Question

Я хочу убрать тикер из таблицы и проверить, присутствуют ли определенные ключевые слова {"GD", "Daily price", "TSI"} в строке "окончательная расчетная цена". В таблице есть несколько столбцов Dynami c. Ниже приведен один пример: я хочу убрать тикерный символ "Ambuja Cements Futures", который является "ZACE", и сохранить его в списке, а также проверить, присутствуют ли определенные ключевые слова в строке "Окончательная расчетная цена".

Вот код HTML для строки тикерного символа:

<tr>
   <th>Ticker Symbol</th>
   <td>ZADS</td>
   <td>ZACE</td>
   <td>ZAPN</td>
   <td>ZARB</td>
   <td>ZAXS</td>
   <td>ZBJA</td>
   <td>ZBAF</td>
   <td>ZBPC</td>
   <td>ZBHA</td>
   <td>ZBHI</td>
   <td>ZBOS</td>
   <td>ZCIP</td>
   <td>ZCOA</td>
   <td>ZDRR</td>
   <td>ZEIM</td>
   <td>ZGAI</td>
   <td>ZHCL</td>
   <td>ZHDB</td>
   <td>ZHMC</td>
   <td>ZHND</td>
   <td>ZHPC</td>
   <td>ZHUV</td>
   <td>ZHDF</td>
   <td>ZICI</td>
   <td>ZIHF</td>
   <td>ZIOC</td>
   <td>ZIIB</td>
   <td>ZINF</td>
   <td>ZITC</td>
   <td>ZKMB</td>
   <td>ZLT</td>
   <td>ZLPC</td>
   <td>ZMM</td>
   <td>ZMSI</td>
   <td>ZNTP</td>
   <td>ZONG</td>
   <td>ZPWG</td>
   <td>ZRIL</td>
   <td>ZSBI</td>
   <td>ZSUN</td>
   <td>ZTCS</td>
   <td>ZTTM</td>
   <td>ZTAT</td>
   <td>ZTEC</td>
   <td>ZUTC</td>
   <td>ZUPL</td>
   <td>ZVED</td>
   <td>ZWPR</td>
   <td>ZYES</td>
   <td>ZZEE</td>
</tr>

Вот код HTML для строки цены окончательного расчета:

<tr>
   <th>Final Settlement Price</th>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.
      <br>
   </td>
   <td></td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.
      <br>
   </td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.
      <br>
   </td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
   <td>The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.</td>
</tr>

Вы можете посетить веб-сайт для того же: https://www.sgx.com/derivatives/products/indiassf?cc=ZADS

Answer 1

Данные в таблице загружаются динамически, поэтому BeautifulSoup их не видит. Но вы можете использовать модуль re / json / requests для получения данных:

import json
import re
import requests


url = 'https://www.sgx.com/derivatives/products/indiassf?cc=ZADS'

app_config_url = 'https://www.sgx.com/config/appconfig.js'
n = re.search(r"apiVersion: '(.*?)'", requests.get(app_config_url).text).group(1)
variables = '''{{"path":"/derivatives/products/{}","lang":"EN"}}'''.format( url.split('?')[0].split('/')[-1] )
url2 = 'https://api2.sgx.com/content-api?queryId={}:page&variables={}'.format(n, variables)

data = requests.get(url2).json()

# uncomment this to prin all data:
#print(json.dumps(data, indent=4))

for c in data['data']['route']['data']['data']['contracts']:
    if c['data']['title'] == 'Ambuja Cements Futures':
        print(c['data']['tickerSymbol'])
        print(c['data']['finalSettlementPrice'])

Печать:

ZACE
The Final Settlement Price for the Contracts will be the official closing price of the Underlying Shares on the Last Trading Day, rounded to 2 decimal places, expressed as US dollars.

РЕДАКТИРОВАТЬ: Новая версия после обновления сайта:

import json
import requests


url = 'https://www.sgx.com/derivatives/products/indiassf?cc=ZADS'

config = requests.get('https://www.sgx.com/config/appconfig.json').json()

variables = '''{{"path":"/derivatives/products/{}","lang":"EN"}}'''.format( url.split('?')[0].split('/')[-1] )
url2 = 'https://api2.sgx.com/content-api?queryId={}:page&variables={}'.format(config['cms']['apiVersion'], variables)

data = requests.get(url2).json()

# # uncomment this to prin all data:
# print(json.dumps(data, indent=4))

for c in data['data']['route']['data']['data']['contracts']:
    if c['data']['title'] == 'Ambuja Cements Futures':
        print(c['data']['tickerSymbol'])
        print(c['data']['finalSettlementPrice'])