Скребковый стол не будет печататься полностью - PullRequest
Новая
       3

Скребковый стол не будет печататься полностью

1 голос
/ 03 июня 2019

Я очищаю таблицу с веб-сайта, и у меня не было проблем с получением данных, но у меня возникают проблемы с печатью окончательного результата. В предоставленном мною коде он печатает все, без проблем, если он печатает в моем операторе for (см. Закомментированные команды печати). Однако, если я напечатаю это позже в коде, вне оператора for, я получу только первую строку. Я хотел бы взять этот код и поместить его в более крупный проект, где этот вывод (среди прочего) находится в одном электронном письме. Как мне отобразить весь вывод?

Я пытался добавить каждую строку таблицы в список, думаю, что я делаю это неправильно, но он просто печатает одну и ту же строку снова или снова или отдельные буквы из первой строки. 

import requests
from bs4 import BeautifulSoup


headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36'}

#print('Scraping Iowa Dept of Banking...')


url = 'https://www.idob.state.ia.us/bank/docs/applica/app_status.aspx'
r = requests.get(url, headers = headers)


soup = BeautifulSoup(r.text, 'html.parser')


mylist = []
for tr in soup.find_all('tr')[2:]:
    tds = tr.find_all('td')
    if len(tds[5].text) == 1:
        edate = "NA"
    else:
        edate = ""
    if len(tds[6].text) == 1:
        loc = "NA"
    else:
        loc = ""
    output5 = ("Bank: %s, City: %s, Type: %s, Effective Date: %s, Location: %s, Comment: %s \r\n" % (tds[0].text, tds[1].text, tds[2].text.replace(" ", ""), tds[5].text+edate, tds[6].text.replace(" ", "")+loc, tds[7].text))


    global outputs5
    outputs5 = output5
    #print(outputs5) #The whole table prints if printed here

if outputs5 is None:
    outputs5 = "No information available"
    print(outputs5)

print(outputs5) #only prints the first line

Ответы [ 3 ]

2 голосов
/ 03 июня 2019

Я хотел бы использовать pandas, которая является библиотекой python, извлечь таблицу и импортировать в CSV. 

import pandas as pd
tables=pd.read_html("https://www.idob.state.ia.us/bank/docs/applica/app_status.aspx")
tables[1].to_csv('output.csv')

CSV будет выглядеть так.

enter image description here

Настолько просто установить панд. Просто введите в командной строке.

pip install pandas

1 голос
/ 03 июня 2019

Вы пробовали использовать панд '.read_html()? 

import pandas as pd

url ='https://www.idob.state.ia.us/bank/docs/applica/app_status.aspx'

tables = pd.read_html(url)

Выход: 

print (tables[-1].to_string())
                                     Bank             City                                        Type    Accepted    Approved Effective #      Location                                            Comment
0                              State Bank      New Hampton                         Merge With and Into  05/21/2019         NaN         NaN           NaN  Application to merge State Bank, New Hampton, ...
1        Farmers and Traders Savings Bank            Douds                         Merge With and Into  05/20/2019         NaN         NaN           NaN  Application to merge Farmers and Traders Savin...
2                         City State Bank          Norwalk                     Establish a Bank Office  05/15/2019  05/29/2019         NaN      Mesa, AZ  Application by City State Bank, Norwalk, to es...
3                             Availa Bank          Carroll                     Purchase and Assumption  04/16/2019  04/30/2019         NaN           NaN  Application by Availa Bank, Carroll, to purcha...
4                               West Bank  West Des Moines                     Establish a Bank Office  04/16/2019  05/02/2019  05/10/2019   Mankato, MN  Application by West Bank, West Des Moines, to ...
5                               West Bank  West Des Moines                     Establish a Bank Office  04/10/2019  05/02/2019  05/07/2019  Owatonna, MN  Application by West Bank, West Des Moines, to ...
6                               West Bank  West Des Moines                     Establish a Bank Office  04/09/2019  05/02/2019  05/07/2019           NaN  Application by West Bank, West Des Moines, to ...
7                         Iowa State Bank           Algona                     Establish a Bank Office  03/15/2019         NaN         NaN   Phoenix, AZ  Application by Iowa State Bank, Algona, to est...
8                    Peoples Savings Bank             Elma                         Merge With and Into  03/13/2019  04/24/2019         NaN           NaN  Application to merge Peoples State Bank, Elma,...
9                 Two Rivers Bank & Trust       Burlington                     Purchase and Assumption  01/25/2019  01/31/2019  05/31/2019           NaN  Application by Two Rivers Bank & Trust, Burlin...
10                    Westside State Bank         Westside                     Establish a Bank Office  01/25/2019  02/06/2019         NaN  Bellevue, NE  Application by Westside State Bank, Westside, ...
11                         Northwest Bank          Spencer                      Relocate a Bank Office  11/29/2018  12/12/2018         NaN        Ankeny  Application by Northwest Bank, Spencer, to rel...
12                        City State Bank          Norwalk                     Establish a Bank Office  11/21/2018  12/12/2018         NaN       Norwalk  Application by City State Bank, Norwalk, to es...
13  First Security Bank and Trust Company     Charles City                      Relocate a Bank Office  06/21/2018  06/29/2018         NaN         Manly  Application by First Security Bank and Trust C...
14                   Lincoln Savings Bank      Cedar Falls                     Establish a Bank Office  06/04/2018  06/25/2018         NaN    Des Moines  Application by Lincoln Savings Bank, Cedar Fal...
15                    Raccoon Valley Bank            Perry                     Establish a Bank Office  02/12/2018  03/02/2018         NaN        Grimes  Application by Raccoon Valley Bank, Perry, to ...
16                 Community Savings Bank         Edgewood                      Relocate a Bank Office  01/25/2018  01/25/2018         NaN    Manchester  Application by Community Savings Bank, Edgewoo...
17                     Luana Savings Bank            Luana                     Establish a Bank Office  06/05/2017  08/16/2017         NaN       Norwalk  Application by Luana Savings Bank, Luana, to e...
18         Fort Madison Financial Company     Fort Madison                         Change of Ownership         NaN  10/19/2017         NaN           NaN  Application for Linda Sue Baier, Fort Madison,...
19                        Lincoln Bancorp         Reinbeck                         Change of Ownership         NaN  12/10/2018         NaN           NaN  Application for Lincoln Bancorp Employee Stock...
20           Emmetsburg Bank Shares, Inc.       Emmetsburg                         Change of Ownership         NaN  01/17/2019         NaN           NaN  Application for Charles and Maryanna Sarazine,...
21      Albrecht Financial Services, Inc.          Norwalk                           Change of Control         NaN  03/27/2019  05/10/2019           NaN  Application for Dean L. Albrecht 2014 Family T...
22                  Solon Financial, Inc.            Solon                         Change of Ownership         NaN  03/05/2019         NaN           NaN  Application for Cordelia A. Cosgrove, Bruce A....
23                How-Win Development Co.           Cresco                         Change of Ownership         NaN  03/28/2019         NaN           NaN  Application for John Scott Thomson, as trustee...
24                      Lee Capital Corp.     Fort Madison                         Change of Ownership         NaN  04/15/2019         NaN           NaN  Application by Jean M. Humphrey, Kathleen A. M...
25                    FNB BanShares, Inc.       West Union                         Change of Ownership         NaN  05/02/2019         NaN           NaN  Application for James L. Moss, individually an...
26          Old O'Brien Banc Shares, Inc.       Sutherland                         Change of Ownership         NaN  03/06/2019         NaN           NaN  Application for James J. Johnson and Colleen D...
27            Pella Financial Group, Inc.            Pella                           Change of Control         NaN  03/15/2019         NaN           NaN  Application for Pella Financial Group, Inc., P...
28                                   BANK          Wapello  Amend or Restate Articles of Incorporation         NaN  05/07/2019  05/07/2019           NaN          Restatement of Articles of Incorporation.
29                  Security Agency, Inc.          Decorah                         Change of Ownership         NaN  11/28/2018         NaN           NaN  Application for the 2018 Grantor Trust FBO Rac...
30                          Arendt's Inc.        Montezuma                         Change of Ownership         NaN  05/29/2019         NaN           NaN  Application for C. W. Bolen, Montezuma, indivi...
1 голос
/ 03 июня 2019

Попробуйте так: вам нужно добавить выходные данные в список, а затем соединить список вместе перед его печатью.

Причина, по которой ваш отпечаток внутри цикла работал, заключалась в том, что он печатался 5 раз, а нетолько один раз. 

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36'}

#print('Scraping Iowa Dept of Banking...')

url = 'https://www.idob.state.ia.us/bank/docs/applica/app_status.aspx'
r = requests.get(url, headers = headers)

soup = BeautifulSoup(r.text, 'html.parser')

mylist = []
for tr in soup.find_all('tr')[2:]:
    tds = tr.find_all('td')
    if len(tds[5].text) == 1:
        edate = "NA"
    else:
        edate = ""
    if len(tds[6].text) == 1:
        loc = "NA"
    else:
        loc = ""
    output5 = ("Bank: %s, City: %s, Type: %s, Effective Date: %s, Location: %s, Comment: %s \r\n" % (tds[0].text, tds[1].text, tds[2].text.replace(" ", ""), tds[5].text+edate, tds[6].text.replace(" ", "")+loc, tds[7].text))

    mylist.append(outputs5)

if mylist == []:
    print("No information available")

print('\n'.join(mylist))
...