Python / Flask функция для загрузки больших файлов в базу данных sqlite3 и облако (GCP) - PullRequest
Новая
       12

Python / Flask функция для загрузки больших файлов в базу данных sqlite3 и облако (GCP)

0 голосов
/ 10 января 2020

В настоящее время я работаю с: Python, Flask, SQLAlchemy,

Цель: загрузить файл из веб-формы (HTML5, CSS, Jinja2) в Google Cloud Platform до 10 ГБ на файл.

Проблема: во время загрузки возникает ошибка тайм-аута, а затем операция отменяется.

Это внутренняя функция python, которая выполняет загрузку: `

def upload(): 
form = UploadForm() 
if form.validate_on_submit(): 
   file = request.files['localfile'] 
   name = secure_filename(file.filename) 
   print(file) 
   file.save(os.path.join(basedir, name)) 
   blob = bucket.blob(name) 
   blob.upload_from_filename(os.path.join(basedir, file.filename))`

Ошибка: 

127.0.0.1 - - [09/Jan/2020 14:54:00] "POST /ingestion/upload HTTP/1.1" 500 - Traceback (most recent call last): File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 2309, in call return self.wsgi_app(environ, start_response) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 2295, in wsgi_app response = self.handle_exception(e) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 1741, in handle_exception reraise(exc_type, exc_value, tb) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise raise value File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app response = self.full_dispatch_request() File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request rv = self.handle_user_exception(e) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception reraise(exc_type, exc_value, tb) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise raise value File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request rv = self.dispatch_request() File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request return self.view_functionsrule.endpoint File "/Users/gustavocastanheira/Desktop/DDC/DDC/ingestion/views.py", line 46, in upload blob.upload_from_filename(os.path.join(basedir, file.filename)) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1320, in upload_from_filename predefined_acl=predefined_acl, File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1265, in upload_from_file client, file_obj, content_type, size, num_retries, predefined_acl File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1175, in _do_upload client, stream, content_type, size, num_retries, predefined_acl File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1122, in _do_resumable_upload response = upload.transmit_next_chunk(transport) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/resumable_media/requests/upload.py", line 425, in transmit_next_chunk retry_strategy=self._retry_strategy, File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/resumable_media/requests/_helpers.py", line 136, in http_request return _helpers.wait_and_retry(func, RequestsMixin._get_status_code, retry_strategy) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/resumable_media/_helpers.py", line 150, in wait_and_retry response = func() File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/google/auth/transport/requests.py", line 287, in request **kwargs File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/requests/sessions.py", line 508, in request resp = self.send(prep, **send_kwargs) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/requests/sessions.py", line 618, in send r = adapter.send(request, **kwargs) File "/Users/gustavocastanheira/ddc_dev/lib/python3.7/site-packages/requests/adapters.py", line 490, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', timeout('The write operation timed out'))
...