Команды CREATE и DROP завершаются ошибкой внутри OracleOperator

Question

Я недавно начал получать ошибку с OracleOperator в Airflow.Я не могу создавать или удалять таблицы;Я получаю cx_Oracle.DatabaseError: ORA-00901: invalid CREATE command и cx_Oracle.DatabaseError: ORA-00950: invalid DROP option.

. Мне удалось воспроизвести проблему на RHEL с cx_Oracle 7.2.2 и Mac OSX с cx_Oracle 7.0.0, каждый с Python 3.6.8 и Airflow 1.10.5,Это происходит даже на airflow test.Ниже я включил файл dag и оба sql-запроса в один и тот же блок кода.Внизу находится след ошибки.Заранее спасибо за любую помощь!

# thisdag.py
from __future__ import print_function, division

from airflow import DAG
from airflow.operators.oracle_operator import OracleOperator
import datetime

default_args = {
    'owner': 'thisfoo',
    'email': ['email.address@internet.gov'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 0,
    'start_date': datetime.datetime(2019,5,28),
    'retry_delay': datetime.timedelta(seconds=60),
}

with DAG(
    dag_id='thisdag',
    default_args=default_args,
    schedule_interval='@once') as dag:

    create_table = OracleOperator(
        task_id = 'create_table',
        sql = 'create.sql ',
        oracle_conn_id = 'myconn',
        autocommit = True
    )

    drop_table = OracleOperator(
        task_id = 'drop_table',
        sql = 'drop.sql ',
        oracle_conn_id = 'myconn',
        autocommit = True
    )

create_table >> drop_table



--create.sql
CREATE TABLE MY_TEST AS SELECT 'HELLO' AS MYCOL FROM DUAL


--drop.sql
BEGIN
   EXECUTE IMMEDIATE 'DROP TABLE MY_TEST';
EXCEPTION
   WHEN OTHERS THEN
      IF SQLCODE != -942 THEN
         RAISE;
      END IF;
END;

[2019-09-19 15:25:44,822] {taskinstance.py:620} INFO - Dependencies all met for <TaskInstance: thisdag.create_table 2019-06-01T00:00:00+00:00 [queued]>
[2019-09-19 15:25:44,833] {taskinstance.py:620} INFO - Dependencies all met for <TaskInstance: thisdag.create_table 2019-06-01T00:00:00+00:00 [queued]>
[2019-09-19 15:25:44,833] {taskinstance.py:838} INFO - 
--------------------------------------------------------------------------------
[2019-09-19 15:25:44,833] {taskinstance.py:839} INFO - Starting attempt 1 of 1
[2019-09-19 15:25:44,833] {taskinstance.py:840} INFO - 
--------------------------------------------------------------------------------
[2019-09-19 15:25:44,844] {taskinstance.py:859} INFO - Executing <Task(OracleOperator): create_table> on 2019-06-01T00:00:00+00:00
[2019-09-19 15:25:44,845] {base_task_runner.py:133} INFO - Running: ['airflow', 'run', 'thisdag', 'create_table', '2019-06-01T00:00:00+00:00', '--job_id', '321', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/test_oracle_dag.py', '--cfg_path', '/tmp/tmpefh6jlts']
[2019-09-19 15:25:45,531] {base_task_runner.py:115} INFO - Job 321: Subtask create_table [2019-09-19 15:25:45,530] {settings.py:213} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=30350
[2019-09-19 15:25:45,777] {base_task_runner.py:115} INFO - Job 321: Subtask create_table [2019-09-19 15:25:45,776] {__init__.py:51} INFO - Using executor SequentialExecutor
[2019-09-19 15:25:46,267] {base_task_runner.py:115} INFO - Job 321: Subtask create_table [2019-09-19 15:25:46,266] {dagbag.py:90} INFO - Filling up the DagBag from /users/mose/test/airflow/dags/test_oracle_dag.py
[2019-09-19 15:25:46,294] {base_task_runner.py:115} INFO - Job 321: Subtask create_table [2019-09-19 15:25:46,293] {cli.py:516} INFO - Running <TaskInstance: thisdag.create_table 2019-06-01T00:00:00+00:00 [running]> on host csc2cxn00002238.cloud.kp.org
[2019-09-19 15:25:46,315] {oracle_operator.py:58} INFO - Executing: create.sql 
[2019-09-19 15:25:46,372] {logging_mixin.py:95} INFO - [2019-09-19 15:25:46,371] {base_hook.py:84} INFO - Using connection to: id: myconn. Host: ########, Port: 1571, Schema: ####, Login: #####, Password: XXXXXXXX, extra: {'dsn': '#######', 'service_name': '#######'}
[2019-09-19 15:25:46,548] {logging_mixin.py:95} INFO - [2019-09-19 15:25:46,547] {dbapi_hook.py:171} INFO - create.sql 
[2019-09-19 15:25:46,558] {taskinstance.py:1051} ERROR - ORA-00901: invalid CREATE command
Traceback (most recent call last):
  File "/users/mose/test/lib64/python3.6/site-packages/airflow/models/taskinstance.py", line 926, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/users/mose/test/lib64/python3.6/site-packages/airflow/operators/oracle_operator.py", line 63, in execute
    parameters=self.parameters)
  File "/users/mose/test/lib64/python3.6/site-packages/airflow/hooks/dbapi_hook.py", line 172, in run
    cur.execute(s)
cx_Oracle.DatabaseError: ORA-00901: invalid CREATE command
[2019-09-19 15:25:46,561] {taskinstance.py:1082} INFO - Marking task as FAILED.
[2019-09-19 15:25:46,584] {base_task_runner.py:115} INFO - Job 321: Subtask create_table Traceback (most recent call last):
[2019-09-19 15:25:46,584] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/bin/airflow", line 32, in <module>
[2019-09-19 15:25:46,584] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     args.func(args)
[2019-09-19 15:25:46,584] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2019-09-19 15:25:46,584] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     return f(*args, **kwargs)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/bin/cli.py", line 522, in run
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     _run(args, dag, ti)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/bin/cli.py", line 440, in _run
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     pool=args.pool,
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     return func(*args, **kwargs)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/models/taskinstance.py", line 926, in _run_raw_task
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     result = task_copy.execute(context=context)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/operators/oracle_operator.py", line 63, in execute
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     parameters=self.parameters)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table   File "/users/mose/test/lib64/python3.6/site-packages/airflow/hooks/dbapi_hook.py", line 172, in run
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table     cur.execute(s)
[2019-09-19 15:25:46,585] {base_task_runner.py:115} INFO - Job 321: Subtask create_table cx_Oracle.DatabaseError: ORA-00901: invalid CREATE command
[2019-09-19 15:25:49,826] {logging_mixin.py:95} INFO - [2019-09-19 15:25:49,823] {local_task_job.py:105} INFO - Task exited with return code 1

Answer 1

Существует проблема с синтаксисом вашей команды CREATE TABLE:

CREATE TABLE MY_TEST AS( SELECT 'HELLO' FROM DUAL )

ORA-00998: необходимо назвать это выражение псевдонимом столбца

Вам нужно присвоить псевдоним столбцу, который генерирует ваш запрос (поскольку вы выбираете из DUAL, по умолчанию имя столбца отсутствует).

Обратите внимание:

CREATE TABLE MY_TEST AS SELECT 'HELLO' AS MY_COL FROM DUAL;

Примечание: круглые скобки вокруг SELECT излишни.