Я использую SCDF с сервером шкипера 2.3.2 и сервером потока данных 2.4.2. при развертывании с Docker -compose Я создаю задачу с помощью составной задачи 2.1.3. Я пытаюсь выполнить свою задачу, но даже с примерами, составная задача не запускает задачи
Пример:
composed-task-runner && mytask && myTask
2020-04-14 14:40:41.633 INFO 231 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'taskExecutor'
2020-04-14 14:40:41.752 INFO 231 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: POSTGRES
2020-04-14 14:40:41.885 INFO 231 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2020-04-14 14:40:42.083 DEBUG 231 --- [ main] o.s.c.t.a.c.DataFlowConfiguration : Not configuring basic security for accessing the Data Flow Server
2020-04-14 14:40:46.922 DEBUG 231 --- [ main] o.s.c.t.r.s.TaskRepositoryInitializer : Initializing task schema for postgresql database
2020-04-14 14:40:47.461 DEBUG 231 --- [ main] BatchConfiguration$ReferenceTargetSource : Initializing lazy target object
2020-04-14 14:40:47.552 DEBUG 231 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Starting: TaskExecution{executionId=102, parentExecutionId=null, exitCode=null, taskName='testComposedTask2', startTime=Tue Apr 14 14:40:47 GMT 2020, endTime=null, exitMessage='null', externalExecutionId='null', errorMessage='null', arguments=[xx=7777, --spring.cloud.data.flow.platformname=default, --spring.cloud.task.executionid=102, --spring.cloud.data.flow.taskappname=composed-task-runner]}
2020-04-14 14:40:47.675 INFO 231 --- [ main] .t.a.c.ComposedtaskrunnerTaskApplication : Started ComposedtaskrunnerTaskApplication in 33.768 seconds (JVM running for 38.059)
2020-04-14 14:40:47.743 DEBUG 231 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Updating: TaskExecution with executionId=102 with the following {exitCode=0, endTime=Tue Apr 14 14:40:47 GMT 2020, exitMessage='null', errorMessage='null'}
2020-04-14 14:40:47.831 INFO 231 --- [ Thread-7] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'taskExecutor'
2020-04-14 14:40:47.836 INFO 231 --- [ Thread-7] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-04-14 14:40:47.852 INFO 231 --- [ Thread-7] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2020-04-14 14:40:47.909 INFO 231 --- [ Thread-7] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
Не могли бы вы мне помочь? Я с Postgres DB
Новое замечание:
Я создаю новую составную задачу, как сказано в документации:
testComposedTaskA с определением myTask && timestamp
поток данных:> список задач
╔═══════════════════════════╤═══════════════════╤═══════════╤═══════════╗
║ Task Name │ Task Definition │description│Task Status║
╠═══════════════════════════╪═══════════════════╪═══════════╪═══════════╣
║testComposedTaskA-myTask │myTask │ │UNKNOWN ║
║testComposedTaskA-timestamp│timestamp │ │UNKNOWN ║
║testComposedTaskA │myTask && timestamp│ │UNKNOWN ║
╚═══════════════════════════╧═══════════════════╧═══════════╧═══════════╝
поток данных:> запуск задачи testComposedTaskA Запущенная задача 'testComposedTaskA' с идентификатором выполнения 135
поток данных:> список выполнения задачи
╔═════════════════╤═══╤════════════════════════════╤════════════════════════════╤═════════╗
║ Task Name │ID │ Start Time │ End Time │Exit Code║
╠═════════════════╪═══╪════════════════════════════╪════════════════════════════╪═════════╣
║testComposedTaskA│135│Fri Apr 17 15:04:42 GMT 2020│Fri Apr 17 15:04:42 GMT 2020│0 ║
╚═════════════════╧═══╧════════════════════════════╧════════════════════════════╧═════════╝
Подзадачи никогда не выполняются. В примерах мы можем видеть подзадачу как выполненную Что не так?
В качестве запроса, журнал запуска сервера потока данных Привет журнал запуска сервера потока данных:
dataflow-server | 2020-04-20 07:43:24.047 INFO 1 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
dataflow-server | 2020-04-20 07:43:24.061 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
dataflow-server | 2020-04-20 07:43:27.735 DEBUG 1 --- [ main] o.s.c.t.c.SimpleTaskAutoConfiguration : Using org.springframework.cloud.task.configuration.DefaultTaskConfigurer TaskConfigurer
dataflow-server | 2020-04-20 07:43:27.745 DEBUG 1 --- [ main] o.s.c.t.c.DefaultTaskConfigurer : EntityManager was found, using JpaTransactionManager
dataflow-server | 2020-04-20 07:43:28.282 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: POSTGRES
skipper | 2020-04-20 07:43:28.294 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
dataflow-server | 2020-04-20 07:43:28.505 INFO 1 --- [ main] o.s.c.d.s.b.SimpleJobServiceFactoryBean : No database type set, using meta data indicating: POSTGRES
dataflow-server | 2020-04-20 07:43:28.975 WARN 1 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
dataflow-server | 2020-04-20 07:43:29.392 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
dataflow-server | 2020-04-20 07:43:32.150 INFO 1 --- [ main] .s.c.DataFlowControllerAutoConfiguration : Skipper URI [http://skipper-server:7577/api]
dataflow-server | 2020-04-20 07:43:32.614 INFO 1 --- [ main] o.a.coyote.http11.Http11NioProtocol : Starting ProtocolHandler ["http-nio-9393"]
dataflow-server | 2020-04-20 07:43:32.764 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9393 (http) with context path ''
dataflow-server | 2020-04-20 07:43:32.780 INFO 1 --- [ main] o.s.c.d.s.s.DataFlowServerApplication : Started DataFlowServerApplication in 68.947 seconds (JVM running for 78.165)
dataflow-server | 2020-04-20 07:43:33.396 INFO 1 --- [ main] .s.c.d.s.s.LauncherInitializationService : Added 'Local' platform account 'default' into Task Launcher repository.
Журнал в начало задачи:
2020-04-20 08:01:58.886 INFO 115 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data repositories in DEFAULT mode.
2020-04-20 08:01:59.008 INFO 115 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 68ms. Found 0 repository interfaces.
2020-04-20 08:02:00.545 INFO 115 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=56219e3d-5a2d-3fb3-8901-c3dad5be90af
2020-04-20 08:02:00.943 INFO 115 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$cd22d842] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-04-20 08:02:01.049 INFO 115 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$e93cdb3f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-04-20 08:02:01.062 INFO 115 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration' of type [org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$$EnhancerBySpringCGLIB$$529283ce] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-04-20 08:02:01.094 INFO 115 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.listener.BatchEventAutoConfiguration' of type [org.springframework.cloud.task.batch.listener.BatchEventAutoConfiguration$$EnhancerBySpringCGLIB$$9ae88dd1] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-04-20 08:02:02.529 INFO 115 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2020-04-20 08:02:03.239 INFO 115 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2020-04-20 08:02:04.489 INFO 115 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [
name: default
...]
2020-04-20 08:02:07.075 INFO 115 --- [ main] org.hibernate.Version : HHH000412: Hibernate Core {5.3.13.Final}
2020-04-20 08:02:07.111 INFO 115 --- [ main] org.hibernate.cfg.Environment : HHH000206: hibernate.properties not found
2020-04-20 08:02:08.896 INFO 115 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.0.4.Final}
2020-04-20 08:02:10.736 INFO 115 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL95Dialect
2020-04-20 08:02:10.825 INFO 115 --- [ main] o.h.e.j.e.i.LobCreatorBuilderImpl : HHH000422: Disabling contextual LOB creation as connection was null
2020-04-20 08:02:10.843 INFO 115 --- [ main] org.hibernate.type.BasicTypeRegistry : HHH000270: Type registration [java.util.UUID] overrides previous : org.hibernate.type.UUIDBinaryType@3eb91815
2020-04-20 08:02:11.457 INFO 115 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2020-04-20 08:02:11.675 DEBUG 115 --- [ main] o.s.c.t.c.SimpleTaskAutoConfiguration : Using org.springframework.cloud.task.configuration.DefaultTaskConfigurer TaskConfigurer
2020-04-20 08:02:11.679 DEBUG 115 --- [ main] o.s.c.t.c.DefaultTaskConfigurer : EntityManager was found, using JpaTransactionManager
2020-04-20 08:02:11.902 INFO 115 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'taskExecutor'
2020-04-20 08:02:12.005 INFO 115 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: POSTGRES
2020-04-20 08:02:12.107 INFO 115 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2020-04-20 08:02:12.203 DEBUG 115 --- [ main] o.s.c.t.a.c.DataFlowConfiguration : Not configuring basic security for accessing the Data Flow Server
2020-04-20 08:02:15.942 DEBUG 115 --- [ main] o.s.c.t.r.s.TaskRepositoryInitializer : Initializing task schema for postgresql database
2020-04-20 08:02:16.551 DEBUG 115 --- [ main] BatchConfiguration$ReferenceTargetSource : Initializing lazy target object
2020-04-20 08:02:16.617 DEBUG 115 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Starting: TaskExecution{executionId=136, parentExecutionId=null, exitCode=null, taskName='testComposedTaskA', startTime=Mon Apr 20 08:02:16 GMT 2020, endTime=null, exitMessage='null', externalExecutionId='null', errorMessage='null', arguments=[--spring.cloud.data.flow.platformname=default, --spring.cloud.task.executionid=136, --spring.cloud.data.flow.taskappname=composed-task-runner]}
2020-04-20 08:02:16.686 INFO 115 --- [ main] .t.a.c.ComposedtaskrunnerTaskApplication : Started ComposedtaskrunnerTaskApplication in 31.176 seconds (JVM running for 34.541)
2020-04-20 08:02:16.717 DEBUG 115 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Updating: TaskExecution with executionId=136 with the following {exitCode=0, endTime=Mon Apr 20 08:02:16 GMT 2020, exitMessage='null', errorMessage='null'}
2020-04-20 08:02:16.778 INFO 115 --- [ Thread-7] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'taskExecutor'
2020-04-20 08:02:16.784 INFO 115 --- [ Thread-7] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-04-20 08:02:16.803 INFO 115 --- [ Thread-7] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2020-04-20 08:02:16.854 INFO 115 --- [ Thread-7] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
состав docker используется:
dataflow-server:
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
- spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults=false
- spring.jpa.database-platform=org.hibernate.dialect.PostgreSQL95Dialect
entrypoint: "./wait-for-it.sh -t 240 postgres:5432 -- java -jar /maven/spring-cloud-dataflow-server.jar"
skipper-server:
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
entrypoint: "./wait-for-it.sh -t 240 postgres:5432 -- java -Djava.security.egd=file:/dev/./urandom -jar /spring-cloud-skipper-server.jar"
Надеюсь, это поможет.
С уважением
Frederi c