Как поместить FlowFiles из NiFi в Ignite Cache - PullRequest
0 голосов
/ 27 февраля 2019

Я новичок в NiFi и Ignite.Я пытаюсь передавать потоковые файлы на Ignite с помощью процессора PutIgniteCache.Тем не менее, я получаю сообщение об ошибке java.lang.ExceptionInInitializerError.У меня есть конфигурация по умолчанию для PutIgniteCache и я использую имя файла в качестве идентификатора записи Ignite Cache.

Честно говоря, мне не хватает какой-то простой конфигурации NiFi или Ignite, кто-нибудь сталкивался с этой ошибкой раньше?

РЕДАКТИРОВАТЬ: Ниже приведен мой файл bootstrap.conf для NiFi, с добавленными аргументами (17-23) для Ignite для работы на Java 9.

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Java command to use when running NiFi
java=java

# Username to use when running NiFi. This value will be ignored on Windows.
run.as=

# Configure where NiFi's lib and conf directories live
lib.dir=./lib
conf.dir=./conf

# How long to wait after telling NiFi to shutdown before explicitly killing the Process
graceful.shutdown.seconds=20

# Disable JSR 199 so that we can use JSP's without running a JDK
java.arg.1=-Dorg.apache.jasper.compiler.disablejsr199=true

# JVM memory settings
java.arg.2=-Xms512m
java.arg.3=-Xmx512m

# Enable Remote Debugging
#java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000

java.arg.4=-Djava.net.preferIPv4Stack=true

# allowRestrictedHeaders is required for Cluster/Node communications to work properly
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol

# The G1GC is still considered experimental but has proven to be very advantageous in providing great
# performance without significant "stop-the-world" delays.
java.arg.13=-XX:+UseG1GC

#Set headless mode by default
java.arg.14=-Djava.awt.headless=true

# Master key in hexadecimal format for encrypted sensitive configuration values
nifi.bootstrap.sensitive.key=

# Sets the provider of SecureRandom to /dev/urandom to prevent blocking on VMs
java.arg.15=-Djava.security.egd=file:/dev/urandom

# Requires JAAS to use only the provided JAAS configuration to authenticate a Subject, without using any "fallback" methods (such as prompting for username/password)
# Please see https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/single-signon.html, section "EXCEPTIONS TO THE MODEL"
java.arg.16=-Djavax.security.auth.useSubjectCredsOnly=true

# Added arguments to make Ignite work with Java 9
java.arg.17=--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED 
java.arg.18=--add-exports=java.base/sun.nio.ch=ALL-UNNAMED 
java.arg.19=--add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED 
java.arg.20=--add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
java.arg.21=--add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED
java.arg.22=--illegal-access=permit
java.arg.23=-Djdk.tls.client.protocols=TLSv1.2

###
# Notification Services for notifying interested parties when NiFi is stopped, started, dies
###

# XML File that contains the definitions of the notification services
notification.services.file=./conf/bootstrap-notification-services.xml

# In the case that we are unable to send a notification for an event, how many times should we retry?
notification.max.attempts=5

# Comma-separated list of identifiers that are present in the notification.services.file; which services should be used to notify when NiFi is started?
#nifi.start.notification.services=email-notification

# Comma-separated list of identifiers that are present in the notification.services.file; which services should be used to notify when NiFi is stopped?
#nifi.stop.notification.services=email-notification

# Comma-separated list of identifiers that are present in the notification.services.file; which services should be used to notify when NiFi dies?
#nifi.dead.notification.services=email-notification

nifi-app.log

org.apache.ignite.IgniteException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
    at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:906)
    at org.apache.ignite.Ignition.start(Ignition.java:307)
    at org.apache.nifi.processors.ignite.AbstractIgniteProcessor.initializeIgnite(AbstractIgniteProcessor.java:111)
    at org.apache.nifi.processors.ignite.cache.AbstractIgniteCacheProcessor.initializeIgniteCache(AbstractIgniteCacheProcessor.java:108)
    at org.apache.nifi.processors.ignite.cache.PutIgniteCache.initializeIgniteDataStreamer(PutIgniteCache.java:210)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
    at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
    at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
    at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
    at org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$4(StandardProcessorNode.java:1515)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1536)
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:897)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1736)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1589)
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1042)
    at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:964)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:850)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:749)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:499)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:479)
    at org.apache.ignite.Ignition.start(Ignition.java:304)
    ... 18 common frames omitted
Caused by: org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, reconCnt=10, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false]
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:258)
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:677)
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1531)
    ... 28 common frames omitted
Caused by: org.apache.ignite.spi.IgniteSpiException: Thread has been interrupted.
    at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:260)
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1832)
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:255)
    ... 30 common frames omitted
Caused by: java.lang.InterruptedException: null
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
    at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:252)
    ... 32 common frames omitted

nifi-bootstrap.log

2019-02-28 10:05:33,264 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2019-02-28 10:05:33,266 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2019-02-28 10:05:33,266 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2019-02-28 10:05:33,266 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2019-02-28 10:05:33,283 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2019-02-28 10:05:33,283 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /home/rasmus/Desktop/nifi-1.9.0
2019-02-28 10:05:33,284 INFO [main] org.apache.nifi.bootstrap.Command Command: /usr/lib/jvm/jdk-9.0.4/bin/java -classpath /home/rasmus/Desktop/nifi-1.9.0/./conf:/home/rasmus/Desktop/nifi-1.9.0/./lib/jul-to-slf4j-1.7.25.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/logback-core-1.2.3.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/jcl-over-slf4j-1.7.25.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/javax.servlet-api-3.1.0.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/log4j-over-slf4j-1.7.25.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/nifi-framework-api-1.9.0.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/slf4j-api-1.7.25.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/nifi-nar-utils-1.9.0.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/nifi-api-1.9.0.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/logback-classic-1.2.3.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/jetty-schemas-3.1.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/nifi-properties-1.9.0.jar:/home/rasmus/Desktop/nifi-1.9.0/./lib/nifi-runtime-1.9.0.jar -Djdk.tls.client.protocols=TLSv1.2 --illegal-access=permit --add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED  -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.egd=file:/dev/urandom --add-exports=java.base/sun.nio.ch=ALL-UNNAMED  --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED  -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:+UseG1GC -Dnifi.properties.file.path=/home/rasmus/Desktop/nifi-1.9.0/./conf/nifi.properties -Dnifi.bootstrap.listen.port=33547 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/home/rasmus/Desktop/nifi-1.9.0/logs --add-modules=java.xml.bind org.apache.nifi.NiFi 
2019-02-28 10:05:33,308 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 5184
2019-02-28 10:05:33,390 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Unknown module: ALL-UNNAMED  specified to --add-exports
2019-02-28 10:05:33,392 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Unknown module: ALL-UNNAMED  specified to --add-exports
2019-02-28 10:05:33,392 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Unknown module: ALL-UNNAMED  specified to --add-exports
2019-02-28 10:05:33,898 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 41749
2019-02-28 10:05:43,319 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: An illegal reflective access operation has occurred
2019-02-28 10:05:43,319 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Illegal reflective access by org.xerial.snappy.SnappyLoader (file:/home/rasmus/Desktop/nifi-1.9.0/./work/nar/extensions/nifi-hive-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/snappy-java-1.0.5.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
2019-02-28 10:05:43,319 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Please consider reporting this to the maintainers of org.xerial.snappy.SnappyLoader
2019-02-28 10:05:43,319 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
2019-02-28 10:05:43,319 ERROR [NiFi logging handler] org.apache.nifi.StdErr WARNING: All illegal access operations will be denied in a future release

Ответы [ 2 ]

0 голосов
/ 28 февраля 2019

Ничего не поделать ... зависимость воспламенения от NiFi - старая версия, которая не поддерживает Java 9. Вы можете попробовать перекомпилировать ее самостоятельно с новой версией или понизить версию Java иТем временем, создайте проблему относительно этой проблемы.

0 голосов
/ 28 февраля 2019

Кажется, что вы используете очень старую версию Apache Ignite, которая определенно не будет работать под Java 9. Вам следует обновить систему до последней версии Apache Ignite 2.7 (и сохранить эти опции для Java 9).

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...