URL-адрес подключения JDBC к Apache Phoenix в Azure HDInsight

Question

Я использую следующий URL для подключения через JDBC к Apache Phoenix в HDInsight Azure, который в основном состоит из сервера Zookeeper в соответствии с информационной панелью Ambari.

jdbc:phoenix:zk0-apache.xxxxxxxxxxxxxxxxxxxxxxxxx.tx.internal.cloudapp.net,zk1-apache.xxxxxxxxxxxxxxxxxxxxxxxxx.tx.internal.cloudapp.net,zk2-apache.xxxxxxxxxxxxxxxxxxxxxxxxx.tx.internal.cloudapp.net:2181:/hbase-unsecure

Ниже приведенотрассировка стека до исключения.

java.net.UnknownHostException: zk0-apache.xxxxxxxxxxxxxxxxxxxxxxxxx.tx.internal.cloudapp.net: nodename nor servname provided, or not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_172]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[na:1.8.0_172]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[na:1.8.0_172]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[na:1.8.0_172]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_172]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_172]
at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61) ~[zookeeper-3.4.10.jar:3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f]
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445) ~[zookeeper-3.4.10.jar:3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f]
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380) ~[zookeeper-3.4.10.jar:3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f]
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:144) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:131) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:143) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:181) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.<init>(ZooKeeperKeepAliveConnection.java:43) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1737) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:104) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:945) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:721) [hbase-client-1.4.0.jar:1.4.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.8.0_172]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [na:1.8.0_172]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [na:1.8.0_172]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [na:1.8.0_172]
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:439) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:348) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144) [hbase-client-1.4.0.jar:1.4.0]
at org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:425) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(ConnectionQueryServicesImpl.java:267) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2515) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2491) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2491) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221) [phoenix-core-4.14.0-HBase-1.4.jar:4.14.0-HBase-1.4]
at java.sql.DriverManager.getConnection(DriverManager.java:664) [na:1.8.0_172]
at java.sql.DriverManager.getConnection(DriverManager.java:208) [na:1.8.0_172]
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:154) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:145) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:205) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:169) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:151) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:115) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:78) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:324) [spring-jdbc-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.boot.jdbc.EmbeddedDatabaseConnection.isEmbedded(EmbeddedDatabaseConnection.java:123) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.jdbc.AbstractDataSourceInitializer.isEnabled(AbstractDataSourceInitializer.java:74) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.jdbc.AbstractDataSourceInitializer.initialize(AbstractDataSourceInitializer.java:54) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_172]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_172]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_172]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_172]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1737) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:576) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:498) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) [spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:846) ~[spring-beans-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:863) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at com.myntra.asn_api.AsnApiApplication.main(AsnApiApplication.java:14) ~[classes/:na]

Кто-нибудь здесь, кто работает над Phoenix в Azure HDInsight?

Что такое URL JDBC (для толстых и тонких клиентов), который будет использоваться для подключения кФеникс на Azure HDInsight?

Заранее спасибо

Answer 1

Покопавшись, URL JDBL в порядке и обнаружил две проблемы с настройкой Azure HDI:

1. Сервер Zookeeper не был запущен
2. Порт Zookeeper (2181) не работаетopen

1. Сервер Zookeeper не запускается автоматически, если вы используете Azure HDI, его нужно запустить, выполнив следующую команду

 `./usr/hdp/2.xxcx/zookeeper/bin/zkServer.sh start`

Приведенная выше команда запускает сервер Zookeeper в автономном режиме.Если вам нужно запустить Zookeeper в кластерном режиме, вам нужно добавить FQDN сервера Zookeeper Server (server. {1..n}) в файл ./usr/hdp/2.xxcx/zookeeper/conf/zoo.cfg, если он еще не существует.

maxClientCnxns=50
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181

server.1=zk0-apache.xxxxxxxxx.tx.internal.cloudapp.net:2888:3888
server.2=zk1-apache.xxxxxxxxx.tx.internal.cloudapp.net:2888:3888
server.3=zk2-apache.xxxxxxxxx.tx.internal.cloudapp.net:2888:3888

Вам нужно остановить и запустить zkServer после изменения конфигурации. Дополнительная информация о документации Zookeeper .

2. Кластер Azure HDI по умолчанию создается внутри сети, которая не предоставляет порты кластера (например, 2181, 8765) для публики.Вам необходимо создать виртуальную сеть, как описано в в этой статье , и создать кластер HDI внутри этой сети.Через сеть вы сможете открыть нужные порты.