При использовании потокового вещания с использованием широковещательной рассылки клиент hbase-обертки получает прямой доступ к hbase, в котором включен Kerberos, но я не знаю, как обновить токен sh - PullRequest
Новая
       26

При использовании потокового вещания с использованием широковещательной рассылки клиент hbase-обертки получает прямой доступ к hbase, в котором включен Kerberos, но я не знаю, как обновить токен sh

0 голосов
/ 30 марта 2020

Эта проблема беспокоила меня в течение долгого времени, пожалуйста, помогите мне увидеть ее, большое спасибо

, когда я использую spark broadcast для того, чтобы обернуть клиента hbase для прямого доступа к hbase, я не знаю как для повторной регистрации с помощью keytab.

вот мой код 

class HBaseSink(createHbaseConnection: () => Connection) extends Serializable {

  lazy val connection: Connection = createHbaseConnection()

  def getTable(tableName: String): Table = {
    UserGroupInformation.getLoginUser.reloginFromKeytab()
    connection.getTable(TableName.valueOf(tableName))
  }

}

object HBaseSink {

  private val executor = Executors.newScheduledThreadPool(1)

  def apply(): HBaseSink = {
    val createConnectionFunc = () => {
      System.setProperty("java.security.krb5.conf", "/etc/krb5.conf")
      val configuration = HBaseConfiguration.create
      configuration.set("hbase.zookeeper.quorum", AppConfig.hbaseZKServer)
      configuration.set("hbase.zookeeper.property.clientPort", "2181")
      configuration.set("hbase.security.authentication", "kerberos")
      configuration.set("hadoop.security.authentication", "kerberos")
      configuration.set("hbase.master.kerberos.principal", "hbase/_HOST@OYOHOTELS.CN")
      configuration.set("hbase.regionserver.kerberos.principal", "hbase/_HOST@OYOHOTELS.CN")
      configuration.set("hadoop.kerberos.keytab.login.autorenewal.enabled", "true")
      configuration.set("hadoop.kerberos.min.seconds.before.relogin", "60")
      UserGroupInformation.setConfiguration(configuration)
      val keytabPath = if (AppConfig.env.equalsIgnoreCase("local")) {
        AppConfig.keytabFile
      } else {
        SparkFiles.get(AppConfig.keytabFile)
      }
      configuration.set("hbase.master.keytab.file", keytabPath)
      configuration.set("hbase.regionserver.keytab.file", keytabPath)
      val ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("xxx@xxx.CN", keytabPath)
      UserGroupInformation.setLoginUser(ugi)
      UserGroupInformation.getLoginUser.checkTGTAndReloginFromKeytab()

      val conn = ugi.doAs(new PrivilegedAction[Connection]() {
        override def run: Connection = {
          val connection = ConnectionFactory.createConnection(configuration)
          connection
        }
      })
      val runnable = new Runnable {
        override def run(): Unit = {
          ugi.reloginFromKeytab()
        }
      }
      executor.scheduleAtFixedRate(runnable, 0, 10, TimeUnit.MINUTES)
      conn
    }
    new HBaseSink(createConnectionFunc)
  }
}

и мой скрипт spark-submit 

#!/bin/sh
source /etc/profile
cp=""
for jar in `ls /home/user/spark/common/lib`
do
   cp=/home/user/spark/common/lib/$jar,$cp
done
echo $cp
SPARK_URL=/home/oyo_search/spark/message-handler
MAIN_CLASS=com.message.handler.scala.MessageHandlerMain
APP_ID=$(yarn application -list |grep  $MAIN_CLASS |awk -F'\t' '{print $1}')
if [ -n "${APP_ID}" ]; then
    yarn application -kill $APP_ID
    echo "Stop instance $APP_ID"
fi
echo "start main class $MAIN_CLASS"
nohup \
spark-submit \
--class $MAIN_CLASS \
--master yarn \
--deploy-mode cluster \
--executor-memory 1g \
--executor-cores 1 \
--driver-memory 1g \
--driver-cores 1 \
--total-executor-cores 3 \
--num-executors 3 \
--conf "spark.executor.extraJavaOptions=-Dsun.security.krb5.debug=true -Djava.security.krb5.conf=krb5.conf -Dkeytab.file=xxx.keytab" \
--conf "spark.driver.extraJavaOptions=-Dsun.security.krb5.debug=true -Djava.security.krb5.conf=krb5.conf -Dkeytab.file=xxx.keytab" \
--conf spark.dynamicAllocation.enabled=false \
--conf spark.security.credentials.hbase.enabled=false \
--conf spark.yarn.dist.archives=/etc/krb5.conf,/opt/keytab/xxx.keytab \
--jars $cp \
$SPARK_URL/message-handler-1.0.0-RELEASE.jar \
 >>$SPARK_URL/log/logs.log 2>&1 &

сообщение об ошибке: 

allTimeout=1200000, callDuration=1290362: Failed after attempts=16, exceptions:
Mon Mar 30 11:51:41 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:41 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:41 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:41 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:42 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:43 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, java.io.IOException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
Mon Mar 30 11:51:45 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:51:49 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:51:59 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:52:09 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:52:19 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:52:29 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:52:49 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:53:09 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:53:29 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Mon Mar 30 11:53:50 CST 2020, RpcRetryingCaller{globalStartTime=1585540301110, pause=100, maxAttempts=16}, javax.security.sasl.SaslException: Call to oyoshbddwcmdev3.ahotels.tech/192.168.71.238:16020 failed on local exception: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [Caused by javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]
Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.

Похожие темы

...