Hbase ExportSnapshot не работает с временными учетными данными aws - PullRequest
Новая
       19

Hbase ExportSnapshot не работает с временными учетными данными aws

0 голосов
/ 15 мая 2019

Я пытаюсь экспортировать снимок таблицы hbase из on-prem (linux) коробки в корзину s3.

Используется следующая команда: 

HADOOP_CLASSPATH=~/hadoop-aws-2.6.0-cdh5.4.0.jar:~/aws-java-sdk-sts-1.11.550.jar ~/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dhadoop.tmp.dir=file:///home/users/aws/ -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec -Dfs.s3a.access.key=ASIAXXXXXX -Dfs.s3a.secret.key=XXXXXX -Dfs.s3a.session.token=XXXXX -Dfs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider -snapshot TABLE_NAME -copy-to s3a://bucket-name/

Журналы 

2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Error Message: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXXX, AWS Error Code: null, AWS Error Message: Forbidden
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: HTTP Status Code: 403
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: AWS Error Code: null
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Error Type: Client
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Request ID: XXXXX
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Class Name: com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception
Exception in thread "main" com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXX, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: XXXXX
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:976)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:956)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:849)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:76)
  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:895)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:1024)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:1028)```

команды cli работают нормально.
Например, 

aws s3 ls
2019-01-10 23:19:47 zzzz
2019-05-06 13:31:21 xxxx

Невозможно экспортировать снимок с использованием временных учетных данных aws.

...