Как установить RcppArmadillo в кластер Spark EMR? - PullRequest
Новая
       186

Как установить RcppArmadillo в кластер Spark EMR?

0 голосов
/ 06 апреля 2020

У меня SSHed в главном узле кластера EMR, и я пытаюсь установить библиотеку RcppArmadillo.

Мои настройки - 

Release label:emr-5.28.0
Hadoop distribution:Amazon 2.8.5
Applications:Hive 2.3.6, Spark 2.4.4

, а вот sessionInfo () 

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Amazon Linux AMI 2018.03

Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] SparkR_2.4.4

loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1

когда я запускаю эту команду, 

install.packages("RcppArmadillo", repos = "http://cran.us.r-project.org")

возвращается ниже ошибки 

Installing package into ‘/usr/lib/spark/R/lib’
(as ‘lib’ is unspecified)
trying URL 'http://cran.us.r-project.org/src/contrib/RcppArmadillo_0.9.850.1.0.tar.gz'
Content type 'application/x-gzip' length 1381269 bytes (1.3 MB)
==================================================
downloaded 1.3 MB

Launching java with spark-submit command /usr/lib/spark/bin/spark-submit   "sparkr-shell" /tmp/RtmpmSVscJ/backend_port36976f8c01ba 
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
20/04/06 16:08:23 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
20/04/06 16:08:24 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.

 Welcome to
    ____              __ 
   / __/__  ___ _____/ /__ 
  _\ \/ _ \/ _ `/ __/  '_/ 
 /___/ .__/\_,_/_/ /_/\_\   version  2.4.4 
    /_/ 


 SparkSession available as 'spark'.
* installing *source* package ‘RcppArmadillo’ ...
** package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -m64 accepts -g... yes
checking how to run the C++ preprocessor... g++ -m64 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -m64 accepts -g... (cached) yes
checking whether g++ version is sufficient... yes, with OpenMP as version 7.2.1
checking for macOS... checking LAPACK_LIBS... R-supplied partial LAPACK found
configure: WARNING: Some complex-valued LAPACK functions may not be available
checking for OpenMP... found and suitable
configure: creating ./config.status
config.status: creating inst/include/RcppArmadilloConfigGenerated.h
config.status: creating src/Makevars
** libs
g++ -m64 -std=gnu++11 -I/usr/include/R -DNDEBUG  -I"/usr/lib/spark/R/lib/Rcpp/include" -I/usr/local/include  -I../inst/include -fopenmp -fpic  -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c RcppArmadillo.cpp -o RcppArmadillo.o
In file included from ../inst/include/RcppArmadilloForward.h:49:0,
                 from ../inst/include/RcppArmadillo.h:31,
                 from RcppArmadillo.cpp:22:
../inst/include/armadillo:92:12: fatal error: omp.h: No such file or directory
   #include <omp.h>
            ^~~~~~~
compilation terminated.
make: *** [RcppArmadillo.o] Error 1
ERROR: compilation failed for package ‘RcppArmadillo’
* removing ‘/usr/lib/spark/R/lib/RcppArmadillo’

The downloaded source packages are in
    ‘/mnt/tmp/RtmpibmiJR/downloaded_packages’
Warning message:
In install.packages("RcppArmadillo", repos = "http://cran.us.r-project.org") :
  installation of package ‘RcppArmadillo’ had non-zero exit status

Я хочу установить это, потому что я хочу установить библиотеку ChannelAttribution и это Библиотека зависит от RcppArmadillo. (Линк - https://cran.r-project.org/web/packages/ChannelAttribution/index.html)

...