Как решить count () в pyspark

Question

Я использую этот учебник: https://supergloo.com/spark-python/apache-spark-quick-start-with-python/.

Я пробовал:

ut = s c .textFile ("Uber-Jan-Feb-FOIL.csv")

ut.count ()

Uber-Jan-Feb-FOIL.csv находится в моем рабочем каталоге.

Я получаю следующее:

  File "<stdin>", line 1, in <module>
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/pyspark/rdd.py", line 1055, in count
    return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum()
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/pyspark/rdd.py", line 1046, in sum
    return self.mapPartitions(lambda x: [sum(x)]).fold(0, operator.add)
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/pyspark/rdd.py", line 917, in fold
    vals = self.mapPartitions(func).collect()
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/pyspark/rdd.py", line 816, in collect
    sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
  File "/opt/apache-spark/spark-2.4.5-bin-hadoop2.7/python/pyspark/sql/utils.py", line 79, in deco
    raise IllegalArgumentException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.IllegalArgumentException: u'Unsupported class file major version 55'```