Показать конкретное значение в искровой sql

Question

вот моя часть Json, которую я преобразовал в Df

{"business_id": "vcNAWiLM4dR7D2nwwJ7nCA", "full_address": "4840 E Indian School Rd\nSte 101\nPhoenix, AZ 85018", "hours": {"Tuesday": {"close": "17:00", "open": "08:00"}, "Friday": {"close": "17:00", "open": "08:00"}, "Monday": {"close": "17:00", "open": "08:00"}, "Wednesday": {"close": "17:00", "open": "08:00"}, "Thursday": {"close": "17:00", "open": "08:00"}}, "open": true, "categories": ["Doctors", "Health & Medical"], "city": "Phoenix", "review_count": 9, "name": "Eric Goldberg, MD", "neighborhoods": [], "longitude": -111.98375799999999, "state": "AZ", "stars": 3.5, "latitude": 33.499313000000001, "attributes": {"By Appointment Only": true}, "type": "business"} ... и многое другое

Теперь мне нужно отобразить только время открытия и закрытия всех вторников для всего бизнеса. Я попытался с помощью isin с условием filter. Но это не сработало.

Answer 1

Если датафрейм загружен правильно с этой схемой (я загрузил ваш пример с spark.read.json):

    scala> df.printSchema
    root
    |-- attributes: struct (nullable = true)
    |    |-- By Appointment Only: boolean (nullable = true)
    |-- business_id: string (nullable = true)
    |-- categories: array (nullable = true)
    |    |-- element: string (containsNull = true)
    |-- city: string (nullable = true)
    |-- full_address: string (nullable = true)
    |-- hours: struct (nullable = true)
    |    |-- Friday: struct (nullable = true)
    |    |    |-- close: string (nullable = true)
    |    |    |-- open: string (nullable = true)
    |    |-- Monday: struct (nullable = true)
    |    |    |-- close: string (nullable = true)
    |    |    |-- open: string (nullable = true)
    |    |-- Thursday: struct (nullable = true)
    |    |    |-- close: string (nullable = true)
    |    |    |-- open: string (nullable = true)
    |    |-- Tuesday: struct (nullable = true)
    |    |    |-- close: string (nullable = true)
    |    |    |-- open: string (nullable = true)
    |    |-- Wednesday: struct (nullable = true)
    |    |    |-- close: string (nullable = true)
    |    |    |-- open: string (nullable = true)
    |-- latitude: double (nullable = true)
    |-- longitude: double (nullable = true)
    |-- name: string (nullable = true)
    |-- neighborhoods: array (nullable = true)
    |    |-- element: string (containsNull = true)
    |-- open: boolean (nullable = true)
    |-- review_count: long (nullable = true)
    |-- stars: double (nullable = true)
    |-- state: string (nullable = true)
    |-- type: string (nullable = true)

вы можете сделать

    scala> df.select("hours.Tuesday").show
    +--------------+
    |       Tuesday|
    +--------------+
    |[17:00, 08:00]|
    +--------------+

, если вы хотите просто закрыть времявы можете сделать

scala> df.select("hours.Tuesday.close").show
+-----+
|close|
+-----+
|17:00|
+-----+