Разделить столбец String с несколькими разделителями на массив

Question

У меня есть Dataframe с таким столбцом, как этот

Title
"Over the Hill,to the Poorhouse"
"Wilson"                                    
"Darling Lili"                            
"The Ten Commandments"                      
"12 Angry Men"                              
"Twelve Monkeys"                            
"1776"                                      
"1941"                                                                                
"Chacun sa nuit"                                                            
"2001: A Space Odyssey"                                            
"20,000 Leagues Under the Sea"                             
"20,000 Leagues Under the Sea"                             
"24,7: Twenty Four Seven"                                       
"Twin Falls Idaho"                                                        
"Three Kingdoms: Resurrection of the Dragon"
.......
.......

, и я хотел бы преобразовать этот столбец в такой массив.

[Over, the, Hill, to, the, Poorhouse] 
[Wilson] 
[Darling, Lili]                                   
[The, Ten, Commandments]  
[12, Angry, Men]
[Twelve, Monkeys] 
[1776]   
[1941] 
[Chacun, sa, nuit]   
[2001, , A, Space, Odyssey] 
[20, 000, Leagues, Under, the, Sea]
[20, 000, Leagues, Under, the, Sea]
[24, 7, , Twenty, Four, Seven]
[Twin, Falls, Idaho]
[Three, Kingdoms, , Resurrection, of, the, Dragon]

, чтобы у меня были эти два столбцы

Title                     Title_Words
Over the Hill to the Poorhouse            [Over, the, Hill, to, the, Poorhouse]             
Wilson                                    [Wilson]                                          
Darling Lili                              [Darling, Lili]                                   
The Ten Commandments                      [The, Ten, Commandments]                          
12 Angry Men                              [12, Angry, Men]                                  
Twelve Monkeys                            [Twelve, Monkeys]                                 
1776                                      [1776]                                            
1941                                      [1941]                                            
Chacun sa nuit                            [Chacun, sa, nuit]                                
2001: A Space Odyssey                     [2001, , A, Space, Odyssey]                       
20,000 Leagues Under the Sea              [20, 000, Leagues, Under, the, Sea]               
20,000 Leagues Under the Sea              [20, 000, Leagues, Under, the, Sea]               
24 7: Twenty Four Seven                   [24, 7, , Twenty, Four, Seven]                    
Twin Falls Idaho                          [Twin, Falls, Idaho]                              
Three Kingdoms: Resurrection of the Dragon[Three, Kingdoms, , Resurrection, of, the, Dragon]

Проблема в том, что строка может иметь несколько разделителей: пробелы, запятая, двоеточие.

Как это можно сделать?

Answer 1

Попробуйте это -

df.withColumn("Title_Words", split(col("Title"), "\\s+|[,:]"))