Удалить все знаки препинания из текста, включая апострофы для пакета tm

Question

У меня есть вектор, состоящий из твитов (просто текст сообщения), который я очищаю для целей анализа текста.Я использовал removePunctuation из пакета tm следующим образом:

clean_tweet_text = removePunctuation(tweet_text)

Это привело к вектору со всеми пунктуациями, удаленными из текста , за исключением апострофов, что разрушает мое ключевое словопоиск, потому что слова, затрагивающие апострофы, не зарегистрированы.Например, одним из моих ключевых слов является climate, но если твит имеет 'climate, он не будет засчитан.

Как я могу удалить все апострофы / одинарные кавычки из моего вектора?

Вот заголовок из dput для воспроизводимого примера:

c("expert briefing on climatechange disarmament sdgs nmun httpstco5gqkngpkap", 
"who uses nasa earth science data he looks at impact of aerosols on climateamp weather httpstcof4azsiqkw1 https…", 
"rt oddly enough some republicans think climate change is real oddly enough… httpstcomtlfx1mnuf uniteblue https…", 
"better dead than red bill gates says that only socialism can save us from climate change httpstcopypqmd1fok", 
"i see red people bill gates says that only socialism can save us from climate change httpstcopypqmd1fok", 
"why go for ecosystem basses conservation climatechange raajje maldives ecocaremv httpstcorauhjbasyl", 
"ted cruz ‘climate change is not science it’s religion’ httpstco0qqtbofe0h via glennbeck", 
"unusual warming kills gulf of maine cod  discovery news globalwarming  httpstco39uvock3xe", 
"this is an amusing headline bill gates says that only socialism can save us from climate change httpstcobfs5zbcijc", 
"what do the remaining republican candidates have to say about climate change fixgov httpstcoxpszwbrcnh httpstcodgqyidkw6o"
)

Answer 1

Чтобы удалить все знаки пунктуации (включая апострофы и одинарные кавычки), вы можете просто использовать gsub():

x <- c("expert briefing on climatechange disarmament sdgs nmun httpstco5gqkngpkap",
       "who uses nasa earth science data he looks at impact of aerosols on climateamp weather httpstcof4azsiqkw1 https…",
       "rt oddly enough some republicans think climate change is real oddly enough… httpstcomtlfx1mnuf uniteblue https…",
       "better dead than red bill gates says that only socialism can save us from climate change httpstcopypqmd1fok",
       "i see red people bill gates says that only socialism can save us from climate change httpstcopypqmd1fok",
       "why go for ecosystem basses conservation climatechange raajje maldives ecocaremv httpstcorauhjbasyl",
       "ted cruz ‘climate change is not science it’s religion’ httpstco0qqtbofe0h via glennbeck",
       "unusual warming kills gulf of maine cod discovery news globalwarming httpstco39uvock3xe",
       "this is an amusing headline bill gates says that only socialism can save us from climate change httpstcobfs5zbcijc",
       "what do the remaining republican candidates have to say about climate change fixgov httpstcoxpszwbrcnh httpstcodgqyidkw6o")

gsub("[[:punct:]]", "", x)
#>  [1] "expert briefing on climatechange disarmament sdgs nmun httpstco5gqkngpkap"                                                
#>  [2] "who uses nasa earth science data he looks at impact of aerosols on climateamp weather httpstcof4azsiqkw1 https"           
#>  [3] "rt oddly enough some republicans think climate change is real oddly enough httpstcomtlfx1mnuf uniteblue https"            
#>  [4] "better dead than red bill gates says that only socialism can save us from climate change httpstcopypqmd1fok"              
#>  [5] "i see red people bill gates says that only socialism can save us from climate change httpstcopypqmd1fok"                  
#>  [6] "why go for ecosystem basses conservation climatechange raajje maldives ecocaremv httpstcorauhjbasyl"                      
#>  [7] "ted cruz climate change is not science its religion httpstco0qqtbofe0h via glennbeck"                                     
#>  [8] "unusual warming kills gulf of maine cod discovery news globalwarming httpstco39uvock3xe"                                  
#>  [9] "this is an amusing headline bill gates says that only socialism can save us from climate change httpstcobfs5zbcijc"       
#> [10] "what do the remaining republican candidates have to say about climate change fixgov httpstcoxpszwbrcnh httpstcodgqyidkw6o"

gsub() заменяет все вхождения первого аргумента в третьем аргументе на второй аргумент(см. help("gsub")).Здесь это означает, что он заменяет все вхождения в нашем векторе x любого из символов в наборе [[:punct:]] на "" (удалить их).

Какие символы удаляются?С help("regex"):

[: punct:]

Знаки пунктуации:
!"# $% & '() * +, -. /:; <=>? @ [\] ^ _` {|} ~.

Обновление

Появляетсяэто происходит потому, что ваши апострофы похожи на ‘ вместо '. Итак, если вы хотите придерживаться tm::removePunctuation(), вы также можете использовать

tm::removePunctuation(x, ucp = TRUE)
#>  [1] "expert briefing on climatechange disarmament sdgs nmun httpstco5gqkngpkap"                                                
#>  [2] "who uses nasa earth science data he looks at impact of aerosols on climateamp weather httpstcof4azsiqkw1 https"           
#>  [3] "rt oddly enough some republicans think climate change is real oddly enough httpstcomtlfx1mnuf uniteblue https"            
#>  [4] "better dead than red bill gates says that only socialism can save us from climate change httpstcopypqmd1fok"              
#>  [5] "i see red people bill gates says that only socialism can save us from climate change httpstcopypqmd1fok"                  
#>  [6] "why go for ecosystem basses conservation climatechange raajje maldives ecocaremv httpstcorauhjbasyl"                      
#>  [7] "ted cruz climate change is not science its religion httpstco0qqtbofe0h via glennbeck"                                     
#>  [8] "unusual warming kills gulf of maine cod discovery news globalwarming httpstco39uvock3xe"                                  
#>  [9] "this is an amusing headline bill gates says that only socialism can save us from climate change httpstcobfs5zbcijc"       
#> [10] "what do the remaining republican candidates have to say about climate change fixgov httpstcoxpszwbrcnh httpstcodgqyidkw6o"