Извлечение предложений из текста без остановок - PullRequest
0 голосов
/ 05 февраля 2019

Я новичок в НЛП и мне нужна помощь в извлечении предложений из текста, который не содержит полных остановок.

Например: (текст ниже не содержит полных остановок)

global warming is the term used to describe a gradual increase in the average
temperature of the earths atmosphere and its oceans a change that is believed to 
be permanently changing the earths climate there is great debate among many 
people and sometimes in the news on whether global warming is real (some call it 
a hoax) but climate scientists looking at the data and facts agree the planet is 
warming while many view the effects of global warming to be more substantial and 
more rapidly occurring than others do the scientific consensus on climatic 
changes related to global warming is that the average temperature of the earth 
has risen between 04 and 08 °c over the past 100 years the increased volumes of 
carbon dioxide and other greenhouse gases released by the burning of fossil 
fuels land clearing agriculture and other human activities are believed to be 
the primary sources of the global warming that has occurred over the past 50 
years scientists from the intergovernmental panel on climate carrying out global 
warming research have recently predicted that average global temperatures could 
increase between 14 and 58 °c by the year 2100 changes resulting from global 
warming may include rising sea levels due to the melting of the polar ice caps 
as well as an increase in occurrence and severity of storms and other severe 
weather events

Есть ли какая-нибудь библиотека НЛП, которую я могу использовать для извлечения предложений сверху.

Ожидаемый результат:

global warming is the term used to describe a gradual increase in the average temperature of the earths atmosphere and its oceans a change that is believed to be permanently changing the earths climate
there is great debate among many people and sometimes in the news on whether global warming is real (some call it a hoax)
but climate scientists looking at the data and facts agree the planet is warming
while many view the effects of global warming to be more substantial and more rapidly occurring than others do the scientific consensus on climatic changes related to global warming is that the average temperature of the earth has risen between 4 and 8 °c over the past 100 years
the increased volumes of carbon dioxide and other greenhouse gases released by the burning of fossil fuels land clearing agriculture and other human activities are believed to be the primary sources of the global warming that has occurred over the past 50 years
scientists from the intergovernmental panel on climate carrying out global warming research have recently predicted that average global temperatures could increase between 4 and 8 °c by the year 2100
changes resulting from global warming may include rising sea levels due to the melting of the polar ice caps as well as an increase in occurrence and severity of storms and other severe weather events

Спасибо.

1 Ответ

0 голосов
/ 05 февраля 2019

На самом деле понимание того, где предложение начинается и заканчивается, является относительно дорогой операцией.В вашем случае вы могли бы составить список слов, которые не будут начинаться с предложений, например «Земля» и «Межправительственный», разбить текст на слова, обнаружить заглавные слова (это будут начинающиеся предложения) и объединить подсписки встроки.Вот как я это сделал:

a = """
Global warming is the term used to describe a gradual increase in the average 
temperature of the Earths atmosphere and its oceans a change that is believed
to be permanently changing the Earths climate There is great debate among many
people and sometimes in the news on whether global warming is real (some call 
it a hoax) But climate scientists looking at the data and facts agree the 
planet is warming While many view the effects of global warming to be more 
substantial and more rapidly occurring than others do the scientific consensus 
on climatic changes related to global warming is that the average temperature 
of the Earth has risen between 4 and 8 °C over the past 100 years The 
increased volumes of carbon dioxide and other greenhouse gases released by the 
burning of fossil fuels land clearing agriculture and other human activities 
are believed to be the primary sources of the global warming that has occurred
over the past 50 years Scientists from the Intergovernmental Panel on Climate 
carrying out global warming research have recently predicted that average 
global temperatures could increase between 4 and 8 °C by the year 2100 
Changes resulting from global warming may include rising sea levels due to the 
melting of the polar ice caps as well as an increase in occurrence and severity 
of storms and other severe weather events
"""

preProc = a.replace("\n", "")
preProc = preProc.split(" ")

capitalizedWords = ['Intergovernmental','Panel', 'Climate', 'Earths', 'Earth', '°C']
results = []
previousIndex = 0
for idx, word in enumerate(preProc):
  if word.istitle() and word not in capitalizedWords:
    results.append(list(preProc[previousIndex:idx]))
    previousIndex = idx
results = [" ".join(x) for x in results]

print(results)
Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...