Литература по проверке правописания? - PullRequest
1 голос
/ 31 мая 2011

Мне было интересно, есть ли хороший список литературы о том, как реализовать проверку орфографии.Один пример, который я могу найти, - «Как написать корректор орфографии» Питера Норвига - http://norvig.com/spell-correct.html очень нереально.

Несколько вещей, которые меня интересуют:

  • Построение проверки орфографии безприбегая к словарю (либо используя существующие корпуса, дамп N-граммы, например дамп Google NGram).
  • Контекстная проверка орфографии.

Ответы [ 2 ]

1 голос
/ 01 июня 2011

Вот классическая статья: Church & Gale (1991) . Там было меньше работы по контекстно-чувствительной коррекции ошибок, но, вероятно, стоит обратить внимание на две статьи: Golding (1995) и Carlson & Fette (2007) .

0 голосов
/ 01 июня 2011

Цитата по ссылке ниже

How does it Work?
The Basic Model
The basic technology works as follows: The documents that the search engine is providing access to are added both to the search index and a language model. The language model stores seen phrases and maintains statistics about them. When a query is submitted, the src/QuerySpellCheck.java class looks for possible character edits, namely substitutions, insertions, replacements, transpositions, and deletions, that make the query a better fit for the lanaguage model. So if you type 'Gretski' as a query, and the underlying data is data from rec.sport.hockey, the language model will be much more familliar with the mildly edited 'Gretzky' and suggests it as an alternative.
Domain Sensitivity
The big advantage of this approach over dictionary-based spell checking is that the corrections are motivated by data in the search index. So "trt" will be corrected to "tort" in a legal domain, "tart" in a cooking domain, and "TRt" in a bio-informatics domain. On Google, there is no suggested correction, presumably because of web domains "trt.com", Thessaly Radio Television as well as Turkiye Radyo Televizyon, both aka TRT, etc.
Context-Sensitive Correction
Both Yahoo and Google perform context-sensitive correction. For instance, the query frod (an Old English term from the German meaning wise or experienced) has a suggested correction of ford (the automotive company, among others), whereas the query frod baggins has the corrected query frodo baggins (a 20th century English fictional character). That's the Yahoo behavior. Google doesn't correct frod baggins, even though there are about 785 hits for it versus 820,000 for Frodo Baggins. On the other hand, Google does correct frdo and frdo baggins. Amazon behaves similarly, but MSN corrects frd baggins to ford baggins rather than frodo baggins.
LingPipe's model supports exactly this kind of context-sensitive correction.

прочитайте этот замечательный урок

...