"" имеет особое значение в регулярных выражениях. Побег, чтобы соответствовать периоду
MatchCollection match = Regex.Matches(entireText, @"alphabet\.");
Edit:
Полный код, дающий ожидаемый результат:
string entireText = @"The quick brown fox jumps over the lazy dog is an English-language pangram, alphabet! that is, a phrase that contains all of the letters of the alphabet. It has been used to test typewriters alphabet. and computer keyboards, and in other applications involving all of the letters in the English alphabet.";
MatchCollection matches = Regex.Matches(entireText, @"alphabet\.");
foreach (Match match in matches)
{
foreach (Group group in match.Groups)
{
Console.WriteLine(group);
}
}