Я изменил ваш код одним касанием, чтобы он был работоспособным, и, кажется, он "работает" правильно для меня.Полученный результат имеет окончания строк там, где ожидается.Я вижу некоторые проблемы с кодировкой, но нет проблем с окончанием строки.
Код
import urllib2
class BottomPipeResult :
AGENT_ID = "Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
BOTTOMPIPE_URL = "http://boilerpipe-web.appspot.com/extract?url={0}&extractor=LargestContentExtractor&output=text"
_myBPPage = ""
# scrape and get results from bottompipe
def scrapeResult(self, theURL, user_agent=AGENT_ID) :
request = urllib2.Request(self.BOTTOMPIPE_URL.format(theURL))
if user_agent:
request.add_header("User-Agent", user_agent)
pagefile = urllib2.urlopen(request)
realurl = pagefile.geturl()
f = pagefile
self._myBPPAge = f.read()
return(self._myBPPAge)
bpr = BottomPipeResult()
myLinkResult = 'http://result.com'
f = open('out.html', 'a')
f.write(bpr.scrapeResult(myLinkResult))
f.close()
Вывод
Result-Expand.flv
We want to help your company grow. Our Result offices around the world can help you expand your business faster and more cost efficiently. And at the same time bring the experience of having expanded more than 150 companies during the past ten years.
Result can help you grow in your local market, regionally, or globally through our team of experienced business builders, our industry know-how and our know-who.
Our services range from well designed expansion strategies to assuming operational responsibility for turning these strategies into successful business.
We don’t see ourselves as mere consultants who give you a strategy presentation and then leave you to your own devices. We prefer to be considered as an extended, entirely practical arm of your management team. We’re hands-on and heads-on. We’re business builders.
We’re co-entrepreneurs. This is also reflected in our compensation structure – a significant part of our compensation is result  based.
Делаем результаты более "HTML", как
Что касается вывода html, вы, вероятно, хотите заключить каждую строку в тег <p>
абзаца.
output = BottomPipeResult.scrapeResult(myLinkResult)
f.write('\n'.join(['<p>' + x + '</p>' for x in output.split('\n')]))