Merge pull request #1021 from misnyo/google_news

[fix] google news dom xpath fix
This commit is contained in:
Adam Tauber 2017-09-03 23:08:26 +02:00 committed by GitHub
commit 806cb08750
2 changed files with 58 additions and 8 deletions

View file

@ -67,8 +67,8 @@ def response(resp):
for result in dom.xpath('//div[@class="g"]|//div[@class="g _cy"]'):
try:
r = {
'url': result.xpath('.//div[@class="_cnc"]//a/@href')[0],
'title': ''.join(result.xpath('.//div[@class="_cnc"]//h3//text()')),
'url': result.xpath('.//a[@class="l _PMs"]')[0].attrib.get("href"),
'title': ''.join(result.xpath('.//a[@class="l _PMs"]//text()')),
'content': ''.join(result.xpath('.//div[@class="st"]//text()')),
}
except:

File diff suppressed because one or more lines are too long