尝试根据网络上自学教程使用scrapy
开始就崩了
spider代码如下
import scrapy
class DoubanSpider(scrapy.Spider):
name = 'douban'
allowed_domains = ['moive.douban.com']
start_urls = ['https://movie.douban.com/top250']
def parse(self, response):
sel = scrapy.Selector(response)
a = sel.xpath('//div')
print(a)
# for base in base_list:
# name = base.xpath('.//div')
始终找不到元素,已经确定robot协议为False
在直接使用xpath进行解析时会得到数据,selector之后无法查询到