初学python爬虫，爬取不到网页的阅读数是怎么回事？求大神解答

Question

import scrapy


class JobboleSpider(scrapy.Spider):
    name = 'jobbole'
    allowed_domains = ['blog.jobbole.com']
    start_urls = ['http://blog.jobbole.com/gncj/170427.html']

    def parse(self, response):
        re_selector = response.xpath("/html/body/div[3]/div[1]/div[3]/div[1]/h1")
        title = response.xpath('//div[@class="article-head"]/h1/text()').extract()
        create_time = response.xpath("//div[@class='article-detail bgc-fff']/div[1]/div/div/span/text()").extract()[0]
        read_num =  response.xpath("/html/body/div[3]/div[1]/div[3]/div[1]/div/div/span[2]/text()").extract()

read_num无法提取数据

 

 

我通过xpath只能提取到")",这是为啥？请教！！！！

Answer 1

数据是js动态渲染的，scrapy需结合splash使用，用selenium速度虽慢点，但是对js加载数据的获取准确性较高。

driver.implicitly_wait(5)

#page=driver.page_source

read_num = driver.find_element_by_xpath(

    "/html/body/div[3]/div[1]/div[3]/div[1]/div/div/span[2]").text

print(read_num)

##output:

阅读(18)