请教python xpath抓取数据问题

Question

本人新手，想用python 抓取目标网站的认沽/认购信息，代码如下：

import requests
from lxml import etree
#1.将目标网页的数据抓取下来
headers = {
           'User-Agent':"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0"
           }
url ='http://www.sse.com.cn/assortment/options/date/'
resp = requests.get(url,headers=headers)
text = resp.text
#print(text)
#movies = []
#2.讲抓取下来的数据按一定规则进行提取
#print(resp.text)

html = etree.HTML(text)
ul = html.xpath("//div[@class='align_right']/text()")
#for i in ul:
print(ul)  

结果如下：

runfile('E:/Py file/practice/未命名0.py', wdir='E:/Py file/practice')
[]

显示没有，但我用Chrome 的 xpath helper却能查到好几个，请问应该如何解决，信息

 

Answer 1

看了一下，很明显这些数据都是通过js向后台获取的数据

接口是http://sia.sseinfo.com/noc.gif?WS=10000042&RD=record&SWS=&SWSID=&SWSPID=&JSVER=20170109&TDT=&UC=_ck20111316512818492490491945352&LUC=&VUC=_vk1605257488799&UN=&UID=&RC=0&PS=www.sse.com.cn&PU=%2Fassortment%2Foptions%2Fdate%2F&PT=&FS=&RF=&KW=&SS=2560*1440&SW=2543&SCW=2543&SCH=1308&SSH=1308&BR=chrome&LTL=6&MSRC=&EDM=&CLE=||msmv::17663::472::596::::09||msmv::18436::524::670::::09||msmv::19190::645::910::::09&MID=1605257488799233&CHK=118&RDM=0.6934725580305516

1-2秒钟刷新一次