python中用xpath寻找href返回的是空列表(页面源代码有href,xpath路径中没有tbody)
mainurl='https://www.biqugesc.com/28/28625/'
kv={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36 Edg/101.0.1210.47',
"Referer":"https://www.so.com/link?m=baLB1+ld67jrXHmwc7UyUE81E/0ijPpC9/KIOiunbk+LsXrDOk37Tgx8HGJKjjd7u3cXq+2Xs2Fkx6cePZRxyyorxLhCWGsnrlChMOcBWs5Vc7BQHDgRH/GbHBbpytQKTBXFNY+2PAZybNNdDJba+35l6Y2InlSHpCAonBF4pnCmRpYGaunzdq3C11Urie5lOuS26AJcxuGpGYHPYxpRZOg==",
"Cookie": "Hm_lvt_b2b641b8bf68e8104f212725ea3188ed=1653143302; Hm_lpvt_b2b641b8bf68e8104f212725ea3188ed=1653143326"}
def main():
resp=requests.get(mainurl,headers=kv)
html=etree.HTML(resp.text)
for i in range(1,10):
url=html.xpath('/html/body/div/div[7]/div/ul/li[{}]/a//@href'.format(i))
print(url)
if __name__ == '__main__':
main()
[]
一开始我以为是referer或者cookie的问题,但我都试过了还是返回空列表
精确得到href