兰超星 2022-04-29 11:12 采纳率: 50%
浏览 71
已结题

python爬虫问题求解

我想爬取西湖的门票种类和价格,但是爬取的内容为空,不知道哪方面出了问题,求解
代码如下:

from bs4 import BeautifulSoup
import pandas as pd
import requests

def crawer_travel_introduction(url):
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
req = requests.get(url, headers=headers)
content = req.text
bsObj = BeautifulSoup(content, 'lxml')
return bsObj

def get_jd_introduction(url):

cat_tickets_kind = []
cat_tickets_price= []


bsobj = crawer_travel_introduction(url)
bs = bsobj.find_all('dl', {'class': 'clrfix  '})
for j in range(0, len(bs)):
    # try:
        name = bs[j].find('dt')
        cat_tickets_kind.append(name.text)
        price = bs[j].find('dd',{'class': 'e_old_price'}).find('del')
        cat_tickets_price.append(price.text)
    # except:
    #     print('wrong')
return cat_tickets_kind, cat_tickets_price

url = 'http://travel.qunar.com/p-oi708952-xihu'
cat_tickets_kind, cat_tickets_price = get_jd_introduction(url)
city = pd.DataFrame({'tickets_kind': cat_tickets_kind, 'tickets_price': cat_tickets_price})
city.to_csv('travel_introduction.csv', encoding='utf-8')

  • 写回答

1条回答 默认 最新

  • 溪风沐雪 2022-04-29 11:43
    关注

    我给你改了一下,你对比看看吧:

    from bs4 import BeautifulSoup
    import pandas as pd
    import requests
    
    def crawer_travel_introduction(url):
        headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
        req = requests.get(url, headers=headers)
        content = req.text
        bsObj = BeautifulSoup(content, 'lxml')
        return bsObj
    
    def get_jd_introduction(url):
        cat_tickets_kind = []
        cat_tickets_price= []
        bsobj = crawer_travel_introduction(url)
        bs = bsobj.find_all('dl', {'class': 'clrfix'})
        for j in range(0, len(bs)-1):
            name = bs[j].find('dt')
            pricedd = bs[j].find('dd',{'class':'e_now_price'})
            if name is not None and pricedd is not None:
                price= pricedd.find('span',{'class':'e_price_txt'})
                cat_tickets_kind.append(name.text)
                cat_tickets_price.append(price.text)
        return cat_tickets_kind, cat_tickets_price
    url = 'http://travel.qunar.com/p-oi708952-xihu'
    cat_tickets_kind, cat_tickets_price = get_jd_introduction(url)
    print(cat_tickets_kind, cat_tickets_price)
    city = pd.DataFrame({'tickets_kind': cat_tickets_kind, 'tickets_price': cat_tickets_price})
    city.to_csv('travel_introduction.csv', encoding='utf-8')
    

    img

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

问题事件

  • 系统已结题 5月7日
  • 已采纳回答 4月29日
  • 创建了问题 4月29日

悬赏问题

  • ¥15 ajax服务器不能下载
  • ¥15 运用c++和opencv实现二维码的识别和三维坐标的建立
  • ¥15 k210烧入flash报错error:2005
  • ¥15 GIS土地利用预测FLUS模型
  • ¥15 visual studio code打不出来中文双引号
  • ¥20 关于#matlab#的问题:ps:完美符合要求有打赏(相关搜索:自适应)
  • ¥15 声音感应灯,问题出在哪里
  • ¥15 关于#matlab#的问题:(0,0)上的贝塞尔曲线,0)上的贝塞尔曲线,并使用自适应积分计算其从t=0到不大于1的任意值区间的弧长
  • ¥20 关于#r语言#rda分析作图的问题,请各位专家解答!
  • ¥50 找回 股票行情系统的资源问题