兰超星 2022-04-29 11:12 采纳率: 50%
浏览 66
已结题

python爬虫问题求解

我想爬取西湖的门票种类和价格,但是爬取的内容为空,不知道哪方面出了问题,求解
代码如下:

from bs4 import BeautifulSoup
import pandas as pd
import requests

def crawer_travel_introduction(url):
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
req = requests.get(url, headers=headers)
content = req.text
bsObj = BeautifulSoup(content, 'lxml')
return bsObj

def get_jd_introduction(url):

cat_tickets_kind = []
cat_tickets_price= []


bsobj = crawer_travel_introduction(url)
bs = bsobj.find_all('dl', {'class': 'clrfix  '})
for j in range(0, len(bs)):
    # try:
        name = bs[j].find('dt')
        cat_tickets_kind.append(name.text)
        price = bs[j].find('dd',{'class': 'e_old_price'}).find('del')
        cat_tickets_price.append(price.text)
    # except:
    #     print('wrong')
return cat_tickets_kind, cat_tickets_price

url = 'http://travel.qunar.com/p-oi708952-xihu'
cat_tickets_kind, cat_tickets_price = get_jd_introduction(url)
city = pd.DataFrame({'tickets_kind': cat_tickets_kind, 'tickets_price': cat_tickets_price})
city.to_csv('travel_introduction.csv', encoding='utf-8')

  • 写回答

1条回答 默认 最新

  • 溪风沐雪 2022-04-29 11:43
    关注

    我给你改了一下,你对比看看吧:

    from bs4 import BeautifulSoup
    import pandas as pd
    import requests
    
    def crawer_travel_introduction(url):
        headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
        req = requests.get(url, headers=headers)
        content = req.text
        bsObj = BeautifulSoup(content, 'lxml')
        return bsObj
    
    def get_jd_introduction(url):
        cat_tickets_kind = []
        cat_tickets_price= []
        bsobj = crawer_travel_introduction(url)
        bs = bsobj.find_all('dl', {'class': 'clrfix'})
        for j in range(0, len(bs)-1):
            name = bs[j].find('dt')
            pricedd = bs[j].find('dd',{'class':'e_now_price'})
            if name is not None and pricedd is not None:
                price= pricedd.find('span',{'class':'e_price_txt'})
                cat_tickets_kind.append(name.text)
                cat_tickets_price.append(price.text)
        return cat_tickets_kind, cat_tickets_price
    url = 'http://travel.qunar.com/p-oi708952-xihu'
    cat_tickets_kind, cat_tickets_price = get_jd_introduction(url)
    print(cat_tickets_kind, cat_tickets_price)
    city = pd.DataFrame({'tickets_kind': cat_tickets_kind, 'tickets_price': cat_tickets_price})
    city.to_csv('travel_introduction.csv', encoding='utf-8')
    

    img

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

    报告相同问题?

    问题事件

    • 系统已结题 5月7日
    • 已采纳回答 4月29日
    • 创建了问题 4月29日

    悬赏问题

    • ¥20 有人能代写linux应用吗?有偿,酬金可商量,具体要求联系我
    • ¥15 神经网络建立应力预测模型
    • ¥20 反编译一款APP手游,解密出lua脚本。
    • ¥15 改一下,代码,让代码符合要求
    • ¥15 报错问题,只有两个错误,如图片所示,i多次初始化
    • ¥15 matlab runtime 如何收费
    • ¥15 Tpad api账户 api口令
    • ¥30 ppt进度条制作,vba语言
    • ¥15 生信simpleaffy包下载
    • ¥15 在二层网络中,掩码存在包含关系即可通信