ytdhy612 2020-03-18 16:51 采纳率: 0%
浏览 1387

python爬取网站图片:爬出来图片全部都不全且只能爬一张?

## 爬取的时候只有一张,有时候特别慢,而且爬出来文件全部都不全,

图片是我爬了好几次的结果

上代码:

import requests
from urllib import request
from bs4 import BeautifulSoup
import re
import random
# url网址
st_url = "https://konachan.net/post"
# 设置headers
ua = [
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1",
"Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1090.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/19.77.34.5 Safari/537.1",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.0 Safari/536.3",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
]
# 爬取网页代码
headers = {"User-Agent":random.choice(ua)}
rurl = request.Request(st_url,headers=headers)
openurl = request.urlopen(rurl)
soup = BeautifulSoup(openurl,"lxml")
# print(soup)

# 设置beautiful soup标签并爬取
attrs ={"class":"directlink largeimg" or "directlink smallimg"}
list1 =soup.find_all("a",attrs=attrs)
path = "D:\\py\\code\\php\\untitled\\img\\"
# 设置循环下载保存
number = 0
while (number <1):
    for i in list1:
        url_ = (i["href"])
        name = soup.find_all("li", attrs={"style": "width: 170px;"})
    for o in name:
        img_ = (o["id"])
    number =number+1
request.urlretrieve(url_,path+img_+".jpg")
print(img_+".jpg"+"下载完成!")

下面是爬出来的数据:
图片说明

求教各位朋友救火。多谢!!

  • 写回答

2条回答 默认 最新

  • 竟然有高手 2020-03-31 11:10
    关注

    你这写的很让我迷茫啊,你用for遍历了储存图片的列表(for o in name:)但是,重点是你把请求图片的代码(request.urlretrieve(url_,path+img_+".jpg"))放在循环的外面,你这样操作_img的值就是列表的最后一项,所以只有一张图不是很正常的事吗?知道怎么改了吧

    评论

报告相同问题?

悬赏问题

  • ¥15 素材场景中光线烘焙后灯光失效
  • ¥15 请教一下各位,为什么我这个没有实现模拟点击
  • ¥15 执行 virtuoso 命令后,界面没有,cadence 启动不起来
  • ¥50 comfyui下连接animatediff节点生成视频质量非常差的原因
  • ¥20 有关区间dp的问题求解
  • ¥15 多电路系统共用电源的串扰问题
  • ¥15 slam rangenet++配置
  • ¥15 有没有研究水声通信方面的帮我改俩matlab代码
  • ¥15 ubuntu子系统密码忘记
  • ¥15 保护模式-系统加载-段寄存器