ytdhy612 2020-03-18 16:51 采纳率: 0%
浏览 1386

python爬取网站图片:爬出来图片全部都不全且只能爬一张?

## 爬取的时候只有一张,有时候特别慢,而且爬出来文件全部都不全,

图片是我爬了好几次的结果

上代码:

import requests
from urllib import request
from bs4 import BeautifulSoup
import re
import random
# url网址
st_url = "https://konachan.net/post"
# 设置headers
ua = [
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1",
"Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1090.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/19.77.34.5 Safari/537.1",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.0 Safari/536.3",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
]
# 爬取网页代码
headers = {"User-Agent":random.choice(ua)}
rurl = request.Request(st_url,headers=headers)
openurl = request.urlopen(rurl)
soup = BeautifulSoup(openurl,"lxml")
# print(soup)

# 设置beautiful soup标签并爬取
attrs ={"class":"directlink largeimg" or "directlink smallimg"}
list1 =soup.find_all("a",attrs=attrs)
path = "D:\\py\\code\\php\\untitled\\img\\"
# 设置循环下载保存
number = 0
while (number <1):
    for i in list1:
        url_ = (i["href"])
        name = soup.find_all("li", attrs={"style": "width: 170px;"})
    for o in name:
        img_ = (o["id"])
    number =number+1
request.urlretrieve(url_,path+img_+".jpg")
print(img_+".jpg"+"下载完成!")

下面是爬出来的数据:
图片说明

求教各位朋友救火。多谢!!

  • 写回答

2条回答 默认 最新

  • 竟然有高手 2020-03-31 11:10
    关注

    你这写的很让我迷茫啊,你用for遍历了储存图片的列表(for o in name:)但是,重点是你把请求图片的代码(request.urlretrieve(url_,path+img_+".jpg"))放在循环的外面,你这样操作_img的值就是列表的最后一项,所以只有一张图不是很正常的事吗?知道怎么改了吧

    评论

报告相同问题?

悬赏问题

  • ¥15 MCNP里如何定义多个源?
  • ¥20 双层网络上信息-疾病传播
  • ¥50 paddlepaddle pinn
  • ¥20 idea运行测试代码报错问题
  • ¥15 网络监控:网络故障告警通知
  • ¥15 django项目运行报编码错误
  • ¥15 请问这个是什么意思?
  • ¥15 STM32驱动继电器
  • ¥15 Windows server update services
  • ¥15 关于#c语言#的问题:我现在在做一个墨水屏设计,2.9英寸的小屏怎么换4.2英寸大屏