关于#python#的问题：用Python爬取网页时，直接运行for语句下的代码可正常运行，加入for语句进行循环则报错

Question

用Python爬取网页时，直接运行for语句下的代码可正常运行，加入for语句进行循环则报错

遇到的现象和发生背景，请写出第一个错误信息

def getHTMLText(url_):
    try:
        r = requests.get(url_, timeout=30)
        r.raise_for_status()  # 如果状态不是200，引发HTTPError异常
        r.encoding = r.apparent_encoding
        return r.text
    except:
        return "产生异常"


def getSoup(url_):
    html_ = getHTMLText(url_)
    soup_ = BeautifulSoup(html_, 'lxml')
    return soup_


if __name__ == "__main__":
    year = '23'
    month = '01'
    day = '04'
    page = '01'
    for page in range(1, 9): # 注释掉本行则可正常运行
        target = f"http://paper.people.com.cn/rmrb/html/20{year}-{month}/{day}/nbs.D110000renmrb_{page}.htm"
        soup = getSoup(target)
        title_list = soup.find('ul',class_='news-list')
        title_list = title_list.find_all('a')
        for title in title_list:
            print(title.string)

运行结果及详细报错内容Traceback (most recent call last):
  File "C:\Users\songz\PycharmProjects\pythonProject1\spiders\spiderNews.py", line 47, in <module>
    title_list = title_list.find_all('a')
AttributeError: 'NoneType' object has no attribute 'find_all'

如何能通过for语句循环运行？

Answer 1

01.html和1.html很显然不是同一个网址，你在错误的网址下当然抓不到东西，是空的
改成

target = f"http://paper.people.com.cn/rmrb/html/20{year}-{month}/{day}/nbs.D110000renmrb_{page:02d}.htm"