python3 页面抓取怎么设置header

Question

url = 'http://used.21-sun.com/'
user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36'
headers = { 'User-Agent' : user_agent }
data = None
page = urllib.request.urlopen(url)
content = page.read().decode('utf-8')
pattern = re.compile(r'

.*?(.*?).*?(.*?)<!--(.*?)-->.*?(.*?)(.*?)',re.S)
items = re.findall(pattern,content)
这么写的代码一直报错

Answer 1

报错指的什么？还要看对应的网站是不是会检查其他信息来识别你是否为爬虫