python3 请求网页 Request和urlopen的区别

urllib.request.Request和urllib.request.urlopen的区别?求教

1个回答

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
Python+OpenCV计算机视觉

Python+OpenCV计算机视觉

python3.65 urllib.request.Request()增加header报错

# coding:utf-8 import urllib.request import urllib.parse url = 'http://192.168.**.**:9080/api/transactions' header = { 'Content-Type': 'application/json' } values ={ "currentToken": { "simplifiedName": "ETH", "address": "0x5bcd4d0508bc86c48760d0805962261d260d7a88" }, "txid": "" } data = urllib.parse.urlencode(values) data = data.encode(encoding='UTF-8') request = urllib.request.Request(url, data, header) #request = urllib.request.Request(url, data) print("111") html = urllib.request.urlopen(request) print("222") html =html.read().decode('utf-8') print(html) #返回结果 报错 D:\tool\Python36\python.exe D:/Users/Administrator/PycharmProjects/coinPlatform/test/test6.py 111 Traceback (most recent call last): File "D:/Users/Administrator/PycharmProjects/coinPlatform/test/test6.py", line 21, in <module> html = urllib.request.urlopen(request) File "D:\tool\Python36\lib\urllib\request.py", line 223, in urlopen return opener.open(url, data, timeout) File "D:\tool\Python36\lib\urllib\request.py", line 532, in open response = meth(req, response) File "D:\tool\Python36\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "D:\tool\Python36\lib\urllib\request.py", line 570, in error return self._call_chain(*args) File "D:\tool\Python36\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "D:\tool\Python36\lib\urllib\request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found Process finished with exit code 1 # coding:utf-8 import urllib.request import urllib.parse url = 'http://192.168.**.**:9080/api/transactions' header = { 'Content-Type': 'application/json' } values ={ "currentToken": { "simplifiedName": "ETH", "address": "0x5bcd4d0508bc86c48760d0805962261d260d7a88" }, "txid": "" } data = urllib.parse.urlencode(values) data = data.encode(encoding='UTF-8') #request = urllib.request.Request(url, data, header) request = urllib.request.Request(url, data) print("111") html = urllib.request.urlopen(request) print("222") html =html.read().decode('utf-8') print(html) 返回结果 成功 但没得到想要的结果 D:\tool\Python36\python.exe D:/Users/Administrator/PycharmProjects/coinPlatform/test/test6.py 111 222 {"code":0,"message":"success","data":{"currentToken":{},"transactions":[]}} Process finished with exit code 0

python 中的urllib2.urlopen()方法

python中,urllib2中的urlopen()方法可以这样用: ``` response=urllib2.urlopen("http://www.baidu.com") html=response.read() ``` 也可以这样用:先创建一个Request对象 ``` request=urllib2.Request("http://www.baidu.com") response=urllib2.urlopen(request) html=response.read() ``` 查看urllib2.urlopen()方法: ``` urlopen(url, data=None, timeout=<object object>, cafile=None, capath=None, cadefault=False, context=None) ``` 其中并没有Request类型的参数,但是为什么可以这样用呢?

python3爬取豆瓣读书的API报错400,bad request

使用如下python代码读取配置文件中的linkLists(https://api.douban.com/v2/book/1006639),每次爬取到100条左右时就报错bad request 请教各位大神有木有遇到过这种情况,该怎么解决 import json import pymysql import random import time import urllib from urllib import request import doubans.config.linkconfig conn = pymysql.connect( user="root",password="root",database="python",charset='utf8') cur = conn.cursor() def getDetail(url): req = urllib.request.Request(url) req.add_header('User-Agent','Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36') try : res = urllib.request.urlopen(req) jsondata = res.read().decode("UTF-8") res.close() result = json.loads(jsondata) except urllib.error.URLError as e : print(e.reason) if ('title' in result.keys()): title = result["title"] else: title = "" if ('isbn13' in result.keys() and 'ibsn13' != ""): isbn = result["isbn13"] else: isbn = result["isbn10"] l = [] l.append([title,isbn]) sql = "INSERT INTO allbooks values(%s,%s)" cur.executemany(sql, l) conn.commit() for i in range(0,len(doubans.config.linkconfig.linkLists)): print(doubans.config.linkconfig.linkLists[i]) getDetail(doubans.config.linkconfig.linkLists[i]) time.sleep(int(format(random.randint(0, 9)))) 报错信息如下 Traceback (most recent call last): File "F:/Python/doubans/getDetail.py", line 143, in <module> getDetail(doubans.config.linkconfig.linkLists[i]) File "F:/Python/doubans/getDetail.py", line 27, in getDetail if ('title' in result.keys()): UnboundLocalError: local variable 'result' referenced before assignment Bad Request

Python爬虫爬取的数据不全

![图片说明](https://img-ask.csdn.net/upload/201707/28/1501209027_991059.png) ``` url='https://www.zhihu.com/people/yang-ze-yong-3/following' page=urllib.request.urlopen(url).read() soup=BeautifulSoup(page) items=soup.find_all('div',{'class':'List-item'}) ``` 结果只返回前3条List-item...新手求助

请问写python爬虫如何用urllib或者requests模拟用户登录

比如我在百度文库下个课件,http://youke.baidu.com/view/05984a27b4daa58da0114ac6, 我用urllib2或者request,我可以用urllib2.open(url)或者requests.get(url)来打开页面,但是不知道如何写cookie的,就是让浏览器认为你已经登录了,请问如何写urllib或者requests.get呢?谢谢

在Python中模拟HTTP Post请求

<div class="post-text" itemprop="text"> <p>I am trying to send a HTTP post request in a PHP page. I gave a try on both Java and Python (this was the first time I used Python) and I can say that both of them worked almost fine for me. Actually, they worked fine only for my test PHP page. For some reason, neither of them worked for my target PHP page.</p> <p>In my opinion, two reasons that may cause the problem of unsuccessful post request could be:</p> <ol> <li>a kind of redirection may happen</li> <li>the server wants a proper cookie</li> </ol> <p>Next, you will find the HTML code of the target page and the Python code that should be working.</p> <p>HTML:</p> <pre><code>&lt;form id="m713a0moduleform_2" method="post" action="http://www.X.Y/index.php?page=login" class="cms_form"&gt; &lt;input type="hidden" name="mact" value="FrontEndUsers,m713a0,do_login,1" /&gt; &lt;input type="hidden" name="m713a0returnid" value="794" /&gt; &lt;input type="hidden" name="page" value="794" /&gt; &lt;input type="hidden" name="m713a0form" value="login" /&gt; &lt;input type="text" class="cms_textfield" name="m713a0feu_input_username" id="m713a0feu_input_username" value="" size="10" maxlength="40" /&gt; &lt;input type="password" class="cms_password" name="m713a0feu_input_password" value="" size="10" maxlength="10" /&gt; &lt;input class="cms_submit" name="m713a0feu_btn_login" id="m713a0feu_btn_login" value="Sign in" type="submit" class="signin_button" /&gt; &lt;/form&gt; </code></pre> <p>Python:</p> <pre><code>import urllib params = urllib.urlencode({"mact":"FrontEndUsers,m713a0,do_login,1","m713a0returnid":"18","page":"18","m713a0form":"login","m713a0feu_input_username":"Z","m713a0feu_input_password":"W","m713a0feu_btn_login":"Sign in"}) f = urllib.urlopen("http://www.X.Y/index.php?page=login", params) print f.read() </code></pre> <p>I receive the following error. Any ideas?</p> <pre>Traceback (most recent call last): File "/X/Y/Z/NewPythonProject2/src/newpythonproject2.py", line 34, in from paste.proxy import TransparentProxy ImportError: No module named paste</pre> </div>

python爬取大量网页出现卡顿问题

用python爬取新浪新闻,在爬到260多篇的时候程序就会卡在正则表达式匹配处,单独爬取该网页没有问题,求问原因 import urllib2 import urllib import re import time from bs4 import BeautifulSoup def getContent(url,index): headers = {'User-Agent':'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'} myRequest = urllib2.Request(url,headers = headers) for tries in range(5): try: myResponse = urllib2.urlopen(myRequest) myPage = myResponse.read() time.sleep(1) break except: if tries < 5: print 'Attempt to reconnect' continue else: print 'connection failed' textContent = re.findall('<div class="article article_16" id=".*?">(.*?)<p class="article-editor">',myPage,re.S) while len(textContent) == 0: textContent = re.findall('<div class="article article_16" id=".*?">(.*?)<p class="article-editor">',myPage,re.S) retContent = re.findall('<p.{0,100}?>(.*?)</p>',textContent[0],re.S) while len(retContent) == 0: retContent = re.findall('<p.{0,100}?>(.*?)</p>',textContent[0],re.S) #print len(retContent) file = open(str(index)+'.txt','w') for text in retContent: file.write(text) file.close() print str(index)+'.txt has been written' #def crawler_urllist(start,end,typeNum):#para type : string #create date list partialUrl = ['http://roll.news.sina.com.cn/interface/rollnews_ch_out_interface.php?col=','&spec=&type=&date=','&ch=01&k=&offset_page=0&offset_num=0&num=40&asc=&page='] format = "%Y-%m-%d" beginDate = datetime.datetime.strptime(start,format) endDate = datetime.datetime.strptime(end,format) oneDay = datetime.timedelta(days = 1) diffDay = endDate - beginDate dateList = [] for i in range(diffDay.days + 1): dateList.append(datetime.datetime.strftime(beginDate,format)) beginDate += oneDay retList = [] for i in range(len(dateList)): url = partialUrl[0] + typeNum + partialUrl[1] + dateList[i] + partialUrl[2] page = 0 while True: page += 1 continueFlag,myPage = getContent(url + str(self.page)) print(url + str(self.page)) #print self.page #print self.initUrl + str(self.page) if continueFlag: self.pages.append(myPage) else: self.page = 0 break else: time.sleep(1) date_url_list = open(typeNum + '.txt','w') for item in retList: date_url_list.write(item) date_url_list.write('\n') date_url_list.close() return type + '.txt' i = 1 urlFile = open('DomesticNewsUrlList.txt','r') naviurlList = urlFile.readlines() urlList = list(set(naviurlList))[0].split() #urllist1 = urlList[0:5] for url in urlList: getContent(url,i) #time.sleep(1) i += 1 ``` ``` ``` ```

urllib.request.build_opener()的作用是什么,为什么要用这个函数创建一个实例化对象

opener = urllib.request.build_opener() # 请求头 opener.addheaders = [ # ('Host', 'upos-hz-mirrorks3.acgvideo.com'), #注意修改host,不用也行 ('User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:56.0) Gecko/20100101 Firefox/56.0'), ('Accept', '*/*'), ('Accept-Language', 'en-US,en;q=0.5'), ('Accept-Encoding', 'gzip, deflate, br'), ('Range', 'bytes=0-'), # Range 的值要为 bytes=0- 才能下载完整视频 ('Referer', start_url), # 注意修改referer,必须要加的! ('Origin', 'https://www.baidu.com'), ('Connection', 'keep-alive'), ] urllib.request.install_opener(opener) 以上是部分代码片段

python requests post data=编码问题

我在 **data={ 'text':'%D1%A7%C9%FA', }**有这样一个数据 f=request.post(url,headers=heard2,data=data) 发送出去时抓包发现 text=%25D1%25A7%25C9%25FA被编码了 如何跳过这个编码 text=%D1%A7%C9%FA data={ u'text':u'' }不可以

python无法访问https出现URLError

URLError: "urlopen error [Errno 8] _ssl.c:503: EOF occurred in violation of protocol" 直接通过浏览器可以访问,昨天也是好使的,今天就不行了 代码: url="https://ipaddress/xxx/xxx/xxx" kw={'xxx':'xxx'} data = urllib.urlencode(kw) req = urllib2.Request(url,data) base64string = base64.encodestring('%s:%s' % ('xxx', 'xxx'))[:-1] #这里最后会自动添加一个\n authheader = "Basic %s" % base64string req.add_header("Authorization", authheader) return_data = urllib2.urlopen(req) #请求url data = return_data.read() #读取内容 print data 错误: Traceback (most recent call last): File "C:\Python27\Lib\site-packages\Pythonwin\pywin\framework\scriptutils.py", line 323, in RunScript debugger.run(codeObject, __main__.__dict__, start_stepping=0) File "C:\Python27\Lib\site-packages\Pythonwin\pywin\debugger\__init__.py", line 60, in run _GetCurrentDebugger().run(cmd, globals,locals, start_stepping) File "C:\Python27\Lib\site-packages\Pythonwin\pywin\debugger\debugger.py", line 655, in run exec cmd in globals, locals File "C:\Users\wzs\Desktop\get_resulte.py", line 2, in <module> import urllib File "C:\Python27\lib\urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "C:\Python27\lib\urllib2.py", line 394, in open response = self._open(req, data) File "C:\Python27\lib\urllib2.py", line 412, in _open '_open', req) File "C:\Python27\lib\urllib2.py", line 372, in _call_chain result = func(*args) File "C:\Python27\lib\urllib2.py", line 1207, in https_open return self.do_open(httplib.HTTPSConnection, req) File "C:\Python27\lib\urllib2.py", line 1174, in do_open raise URLError(err) URLError:<urlopen error [Errno 8] _ssl.c:503: EOF occurred in violation of protocol>

requests 抓取网页信息 为什么获取不到信息?

利用花瓣网练习获取网页信息,此步骤想获取每个图片的url 网页代码: ![图片说明](https://img-ask.csdn.net/upload/201810/25/1540453448_856699.png) 我的代码: ![图片说明](https://img-ask.csdn.net/upload/201810/25/1540453583_562332.png) 但是输出为空: ![图片说明](https://img-ask.csdn.net/upload/201810/25/1540453635_473534.png) 请问一下哪里出错了,为什么获取不到信息,我用.wrapper也不行。 顺便问一下,data-id 应该怎么获取? 谢谢

Python读取JSON出现b'和\n'问题

使用Python自带的IDLE,发现读取JSON时会增加一些字符串。代码如下: ``` # -*- coding: utf-8 -*- import json from urllib.request import urlopen def getCountry(ipAddress): response=urlopen('http://freegeoip.net/json/'+ipAddress).read() print(response) responseJSON=json.loads(response) return responseJSON.get('country_code') print(getCountry('50.78.253.58')) ``` 运行环境如下: Python版本:python 3.5-64位 Windows系统:win10 64位(中文) 默认浏览器:Google chrome 46.0.2490.86 系统默认语言:中文 如图所示: ![图片说明](https://img-ask.csdn.net/upload/201703/19/1489921509_884593.jpg) 而翻看源码,似乎也没找到这几个字符串的东西: ![图片说明](https://img-ask.csdn.net/upload/201703/19/1489921759_759105.png) 麻烦大家帮忙看看有没有解决办法。系统是不太可能装英文版的了。第一次学习python,就遇到这种问题。

请求python3.7中 的url中文问题

import string import urllib import json import time from quopri import quote ISOTIMEFORMAT='%Y-%m-%d %X' outputFile = 'douban_movie.txt' fw = open(outputFile, 'w') fw.write('id;title;url;cover;rate\n') headers = {} headers["Accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" headers["Accept-Encoding"] = "gzip, deflate, sdch" headers["Accept-Language"] = "zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4,ja;q=0.2" # headers["Cache-Control"] = "max-age=0" headers["Connection"] = "keep-alive" # headers["Cookie"] = 'bid="LJSWKkSUfZE"; ll="108296"; __utmt=1; regpop=1; _pk_id.100001.4cf6=32aff4d8271b3f15.1442223906.2.1442237186.1442224653.; _pk_ses.100001.4cf6=*; __utmt_douban=1; __utma=223695111.736177897.1442223906.1442223906.1442236473.2; __utmb=223695111.0.10.1442236473; __utmc=223695111; __utmz=223695111.1442223906.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=30149280.674845100.1442223906.1442236473.1442236830.3; __utmb=30149280.4.9.1442237186215; __utmc=30149280; __utmz=30149280.1442236830.3.2.utmcsr=baidu|utmccn=(organic)|utmcmd=organic; ap=1' headers["Host"] = "movie.douban.com" headers["Referer"] = "http://movie.douban.com/" headers["Upgrade-Insecure-Requests"] = 1 headers["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36" # 获取tag request = urllib.request.Request(url="http://movie.douban.com/j/search_tags?type=movie") response = urllib.request.urlopen(request) tags = json.loads(response.read())['tags'] # 开始爬取 print ("********** START **********") print (time.strftime( ISOTIMEFORMAT, time.localtime() )) for tag in tags: print ("Crawl movies with tag: " + tag) print (time.strftime( ISOTIMEFORMAT, time.localtime() )) start = 0 while True: url = "http://movie.douban.com/j/search_subjects?type=movie&tag=" +tag.encode("utf-8")+"&page_limit=20&page_start="+str(start) #url = quote(url, safe=string.printable) request = urllib.request.Request(url=url) response = urllib.request.urlopen(request) movies = json.loads(response.read())['subjects'] if len(movies) == 0: break for item in movies: rate = item['rate'] title = item['title'] url = item['url'] cover = item['cover'] movieId = item['id'] record = str(movieId) + ';' + title + ';' + url + ';' + cover + ';' + str(rate) + '\n' fw.write(record.encode('utf-8')) print (tag + '\t' + title) start = start + 20 fw.close() ![图片说明](https://img-ask.csdn.net/upload/201906/02/1559463756_939891.png) ![图片说明](https://img-ask.csdn.net/upload/201906/02/1559463786_165838.png) ![图片说明](https://img-ask.csdn.net/upload/201906/02/1559463796_447639.png) ![图片说明](https://img-ask.csdn.net/upload/201906/02/1559463972_311111.png)

python 抓取POST 网页失败,求大神帮忙看下代码哪里有问题

![![图片说明](https://img-ask.csdn.net/upload/201504/02/1427958956_940687.png)图片说明](https://img-ask.csdn.net/upload/201504/02/1427958897_886192.png) 这是网站请求的是Request Header 和携带的参数 import urllib import urllib2 def GetFormDate( pageNum , catId ): retValue = {} retValue["module"] = "product" retValue["from"] = "category" retValue["page"] = "%d" % pageNum paramJson = {"mobile" : False , "catalog" : "coo8Store" , "siteId" : "coo8Site" , "shopId" : "" , "regionId" : "31010100" , "pageName" : "list" , "et" : "" , "XSearch" : False , "startDate" : 0 , "endDate" : 0 , "pageSize" : 48 , "state" : 4 , "weight" : 0 , "more" : 0 , "sale" : 0 , "instock" : 1 , "filterReqFacets" : None , "rewriteTag" : False , "userId" : "" , "priceTag" : 0} paramJson["catId"] = catId retValue["paramJson"] = paramJson return retValue body=urllib.urlencode(GetFormDate(2,"cat15965738")) request = urllib2.Request("http://www.gome.com.cn/p/asynSearch" ) request.add_header("Referer", "http://www.gome.com.cn/category/cat15965738.html") request.add_header('Content-Type','application/json') request.add_header('X-Requested-With','XMLHttpRequest') request.add_header('HOST','www.gome.com.cn') request.add_header('Accept-Language','zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3') request.add_header('Accept','application/json, text/javascript, */*; q=0.01') f = urllib2.urlopen(request , json.dumps(GetFormDate(2,"cat15965738"))) content = f.read() print content 这样写的话,获得的Content 一直是空,有哪位大神知道为什么吗,求解啊

Laravel API cURL请求Python

<div class="post-text" itemprop="text"> <p>I followed this Laravel token API tutorial: <a href="http://rjv.im/post/95988160186/api-token-authentication-with-laravel-and-sentry-part" rel="nofollow">http://rjv.im/post/95988160186/api-token-authentication-with-laravel-and-sentry-part</a>. I have written the following cURL request to communicate with my API:</p> <pre><code>curl -H "X-Auth-Token:tokenhere" http://localhost:8000/account </code></pre> <p>The request works properly, and accurately returns the expected data. When I translate this to Python I receive <code>urllib2.HTTPError: HTTP Error 401: Unauthorized</code></p> <pre><code>import urllib2 req = urllib2.Request('http://localhost:8000/account') req.add_header("X-Auth-Token", "tokenhere") resp = urllib2.urlopen(req) content = resp.read() print content </code></pre> <p>If I pass user credentials using basic auth instead of an X-Auth-Token, the request works as expected:</p> <pre><code>import urllib2 def basic_authorization(user, password): s = user + ":" + password return "Basic " + s.encode("base64").rstrip() req = urllib2.Request("http://localhost:8000/account", headers = { "Authorization": basic_authorization("usernameHere", "passwordHere"), }) f = urllib2.urlopen(req) print f.read() </code></pre> <p>Any assistance would be much appreciated. </p> </div>

Python向PHP发送POST数据,PHP收到之后如何保存?

这个是Index.php界面,他的作用只有一个,将session中的内容显示出来。 ``` <?php session_start(); ?> <!DOCTYPE html> <html> <head> <meta charset="UTF-8"> <title></title> </head> <body> <?php var_dump($_SESSION); ?> <a href="ShowSession">ShowSession</a> </body> </html> ``` 这是第二个界面OnPostMessage.php,他的作用是接收Python的POST请求,并把接受到的数据设置到SESSION中 ``` <?php session_start(); $loginright = array(); foreach($_POST as $key=>$val) { global $loginright; $loginright[$key] = $val; } $_SESSION['loginright'] = serialize($loginright); print_r($_SESSION); ``` 第三个界面ShowSession.php 作用也是将SESSION显示出来 ``` <?php session_start(); var_dump($_SESSION); ``` 最后一个文件: SendPost.py这里用Python函数 想第二个界面OnPostMessage.php发送POST数据 ``` import urllib import urllib2 import httplib import json test_data={'ServiceCode':'aaaa','b':'bbbbb'} test_data_urlencode=urllib.urlencode(test_data) requrl="http://localhost/PhpProject2/OnPostMessage.php" req=urllib2.Request(url = requrl,data =test_data_urlencode) res_ddd=urllib2.urlopen(req) res = res_ddd.read() print res ``` 原本希望实现的现象 1. 首先启动 这个PHP程序,界面默认在Index.php这个界面上。 2. 这时候因为没有设置任何SESSION,所以显示出来的SESSION内容也是空的 3. 在这个时候 使用 SendPost.py 想第二个界面OnPostMessage发送POST请求. 第二个界面收到发送的数据之后 将他设置到SESSION中 4. 最后将PHP界面跳转到ShowSession界面上,将SESSION打印出来。 现在遇到的问题: 1. 为什么最后界面ShowSession什么都显示不出来呢? 在上面第三步的时候已经把内送设置到SESSION中了,而且可以在Python函数中捕获到SESSION中设置的值。 这个现象是不是从根本上就错了? 如果错了。想要实现这个需求应该怎么错呢?如果不把数据写文件或者写数据库的话。

python 爬虫 出现IOError错误

# -*- coding:utf-8 -*- #! /usr/bin/env python import urllib import urllib2 import re page = 1 url = 'http://www.qiushibaike.com/hot/page/' + str(page) user_agent = 'Mozilla/4.0(compatible;MSIE 5.5;Windows NT)' headers = {'User-Agent':user_agent} try: request = urllib2.Request(url,headers = headers) response = urllib2.urlopen(request) content = response.read().decode('utf-8','ignore') pattern = re.compile(r'<div.*?author.*?>*?<img.*?>.*?<a.*?title="(.*?)">.*?<div.*?>.*?(.*?)</div>', re.S) items = re.findall(pattern,content) for item in items: print item[0],item[1] except urllib2.URLError, e: if hasattr(e,"code"): print e.code if hasattr(e,"reason"): print e.reason ```![图片说明](https://img-ask.csdn.net/upload/201604/29/1461898513_494366.png) ```这是出现错误的信息,在powershell中设置chcp 65001就会出现IOError错误,设置成chcp 437或者936就会出现下面的乱码,请问是什么原因? ``` ```

python爬取页面数据错误,连续爬很多页数。我主要改变的是post里面的参数pageno

爬数据的时候,比如有800页数据要爬,爬到400多页会出现返回400错误,没有数据。但是如果将程序从400页到800再重新爬,又不会报错。 求解决方法

在PHP中处理嵌套的JSON POST请求

<div class="post-text" itemprop="text"> <p>I am trying to have a python client send a post request which contains nested JSON like such</p> <pre><code>{"nested":{"field1":"response1", "field2":"response2"}} </code></pre> <p>My python code is here</p> <pre class="lang-py prettyprint-override"><code>from urllib.parse import urlencode from urllib.request import Request, urlopen url="http://localhost/api/vscore.php" post_fields={"nested":{"field1":"response1", "field2":"response2"}} request = Request(url, urlencode(post_fields).encode()) json = urlopen(request).read().decode() print(json) </code></pre> <p>PHP code:</p> <pre><code>print_r($_POST["nested"]); </code></pre> <p>returns</p> <pre><code>{'field2': 'response2', 'field1': 'response1'} </code></pre> <p>but when I try to access "field1" with $_POST["nested"]["field1"], it returns this:</p> <pre><code>{ </code></pre> <p>instead of returning "response1". How can I get my code to return fields in nested JSON?</p> </div>

2019 Python开发者日-培训

2019 Python开发者日-培训

150讲轻松搞定Python网络爬虫

150讲轻松搞定Python网络爬虫

设计模式(JAVA语言实现)--20种设计模式附带源码

设计模式(JAVA语言实现)--20种设计模式附带源码

YOLOv3目标检测实战:训练自己的数据集

YOLOv3目标检测实战:训练自己的数据集

java后台+微信小程序 实现完整的点餐系统

java后台+微信小程序 实现完整的点餐系统

三个项目玩转深度学习(附1G源码)

三个项目玩转深度学习(附1G源码)

初级玩转Linux+Ubuntu(嵌入式开发基础课程)

初级玩转Linux+Ubuntu(嵌入式开发基础课程)

2019 AI开发者大会

2019 AI开发者大会

玩转Linux:常用命令实例指南

玩转Linux:常用命令实例指南

一学即懂的计算机视觉(第一季)

一学即懂的计算机视觉(第一季)

4小时玩转微信小程序——基础入门与微信支付实战

4小时玩转微信小程序——基础入门与微信支付实战

Git 实用技巧

Git 实用技巧

Python数据清洗实战入门

Python数据清洗实战入门

使用TensorFlow+keras快速构建图像分类模型

使用TensorFlow+keras快速构建图像分类模型

实用主义学Python(小白也容易上手的Python实用案例)

实用主义学Python(小白也容易上手的Python实用案例)

程序员的算法通关课:知己知彼(第一季)

程序员的算法通关课:知己知彼(第一季)

MySQL数据库从入门到实战应用

MySQL数据库从入门到实战应用

机器学习初学者必会的案例精讲

机器学习初学者必会的案例精讲

手把手实现Java图书管理系统(附源码)

手把手实现Java图书管理系统(附源码)

极简JAVA学习营第四期(报名以后加助教微信:eduxy-1)

极简JAVA学习营第四期(报名以后加助教微信:eduxy-1)

.net core快速开发框架

.net core快速开发框架

玩转Python-Python3基础入门

玩转Python-Python3基础入门

Python数据挖掘简易入门

Python数据挖掘简易入门

微信公众平台开发入门

微信公众平台开发入门

程序员的兼职技能课

程序员的兼职技能课

Windows版YOLOv4目标检测实战:训练自己的数据集

Windows版YOLOv4目标检测实战:训练自己的数据集

HoloLens2开发入门教程

HoloLens2开发入门教程

微信小程序开发实战

微信小程序开发实战

Java8零基础入门视频教程

Java8零基础入门视频教程

相关热词 c# cad插入影像 c#设计思想 c#正则表达式 转换 c#form复制 c#写web c# 柱形图 c# wcf 服务库 c#应用程序管理器 c#数组如何赋值给数组 c#序列化应用目的博客园
立即提问