问题背景:
因为要将爬取的内容写入到数据库,所以在navicat内创建了新的数据表,
并设置好字段。在pyspider的调试页面的时候数据可以正常写入到mysql内。
但是已在pyspider的webui页面点击running后就会报错。
调试页面的代码:
try:
cursor = self.db.cursor()
#data=
sql = "insert into zpxx(jobtitle,Jobnature, workplace,Experiences,major,lan,wages,ProfessionalTitle,Number,dates,renzhiyaoqiu,companyjianjie,companysize,companyxingzhi,companytel,companyurl,companyemail) values ('%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s')"%(title,Nature,Place,exp,major,Lan,wages,Zc,num,date,Req,company_profile,company_size,company_nature,tel,url,email)
cursor.execute(sql)
cursor.close()
self.db.commit()
self.db.close()
except Exception as e:
print (e)
self.db.rollback()
只放了写入数据库部分的代码。
在调试页面的时候一切正常,而且数据库也有信息。
但是一运行就会报错:
(2006, '')
E 190417 08:57:22 base_handler:203
Traceback (most recent call last):
File "c:\py36_virtualenv\lib\site-packages\pyspider\libs\base_handler.py", line 196, in run_task
result = self._run_task(task, response)
File "c:\py36_virtualenv\lib\site-packages\pyspider\libs\base_handler.py", line 176, in _run_task
return self._run_func(function, response, task)
File "c:\py36_virtualenv\lib\site-packages\pyspider\libs\base_handler.py", line 155, in _run_func
ret = function(*arguments[:len(args) - 1])
File "", line 63, in detail_page
File "c:\py36_virtualenv\lib\site-packages\MySQLdb\cursors.py", line 183, in execute
while self.nextset():
File "c:\py36_virtualenv\lib\site-packages\MySQLdb\cursors.py", line 136, in nextset
nr = db.next_result()
MySQLdb._exceptions.OperationalError: (2006, '')
上网查了很多,要么是说最大写入数不对等等,可是试了都没有用
求指教