罗家马德里球迷 2019-04-09 22:51 采纳率: 0%
浏览 1269
已结题

Scrapy FormRequest函数中的meta参数值应该如何设置?

我用scrapy进行爬虫,解析函数部分另有下一级回调函数,代码如下:

item = SoccerDataItem()
for i in range(1, 8):
        item['player' + str(i + 1)] = players[i]
for j in range(1, 8):  
    home_sub_list = response.xpath('//div[@class="left"]//li[@class="pl10"]')  
        if home_sub_list[j - 1].xpath('./span/img[contains(@src,"subs_up")]'):  
            item['player' + str(j)]['name'] = home_sub_list[j - 1].xpath('./div[@class="ml10"]').xpath('string(.)').re_first('\d{1,2}\xa0\xa0(.*)')
            item['player' + str(j)]['team_stand'] = 1
            item['player' + str(j)]['is_startup'] = 0
            item['player' + str(j)]['is_subs_up'] = 1
            item['player' + str(j)]['subs_up_time'] = home_sub_list[j].xpath('./span/img[contains(@src,"subs_up")]/following-sibling::span').xpath('string(.)').extract_first(default='')
            yield scrapy.FormRequest(url=data_site, formdata=formdata, meta={'player': item['player' + str(j)]}, callback=self.parse_data)

        else:
            item['player' + str(j)]['name'] = home_sub_list[j-1].xpath('./div[@class="ml10"]').xpath('string(.)').re_first('\d{1,2}\xa0\xa0(.*)')
      item['player' + str(j)]['team_stand'] = 1
      item['player' + str(j)]['is_startup'] = 0
            item['player' + str(j)]['is_subs_up'] = 0

然而运行后一直在报错:

 callback=self.parse_data)
  File "c:\users\pc1\appdata\local\programs\python\python36-32\lib\site-packages\scrapy\http\request\form.py", line 31, in __init__
    querystr = _urlencode(items, self.encoding)
  File "c:\users\pc1\appdata\local\programs\python\python36-32\lib\site-packages\scrapy\http\request\form.py", line 66, in _urlencode
    for k, vs in seq
  File "c:\users\pc1\appdata\local\programs\python\python36-32\lib\site-packages\scrapy\http\request\form.py", line 67, in <listcomp>
    for v in (vs if is_listlike(vs) else [vs])]
  File "c:\users\pc1\appdata\local\programs\python\python36-32\lib\site-packages\scrapy\utils\python.py", line 119, in to_bytes
    'object, got %s' % type(text).__name__)
TypeError: to_bytes must receive a unicode, str or bytes object, got int

据本人百度得知,meta当中的键值对的值应为字符串,字节等类型,这正是当我传入字典类型时报错的原因。
可是,请问我应该如何修改此处呢?

PS:本人所用编程语言为Python,排版可能会引起读者不适,望谅解!

  • 写回答

2条回答 默认 最新

  • devmiao 2019-04-09 23:47
    关注
    评论

报告相同问题?

悬赏问题

  • ¥100 有人会搭建GPT-J-6B框架吗?有偿
  • ¥15 求差集那个函数有问题,有无佬可以解决
  • ¥15 【提问】基于Invest的水源涵养
  • ¥20 微信网友居然可以通过vx号找到我绑的手机号
  • ¥15 寻一个支付宝扫码远程授权登录的软件助手app
  • ¥15 解riccati方程组
  • ¥15 display:none;样式在嵌套结构中的已设置了display样式的元素上不起作用?
  • ¥15 使用rabbitMQ 消息队列作为url源进行多线程爬取时,总有几个url没有处理的问题。
  • ¥15 Ubuntu在安装序列比对软件STAR时出现报错如何解决
  • ¥50 树莓派安卓APK系统签名