dragonlew9876 2019-09-02 09:26
浏览 1560
已采纳

GoColly为每个请求设置代理

I started to deal with Colly. I am trying to set a proxy for every request. But all requests do not go through a proxy

It should work https://github.com/gocolly/colly/blob/master/request.go#L52

I have added this to my code. But requests go without a proxy as before

c.OnRequest(func(r *colly.Request) { 
  r.ProxyURL = "..." 
}

Maybe I didn’t understand correctly how this should work

  • 写回答

1条回答 默认 最新

  • douxiong4892 2019-09-02 09:38
    关注

    Instead of setting ProxyURL in the callback function, have you tried setting it on the collector ?

    // SetProxy sets a proxy for the collector. This method overrides the previously
    // used http.Transport if the type of the transport is not http.RoundTripper.
    // The proxy type is determined by the URL scheme. "http"
    // and "socks5" are supported. If the scheme is empty,
    // "http" is assumed.
    func (c *Collector) SetProxy(proxyURL string) error {
        proxyParsed, err := url.Parse(proxyURL)
        if err != nil {
            return err
        }
    
        c.SetProxyFunc(http.ProxyURL(proxyParsed))
    
        return nil
    }
    
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥50 MATLAB实现圆柱体容器内球形颗粒堆积
  • ¥15 python如何将动态的多个子列表,拼接后进行集合的交集
  • ¥20 vitis-ai量化基于pytorch框架下的yolov5模型
  • ¥15 如何实现H5在QQ平台上的二次分享卡片效果?
  • ¥15 python爬取bilibili校园招聘网站
  • ¥30 求解达问题(有红包)
  • ¥15 请解包一个pak文件
  • ¥15 不同系统编译兼容问题
  • ¥100 三相直流充电模块对数字电源芯片在物理上它必须具备哪些功能和性能?
  • ¥30 数字电源对DSP芯片的具体要求