GoColly为每个请求设置代理

Question

I started to deal with Colly. I am trying to set a proxy for every request. But all requests do not go through a proxy

It should work https://github.com/gocolly/colly/blob/master/request.go#L52

I have added this to my code. But requests go without a proxy as before

c.OnRequest(func(r *colly.Request) { 
  r.ProxyURL = "..." 
}

Maybe I didn’t understand correctly how this should work

Answer 1

Instead of setting ProxyURL in the callback function, have you tried setting it on the collector ?

// SetProxy sets a proxy for the collector. This method overrides the previously
// used http.Transport if the type of the transport is not http.RoundTripper.
// The proxy type is determined by the URL scheme. "http"
// and "socks5" are supported. If the scheme is empty,
// "http" is assumed.
func (c *Collector) SetProxy(proxyURL string) error {
    proxyParsed, err := url.Parse(proxyURL)
    if err != nil {
        return err
    }

    c.SetProxyFunc(http.ProxyURL(proxyParsed))

    return nil
}