TL:DR - Look at edit 2 for a C# equivalent of code http client code, resulting in ~same issue, so Go http.Client is not the real problem, but C# Web API once deployed to Azure...
I'm getting very bad performance from a C# Web API once deployed to Azure Web App [2x Standard S3]. At first I was asking re: Go's http.Client timeout, but writing a similar client in C# and NodeJs gave same results.
This is my http.Client:
func getWebClient() *http.Client {
var netTransport = &http.Transport{
Dial: (&net.Dialer{
Timeout: 5 * time.Second,
}).Dial,
TLSHandshakeTimeout: 10 * time.Second,
MaxIdleConnsPerHost: 2000,
ResponseHeaderTimeout: 10 * time.Second,
}
var netClient = &http.Client{
Timeout: time.Second * 10,
Transport: netTransport,
}
return netClient
}
Error I'm getting:
net/http: request canceled (Client.Timeout exceeded while awaiting headers)
It can be on GET, POST, PUT. I'm getting those error, yet running a failing GET with curl immediately got a reply.
This is a sample Get function I'm using to call an API:
func get(path string, result interface{}) error {
req, err := http.NewRequest("GET", webDALAPIURL+path, nil)
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
wc := getWebClient()
res, err := wc.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode >= 400 {
return fmt.Errorf("[GET] %d - %s", res.StatusCode, webDALAPIURL+path)
}
decoder := json.NewDecoder(res.Body)
return decoder.Decode(result)
}
Fun fact, never encountered those error when the API was running locally. API is a C# ASP.NET Web API app.
I started to have lots of TLS Handshake error, so I dropped the https for the Azure app endpoint. Now I'm getting this error.
I'm tailing the logs for the app, and nothing is occurring [the API being called]. Seems like Go is unable to make multiple call to same host. I'm not event using goroutine in one cmd and using them in another, both results in same errors.
When the API is running on a Windows computer in same network, never had that error during development.
Edit 1:
Note that 80-85% of the requests are working well, the scenario is that (pseudo code):
for item in items {
http get /item/{id} => works 90% of the time, 10% timeout
change item properties
http PUT /item/{id} => works 80% of the time
}
I added a retry in the get() function, so if the timeout occurs it's retrying the get, and this seems to works. Although, I don't like this workaround at all.
Also note that we are talking about quick GET that are returning timeout, when I run them from curl, it's < 1sec. Same for the PUT, those are highly simplistic database SELECT * FROM TABLE WHERE ID = {id} AND UPDATE.
The Azure Web app that runs the API is 2 instance of Standard S3.
The fact that the retry for the GET worked looks like, it's most certainly the API / Azure app not taking the load, which is impossible for the simplicity and we are talking about less than 10 requests / seconds.
Another point not negligible, when running on dev server, the same Azure SQL Database was used, so the SELECT/UPDATE perf should be the exact same on dev and on Azure Web App.
Edit 2:
The speed comparing same C# Web API from local to Azure is disturbing. I wrote a similar C# http client to test Azure vs Local Web API.
class Program
{
static int fails = 0;
static void Main(string[] args)
{
for (int i = 0; i < 2000; i++)
{
get(i);
}
Console.WriteLine("completed: " + fails.ToString());
Console.ReadLine();
}
static void get(int id)
{
id += 22700;
var c = new HttpClient();
var resp = c.GetAsync("http://myapphere.azurewebsites.net/api/users/" + id).Result;
if (!resp.IsSuccessStatusCode)
{
Console.WriteLine("");
fails++;
Console.WriteLine(string.Format("error getting /users status code {0}", resp.StatusCode));
}
else
{
Console.Write(".");
}
}
}
Running this console app against Azure I can clearly see where Go's getting time out, it's painfully slow, no error are returned, but the Console.Write("."); takes forever to print, are are periodic, can print ~fast for 3-4 than it stopped.
Changing this to localhost:1078 again same database used there's no paused, and the Console.Write(".") are printing like 20x the speed compare to Azure.
How is that possible?
Edit 3:
Just added global error handler on the web api, maybe the slovenliness could be caused by too much Exception being thrown. Added a Trace.TraceError and I azure site log tail it, again nothing was displayed.
I would go as far as saying that the local web api is running 25-30x faster than the 2x instance of Azure Standard S3. Clearly that can't be true, but the web api is so simple that I don't see what I can do to have Azure run it at full speed.
