HTTP/1.1 uses KeepAlive by default. Since the connections aren't closed, HAProxy can't balance the requests between different backends.
You have a couple options:
Force the connection to close after each request in your code. Setting
Request.Close = true on either the client or the server will send a
Connection: close header, telling both sides to close the tcp connection.
Alternatively you could have HAPoxy alter the requests by setting
http-server-close so the backend is closed after each request, or
http-closeto shutdown both sides after each request.
http-server-close is usually the best option, since that still maintains persistent connections for the client, while proxying each request individually.