Or is there some workaround that I don't know about so that I can continue terminating SSL at the load balancer and still use HTTP/2?
If I've got you right, you need a HTTP/2 aware load balancer which can proxy your requests to your application. You may use nginx. It can excellent do http2 termination, load balancing and many other things. So probably you may not need haproxy at all. However haproxy is useful if you want some advanced features like sticky sessions. Nginx also can do them, but only in a commercial version.
Here's an article (in Google cache) how to setup nginx + haproxy.