I have build a proxy server that can balance between multiple nodes. I also made it that it can reload with zero downtime. Problem is that most of the nodes have keep alive connections and i have no clue how to handle these. Sometimes the server cant shutdown off 1 or 2 open connections that wont close.
My first opinion is to set a timeout on the shutdown but that does not secures me that every connection is terminated correctly. I think of a download that takes some minutes to complete.
Anyone can give me some good advise what to do in this case?