Scanner.Scan（）阻止，直到在Linux上退出，但在Windows上退出

Question

I'm working on a devtool and one of the features of this tool involves spawning a child process and reading the stdout stream of that process. I need to read each line of output into memory in order to process it in some way (one of the future features of the tool will involve processing logs and sending them to external locations such as log managers and dashboards etc) so that's why I don't simply do cmd.Stdout = os.Stdout)

It works fine, and has done for a while, but only on Windows apparently. I got a rather confusing bug report recently where a user reported the output isn't "real time" and upon testing on Linux, I found that it's true and the output is only dumped to the console when the process exits.

Here is the code that scans reader, works as expected on Windows but not on Linux or in a Linux container on Windows/MacOS (tested both)

https://github.com/Southclaws/sampctl/blob/f639c941dd8f9ca7c7c819a152909044ad63be08/runtime/run.go#L133-L137

And if you poke around the code you'll find where the reader is created with io.Pipe() and bound to the cmd's Stdout/Stderr outputs.

That line 134 is where the program just blocks until the cmd in the goroutine below stops running, on line 161.

I'm assuming this is related to buffers and flushing but I don't know enough about Go's internals to really pinpoint the problem. What exactly is different about scanner.Scan() on Windows and Linux? Why does it block on one platform but not on another? Is it related to threads/goroutines being scheduled differently? (both test machines have multiple cores, even the Docker container had 4 vCPUs)

Here's the issue for reference: https://github.com/Southclaws/sampctl/issues/100

I'm really stumped on this one, would love some help figuring it out!

Edit:

So I messed around some more, still no solution. I attempted to use a Python script and got the same result, stdout works fine when directed to a tty but when it's read by a process it just hangs:

from subprocess import Popen, PIPE
from time import sleep

p = Popen(
    ['/root/.samp/runtime/0.3.7/samp03svr'],
    stdin=PIPE,
    stdout=PIPE,
    stderr=PIPE,
    shell=False,
    cwd="/root/.samp/runtime/0.3.7/")

while True:
    print "attempting to read a line"
    output = p.stdout.read()
    print "read a line"
    if not output:
        print '[No more data]'
        break
    print output

attempting to read a line is where it hangs.

Answer 1

By default, Linux buffers output when not in interactive mode (i.e. not in a terminal), so the output is only flushed when the buffer is full (e.g. every 4096 bytes, but this is implmentation defined); when the program explicitly calls flush (which is apparently not happening here); or when the process ends (as you are seeing).

You can change this default behaviour by adjusting the buffer size. For example, by launching a program via stdbuf:

stdbuf -oO /root/.samp/runtime/0.3.7/samp03svr

-o for stdout (also -e and -i), O for "Off" (also L for "line buffered" or a size for an explicit buffer size).

Alternatively there is an unbuffer command or the script command:

https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe/61833#61833