从ajax请求并发写入文件

Question

I want to write requests to one file from some ajax script. The problem arises when there will be many of those in a second and writing to file will take more time than the break between requests, and when there will be two requests at the same time.

How could I solve this?

I've came up using mutex, like:

var mu sync.Mutex
func writeToFile() {
    mu.Lock()
    defer mu.Unlock()
    // write to file
}

But it makes the whole thing synchronous and I don't really know what happens when there are two requests at the same time. And it still does not lock the file itself.

Uh, what's the proper way to do this?

Answer 1

You only need to make writing to the file "sequential", meaning don't allow 2 concurrent goroutines to write to the file. Yes, if you use locking in the writeToFile() function, serving your ajax requests may become (partially) sequential too.

What I suggest is open the file once, when your application starts. And designate a single goroutine which will be responsible writing to the file, no other goroutines should do it.

And use a buffered channel to send data that should be written to the file. This will make serving ajax requests non-blocking, and still the file will not be written concurrently / parallel.

Note that this way ajax requests won't even have to wait while the data is actually written to the file (faster response time). This may or may not be a problem. For example if later writing fails, your ajax response might already be committed => no chance to signal failure to the client.

Example how to do it:

var (
    f      *os.File
    datach = make(chan []byte, 100) // Buffered channel
)

func init() {
    // Open file for appending (create if doesn't exist)
    var err error
    f, err = os.OpenFile("data.txt", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666)
    if err != nil {
        panic(err)
    }

    // Start goroutine which writes to the file
    go writeToFile()
}

func writeToFile() {
    // Loop through any data that needs to be written:
    for data := range datach {
        if _, err := f.Write(data); err != nil {
            // handle error!
        }
    }
    // We get here if datach is closed: shutdown
    f.Close()
}

func ajaxHandler(w http.ResponseWriter, r *http.Request) {
    // Assmeble data that needs to be written (appended) to the file
    data := []byte{1, 2, 3}
    // And send it:
    datach <- data
}

To gracefully exit from the app, you should close the datach channel: when it's closed, the loop in the writeToFile() will terminate, and the file will be closed (flushing any cached data, releasing OS resources).

If you want to write text to the file, you may declare the data channel like this:

var datach = make(chan string, 100) // Buffered channel

And you may use File.WriteString() to write it to the file:

if _, err := f.WriteString(data); err != nil {
    // handle error!
}