I wrote a small utility to pipe the output of the mysqldump command to an s3 bucket in golang, which I intend to use to snapshot and restore our database in an end-to-end test environment. When I run this locally, everything works as expected: the utility correctly pipes the mysqldump output to the s3 bucket. However, when running in the Alpine Linux Docker image, I get a broken pipe error. The utility then sends a empty file to the s3 bucket.
To execute mysqldump, I simply use the exec package. Code for mysqldump:
func (conf *MySQLConfig) DumpDatabases(stdout io.Writer) error {
args := []string{"-u", conf.User, fmt.Sprintf("--password=%s", conf.Password), "-h", conf.Host, "--databases"}
args = append(args, conf.Databases...)
cmd := exec.Command("mysqldump", args...)
cmd.Stdout = stdout
cmd.Stderr = os.Stderr
err := cmd.Start()
if err != nil {
return err
}
cmd.Wait()
return nil
}
The upload to the S3 bucket utilises the go-cdk. It expects a function that expects an io.Writer as input and returns an error. In this case the following snippet is executed to pipe the output of mysqldump to s3:
bucket.Upload(func(w io.Writer) error { return mysqlConf.DumpDatabases(w) })
Code for the upload:
func (b *Bucket) Upload(writeFunc func(io.Writer) error) error {
// Open a connection to the bucket.
ctx := context.Background()
bucket, err := blob.OpenBucket(ctx, b.Url)
if err != nil {
return err
}
defer bucket.Close()
// Add encryption header
beforeWrite := func(asFunc func(interface{}) bool) error {
var input *s3manager.UploadInput
if asFunc(&input) {
input.ServerSideEncryption = aws.String("AES256")
}
return nil
}
opts := &blob.WriterOptions{}
opts.BeforeWrite = beforeWrite
w, err := bucket.NewWriter(ctx, generateKey(), opts)
if err != nil {
return err
}
err = writeFunc(w)
if err != nil {
return err
}
if err := w.Close(); err != nil {
return err
}
return nil
}
Any clue why I get a broken pipe error when running this code on Alpine Linux, while it works as expected locally? Any suggestions on how to debug this issue to figure out what exactly is happening? Should I utilise buffered IO?
