var srcFile byte _, err = reader.Read(srcFile)
Read puts the read bytes into its argument. Since srcFile is a nil slice, this instructs the reader to read zero bytes. Use ioutil.ReadAll to read all bytes.
Next up is your use of Write.
Write(b) writes up to len(b) bytes, but not necessarily all of it. You must check the return values and call Write repeatedly if necessary.
However, in your case you just want to connect an io.Reader (*Response implements io.Reader) and io.Writer (*os.File). That's what io.Copy is for:
reader, err := client.Retr(path + file) dstFile, err := ioutil.TempFile("", fileID) _, err := io.Copy(dstFile, reader) err := dstFile.Close()