AppEngine IO复制问题导致“ API错误10（文件：FILE_NOT_OPENED）”

Question

My GoLang AppEngine code is uncompressing a ZIP file and then saving off each file as BlobStore content. I've seen it take over 30 seconds to save the data and then it fails with "API error 10 (file: FILE_NOT_OPENED)." The size of the uncompressed file being written is about 1.5 megabytes. Here is the code that does the copy to a new BlogStore item from the Zip reader:

func storeBlob(c appengine.Context, rc io.Reader, mimeType string) (appengine.BlobKey, error) {

c.Infof("Creating new blob of type: [%v]", mimeType)
var key appengine.BlobKey
w, err := blobstore.Create(c, mimeType)
if err != nil {
    return "", err
}

c.Infof("Copying blob data")
_, err = io.Copy(w, rc)
if err != nil {
    return "", err
}

c.Infof("Closing Blob")
err = w.Close()
if err != nil {
    return "", err
}

c.Infof("Getting Blob Key")
key, err = w.Key()
if err != nil {
    return "", err
}
return key, nil

}

The production appengine server locks up for 30 seconds and then reports the error on the io.Copy call. Any thoughts on how to troubleshoot this?

Answer 1

This error is because your request is timing out. Create makes a protocol buffer request that looks like this:

oreq := &filepb.OpenRequest{
    Filename:      res.Filename,
    ContentType:   filepb.FileContentType_RAW.Enum(),
    OpenMode:      filepb.OpenRequest_APPEND.Enum(),
    ExclusiveLock: proto.Bool(true),
}

The corresponding definition:

type OpenRequest struct {
    Filename             *string                      `protobuf:"bytes,1,req,name=filename" json:"filename,omitempty"`
    ContentType          *FileContentType_ContentType `protobuf:"varint,2,req,name=content_type,enum=files.FileContentType_ContentType" json:"content_type,omitempty"`
    OpenMode             *OpenRequest_OpenMode        `protobuf:"varint,3,req,name=open_mode,enum=files.OpenRequest_OpenMode" json:"open_mode,omitempty"`
    ExclusiveLock        *bool                        `protobuf:"varint,4,opt,name=exclusive_lock,def=0" json:"exclusive_lock,omitempty"`
    BufferedOutput       *bool                        `protobuf:"varint,5,opt,name=buffered_output,def=0" json:"buffered_output,omitempty"`
    OpenLeaseTimeSeconds *int32                       `protobuf:"varint,6,opt,name=open_lease_time_seconds,def=30" json:"open_lease_time_seconds,omitempty"`
    XXX_unrecognized     []byte                       `json:"-"`
}

Notice OpenLeaseTimeSeconds is unset. It will default to 30 seconds:

const Default_OpenRequest_OpenLeaseTimeSeconds int32 = 30

So it's probably closing the connection underneath you.

You have two options:

1. Write your own Create method which sets the lease time to something greater than 30 seconds. The most you can do is 60 seconds since then your request will be killed, in which case you will need to use a Task Queue.

2. The API is deprecated:

   Warning: The Files API feature used here to write files to Blobstore has been deprecated and is going to be removed at some time in the future, in favor of writing files to Google Cloud Storage and using Blobstore to serve them. For more information, please visit Google Cloud Storage.

   So maybe you should try google storage instead.