dongzhuange2625
2017-10-17 17:35
浏览 145
已采纳

如何使用适用于GOlang的AWS开发工具包清空S3存储桶?

Goal: To empty an existing S3 bucket using the AWS SDK for GOlang.

图片转代码服务由CSDN问答提供 功能建议

目标::使用适用于GOlang的AWS开发工具包清空现有的S3存储桶。

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

4条回答 默认 最新

  • duanfei8399 2017-10-17 18:00
    已采纳

    NOTE: These are code snippets that might require YOU to make changes on YOUR SIDE to make it run.

    You will need to implement the below method:

      //EmptyBucket empties the Amazon S3 bucket
        func (s awsS3) EmptyBucket(bucket string) error {
            log.Info("removing objects from S3 bucket : ", bucket)
            params := &s3.ListObjectsInput{
                Bucket: aws.String(bucket),
            }
            for {
                //Requesting for batch of objects from s3 bucket
                objects, err := s.Client.ListObjects(params)
                if err != nil {
                    return err
                }
                //Checks if the bucket is already empty
                if len((*objects).Contents) == 0 {
                    log.Info("Bucket is already empty")
                    return nil
                 }
                log.Info("First object in batch | ", *(objects.Contents[0].Key))
    
                //creating an array of pointers of ObjectIdentifier
                objectsToDelete := make([]*s3.ObjectIdentifier, 0, 1000)
                for _, object := range (*objects).Contents {
                    obj := s3.ObjectIdentifier{
                        Key: object.Key,
                    }
                    objectsToDelete = append(objectsToDelete, &obj)
                }
                //Creating JSON payload for bulk delete
                deleteArray := s3.Delete{Objects: objectsToDelete}
                deleteParams := &s3.DeleteObjectsInput{
                    Bucket: aws.String(bucket),
                    Delete: &deleteArray,
                }
                //Running the Bulk delete job (limit 1000)
                _, err = s.Client.DeleteObjects(deleteParams)
                if err != nil {
                    return err
                }
                if *(*objects).IsTruncated { //if there are more objects in the bucket, IsTruncated = true
                    params.Marker = (*deleteParams).Delete.Objects[len((*deleteParams).Delete.Objects)-1].Key
                    log.Info("Requesting next batch | ", *(params.Marker))
                } else { //if all objects in the bucket have been cleaned up.
                    break
                }
            }
            log.Info("Emptied S3 bucket : ", bucket)
            return nil
        } 
    

    UPDATE : The latest version of AWS SDK for GO has resolved the prior issue I had.

    点赞 评论
  • dslk6326846 2017-10-27 02:44

    The AWS SDK for Go has a Amazon S3 batching abstraction. Take a look here.

    点赞 评论
  • douyu3145 2017-12-01 17:31

    Don't forget that by default ListObjects only returns up to 1000 bucket items. If you might have more than 1000, check the IsTruncated property on the return value. If true, use the NextMarker property from the return value to get the next 1000 items.

    See my example in the Go dev guide: http://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-basic-bucket-operations.html#s3-examples-bucket-ops-delete-all-bucket-items

    点赞 评论
  • drnzpd579935 2019-04-15 03:12

    AWS SDK now has BatchDeleteIterator that can do the job. Example provided via Amazon docs.

    package main
    
    import (
        "github.com/aws/aws-sdk-go/aws"
        "github.com/aws/aws-sdk-go/aws/session"
        "github.com/aws/aws-sdk-go/service/s3"
        "github.com/aws/aws-sdk-go/service/s3/s3manager"
    
        "fmt"
        "os"
    )
    
    //    go run s3_delete_objects BUCKET
    
    func main() {
        if len(os.Args) != 2 {
            exitErrorf("Bucket name required
    Usage: %s BUCKET", os.Args[0])
        }
    
        bucket := os.Args[1]
    
        // Initialize a session in us-west-2 that the SDK will use to load
        // credentials from the shared credentials file ~/.aws/credentials.
        sess, _ := session.NewSession(&aws.Config{
            Region: aws.String("us-west-2")},
        )
    
        // Create S3 service client
        svc := s3.New(sess)
    
        // Setup BatchDeleteIterator to iterate through a list of objects.
        iter := s3manager.NewDeleteListIterator(svc, &s3.ListObjectsInput{
            Bucket: aws.String(bucket),
        })
    
        // Traverse iterator deleting each object
        if err := s3manager.NewBatchDeleteWithClient(svc).Delete(aws.BackgroundContext(), iter); err != nil {
            exitErrorf("Unable to delete objects from bucket %q, %v", bucket, err)
        }
    
        fmt.Printf("Deleted object(s) from bucket: %s", bucket)
    }
    
    func exitErrorf(msg string, args ...interface{}) {
        fmt.Fprintf(os.Stderr, msg+"
    ", args...)
        os.Exit(1)
    }
    
    点赞 评论

相关推荐 更多相似问题