weixin_40005795
2021-01-12 12:26 阅读 5

Amazon S3 Replication to multiple destinations

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Amazon S3 Replication now gives you the ability to replicate data from one source bucket to multiple destination buckets in the same, or different AWS Regions.

New or Affected Resource(s)

Potential Terraform Configuration

hcl
resource "aws_s3_bucket" "source" {
  bucket = "example"

  versioning {
    enabled = true
  }

  replication_configuration {
    role = aws_iam_role.replication.arn

    rules {
      id       = "rule1"
      priority = 1
      status   = "Enabled"

      filter {}

      destination {
        bucket        = aws_s3_bucket.destination1.arn
        storage_class = "STANDARD"
      }
    }

    rules {
      id       = "rule2"
      priority = 2
      status   = "Enabled"

      filter {
        prefix = "prefix2"
      }

      destination {
        bucket        = aws_s3_bucket.destination2.arn
        storage_class = "STANDARD_IA"
      }
    }
  }
}

References

Announcement. Blog post. Developer Guide.

该提问来源于开源项目:hashicorp/terraform-provider-aws

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

6条回答 默认 最新

  • weixin_40005795 weixin_40005795 2021-01-12 12:26

    It looks like this should be supported in the current code (there is no limitation on rules that would prevent this scenario). We should add an explicit acceptance tests case for this.

    点赞 评论 复制链接分享
  • weixin_39853210 weixin_39853210 2021-01-12 12:26

    It looks like this should be supported in the current code (there is no limitation on rules that would prevent this scenario). We should add an explicit acceptance tests case for this.

    It seems to be limited by aws go sdk version of terraform aws provider (3.20.0). At moment, I am still getting below error

    
    Error: Error putting S3 replication configuration: InvalidRequest: Number of distinct destination bucket ARNs cannot exceed 1
        status code: 400, request id: C0A5C4E04398B311, host id: vEVcHcwOS2lN6DRX9Twqt8gwBcF8tf9DM94PGKgtA0op6JmfgPvvBaA3Z8z1bL5oLOCOvnFcE8M=
    

    it's been addressed in https://github.com/aws/aws-sdk-go/pull/3668

    点赞 评论 复制链接分享
  • weixin_40005795 weixin_40005795 2021-01-12 12:26

    Hmm, v3.20.0 built against AWS SDK v1.36.0: https://github.com/hashicorp/terraform-provider-aws/pull/16530. I'll add a new acceptance test and try with the latest trunk code.

    console
    $ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_Replication_MultipleDestinations'
    ==> Checking that code complies with gofmt requirements...
    TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSS3Bucket_Replication_MultipleDestinations -timeout 120m
    === RUN   TestAccAWSS3Bucket_Replication_MultipleDestinations
    === PAUSE TestAccAWSS3Bucket_Replication_MultipleDestinations
    === CONT  TestAccAWSS3Bucket_Replication_MultipleDestinations
        resource_aws_s3_bucket_test.go:1560: Step 1/2 error: Error running apply: 2020/12/09 10:01:38 [DEBUG] Using modified User-Agent: Terraform/0.12.26 HashiCorp-terraform-exec/0.10.0
    
            Error: Error putting S3 replication configuration: InvalidRequest: Number of distinct destination bucket ARNs cannot exceed 1
                status code: 400, request id: A786AF26FAF89ABB, host id: Ek/dB+mcb4roTI+fTsL4jxFAMhR4RuoDNsa1bER6kK9VoKsoe0gWG/iDyxYGBPc23GovYJqebDo=
    
    
    --- FAIL: TestAccAWSS3Bucket_Replication_MultipleDestinations (14.04s)
    FAIL
    FAIL    github.com/terraform-providers/terraform-provider-aws/aws   14.143s
    FAIL
    GNUmakefile:27: recipe for target 'testacc' failed
    make: *** [testacc] Error 1
    

    Investigating...

    点赞 评论 复制链接分享
  • weixin_40005795 weixin_40005795 2021-01-12 12:26

    Priority is used when multiple destinations are specified, so priority must be specified. ~Currently we are only sending priority if a filter is also specified - that needs to be addressed for the example above to work.~

    点赞 评论 复制链接分享
  • weixin_39628247 weixin_39628247 2021-01-12 12:26

    👋 hey , I come about on this error Number of distinct destination bucket ARNs cannot exceed 1 , and while I set priority to all rules I kept getting this error even on latest version of the provider (3.21).

    The problem was that I was not specifying filter {} as I don't have any filters at all. The moment I specified this it worked, even as an empty object. Not sure what will be the best behavior here but just in case anyone else has the same problem.

    Maybe we can update the docs?

    点赞 评论 复制链接分享
  • weixin_40005795 weixin_40005795 2021-01-12 12:26

    Yes, I found that I had to add filter {} when adding the additional acceptance tests. I will make sure that the documentation is updated. Thanks.

    点赞 评论 复制链接分享

相关推荐