I am working on a Lambda that needs to list the objects in an S3 bucket and build a URL based on the files *item.Key. The problem I am fast approaching is that soon the bucket will contain more than 1000 items.
Currently, I am using a for
to only return the last file that contains a specific item.Key:
resp, err := s3Client.ListObjects(&s3.ListObjectsInput{Bucket: aws.String(bucket)})
if err != nil {
fmt.Printf("Unable to list items in bucket %q, %v", bucket, err)
}
for _, item := range resp.Contents {
if strings.Contains(*item.Key, "/output.txt") {
lastItem = *item
}
}
s := fmt.Sprintf("https://s3.%s.amazonaws.com/%s/%s", region, bucket, *lastItem.Key)
fmt.Println(s)
My understanding is that once the bucket reaches over 1000 items the 1000th item will be returned each time the Lambda is run.
By default bucket items are returned by oldest first, the best way I can see around this is to list the items by newest first and have the for
stop after the first item. Is this possible?
If not, has anyone else solved a similar issue?