AWS Tips 02/2021

Provisioning an S3 bucket as part of your stack is challenging because you can’t manipulate the bucket once it contains objects.

CloudFormation and S3 bucket resource

CloudFormation provides an S3 bucket resource for your stacks. This resource will behave quite differently once you’ve written objects to it. With CloudFormation stacks, there are major differences between empty buckets and buckets with objects. You can test this yourself:

1. Create a CloudFormation stack with just an S3 bucket resource and no additional options.
2. Write a random object to the bucket.
3. Attempt to delete the stack you created.

The result will be an error during deletion of the CloudFormation stack. The error will say that buckets with objects cannot be deleted. From a CloudFormation perspective, this is logical because you didn’t provision objects inside the bucket as part of your stack. You’re responsible for deleting them before you attempt to delete the stack.

Let’s take a look at the policies that CloudFormation provides for resources. One of them is DeletionPolicy. Repeat the above example with the DeletionPolicy: retain set on the bucket resource. The result will be the stack is successfully deleted. However, the bucket resource will be left behind. CloudFormation will just skip deleting it. If you open the S3 section of the web console or if you list your S3 bucket through a CLI command, you’ll see the bucket you provisioned is still around. The resource is left dangling in your account.

We can conclude that an S3 bucket resource containing data cannot be deleted with the stack it was provisioned from. When a resource is provisioned inside a stack, I expect that resource to also be destroyed with the stack. The lifetime of the resource should be driven by the lifetime of it’s stack. EC2 or RDS instances can be used as examples. Compute resources may be destroyed or terminated with your data with the stack. This outlier example of S3 bucket deletion conditions is something to keep in mind.

Using CloudFormation Custom Resource

This issue came up enough times that I have decided to address it. I’ve created a CloudFormation Custom Resource which handles deletion of S3 bucket objects. The Custom Resource is implemented using an AWS Lambda function listing the bucket content and deleting all objects or their versions.

1
2
3
4
5
6

BucketWipeResource:
  Type: Custom::BucketWipe
  Properties:
    ServiceToken: !GetAtt BucketWipeLambda.Arn
    Buckets:
      - !Ref Bucket

The crhelper module is provided by the CloudFormation team. I recommend using it together with Serverless Application Model or SAM tools. Serverless Application Model helps you manage dependencies and deploy the Custom Resource and Lambda function in one step.

The Lambda function is implemented in Python and is using AWS CloudFormation Custom Resource module. Keep in mind this example is limited and designed to illustrate the problem. If your S3 bucket contains many objects, you may need to extend the Lambda function timeout or even consider making the S3 delete requests in parallel.

1
2
3
4
5
6
7
8

@helper.delete
def delete(event, context):
    properties = event['ResourceProperties']
    buckets = properties['Buckets']
    for bucket_id in buckets:
        bucket = s3.Bucket(bucket_id)
        bucket.object_versions.delete()
        bucket.objects.all().delete()

S3 Bucket Wipe Example Code

You may be interested in seeing or running the full example. I made a demo SAM project containing the Custom Resource available under the MIT license. Inspect the example SAM stack and the S3 Wipe Lambda function code. You can review the full S3 bucket wipe example code or make a copy of the repository using the following command:

git clone https://git.adamkonrad.com/aws_bucket_wipe.git

The demo assumes you have SAM CLI installed and setup. Run sam build and sam deploy from the directory as you’d normally would to deploy other serverless applications.

I hope you find this code useful in your stacks and applications.