Set, Get and Delete AWS S3 bucket policies
In the last blog post, we have learned how to create S3 buckets. By default, all S3 buckets are private and there is no policy attached to them. S3 policies can define which user can perform which kind of actions on this bucket. If you want to know how S3 policies are different from IAM policies you can read this post.
In this tutorial, let us learn how we can manage S3 bucket policies. We will learn how to check existing bucket polices, attach new ones and delete policies from S3 console as well as pragmatically using AWS CLI and Python.
Using S3 Console
Reading Existing Bucket Polices
First, we will understand, how to check existing bucket policies from the S3 console. As mentioned before all S3 buckets have no policy attached by default. You can validate that, when you select any bucket then click on permissions -> and then bucket policy.
Attaching Bucket Policy
We can generate AWS policy using a simple tool provided by AWS. Before we attach policy, let us try to access S3 bucket using “testuser”. This user currently does not have any access to S3. So when we try to list files in the S3 bucket we will see the following output.
We can generate the following policy using the AWS Policy Generator. This policy grants all access to S3 bucket “testbucket-frompython-1” to IAM user “testuser”.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
{ "Id": "Policy1586690842642", "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1586690839614", "Action": "s3:*", "Effect": "Allow", "Resource": "arn:aws:s3:::testbucket-frompython-1", "Principal": { "AWS": [ "arn:aws:iam::195556345987:user/testuser" ] } } ] } |
Now if we have attached this policy correctly and try to list S3 files from “testbucket-frompython-1” bucket, we should see some output now.
Deleting bucket policy
Deleting Bucket’s policy is easy. You can again open the S3 bucket, go to the permissions tab and then to Bucket Policy and click on the Delete button. This will delete all polices attached to this bucket.
Managing Bucket Polices With AWS CLI
Listing Bucket Polices
You can easily list bucket policies using the following AWS CLI command. If you have policies attached to the bucket you will see output otherwise you will get “The bucket policy does not exist” message as shown in the below image. ( We do not have any policy attached to this bucket as we have deleted all attached policies in the last step.)
1 |
aws s3api get-bucket-policy --bucket testbucket-frompython-1 --profile admin-analyticshut |
Attaching Policy to S3 Bucket
For attaching policy to S3 bucket using CLI, we need to create JSON document with policy that we want to attach. We will be using same policy attached mentioend above, which will grant “testuser” all access to S3 bucket. Then we can run following command to attach policy to bucket. (please use proper file path when using command below.)
1 2 3 4 |
aws s3api put-bucket-policy \ --profile admin-analyticshut \ --bucket testbucket-frompython-1 \ --policy file://D:\\Projects\\python\\Working-with-AWS-S3\\S3-Buckets\\AWS-CLI\\bucket_policy.json |
Deleting S3 Bucket Policy
We can also delete S3 bucket policy using simple CLI command.
Managing Bucket Policies With Python
At last we will write python scripts to get, put and delete S3 bucket policies. let us get started.
Get Bucket Policies
We can get S3 bucket policies using following code. If there are no policies attached to the bucket, then get_bucket_policy() function throws error. We have to manage that in code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
import boto3 import pprint from botocore.exceptions import ClientError # # setting up configured profile on your machine. # You can ignore this step if you want use default AWS CLI profile. # boto3.setup_default_session(profile_name='admin-analyticshut') s3 = boto3.client('s3') try: response = s3.get_bucket_policy(Bucket='testbucket-frompython-1') print(pprint.pprint(response)) except ClientError as e: # if you do not have any policy attached to bucket it will throw error # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation: # The bucket policy does not exist print(e) |
As usual, We can use S3 resource as well to list bucket policies.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import boto3 import pprint from botocore.exceptions import ClientError # # setting up configured profile on your machine. # You can ignore this step if you want use default AWS CLI profile. # boto3.setup_default_session(profile_name='admin-analyticshut') # using s3 resource s3_resource = boto3.resource('s3') try: bucket_policy = s3_resource.BucketPolicy('testbucket-frompython-1') # bucket policy resource has policy attribute which returns policy as JSON string print(bucket_policy.policy) except ClientError as e: print(e) |
Put S3 Bucket Policy
We will be using same above policy. When using Python we do not need to store policy in separate document.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
import boto3 import pprint from botocore.exceptions import ClientError import json # # setting up configured profile on your machine. # You can ignore this step if you want use default AWS CLI profile. # boto3.setup_default_session(profile_name='admin-analyticshut') s3 = boto3.client('s3') policy = """{ "Id": "Policy1586690842642", "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1586690839614", "Action": "s3:*", "Effect": "Allow", "Resource": "arn:aws:s3:::testbucket-frompython-1", "Principal": { "AWS": [ "arn:aws:iam::195556345987:user/testuser" ] } } ] }""" try: response = s3.put_bucket_policy(Bucket='testbucket-frompython-1', Policy=policy) print(pprint.pprint(response)) except ClientError as e: # if you do not have any policy attached to bucket it will throw error # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation: # The bucket policy does not exist print(e) |
We can achieve same effect using bucket resource as well.
1 2 3 |
s3_resource = boto3.resource('s3') bucket_policy = s3_resource.BucketPolicy('testbucket-frompython-1') bucket_policy.put(Policy=policy) |
Deleting S3 Bucket Policy
Following python code snippet can be used to delete attached bucket policy.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
import boto3 import pprint from botocore.exceptions import ClientError # # setting up configured profile on your machine. # You can ignore this step if you want use default AWS CLI profile. # boto3.setup_default_session(profile_name='admin-analyticshut') s3 = boto3.client('s3') # return None response = s3.delete_bucket_policy(Bucket='testbucket-frompython-1') # if bucket does not have any policy attached, it will not throw any error # using s3 resource s3_resource = boto3.resource('s3') bucket_policy = s3_resource.BucketPolicy('testbucket-frompython-1') # returns None bucket_policy.delete() |
Conclusion
I hope this article helped you in understanding different ways in which you can manage S3 bucket policies. You can try performing operations at each step to validate if the policy is attached or deleted correctly. You can get code created in this blog from this git repo. If you have any questions please let me know. See you in the next blog.
is it possible to list bucket from console once you attach the bucket policy to the bucket? lets go with your example above. i know it works from CLI perspective but what about console. can i list this bucket with attached policy in console as the allowed user?
food for thought….
Hello Santosh,
Thanks for taking an interest. I am not sure I understand you correctly but if you are saying will you be able to list all buckets on the console to which you have access to then yes you can.
If you have any more questions do reach out to me.
Thanks
Mahesh