Different ways to configure credentials with boto3

Updated On February 12, 2021 | By Mahesh Mogal

Boto3 is python's library to interact with AWS services. When we want to use AWS services we need to provide security credentials of our user to boto3. Like most things in life, we can configure or use user credentials with boto3 in multiple ways. Some are worst and never to be used and others are recommended ways. In this blog, let us take a look at how to configure credentials with boto 3.

Using as method parameters

This is the easiest way to use user credentials with boto3. And in my opinion, this is the worst way to configure boto3. Here we can simply pass our access key id and secret access to boto3 as a parameter while creating service client or resource.

With this approach user, keys are visible to everyone. If you commit such code to GitHub, anyone who has access to your repository can use these user credentials and have access to your AWS account. I do not need to tell you what can happen next. That is why I will recommend not use this way of setting AWS credentials with boto3.

Use a common configuration file

One simple way to abstract access key and secret access key is the starting session in another file.

Then you can import this session file in another python file and use it to start AWS sessions to connect with services.

Though this method prevents direct visible access to AWS credentials, there is still an issue when sharing your code to someone or adding it to GitHub. You have to make sure not to commit that file to GitHub.

Environment variables

Boto3 automatically checks for environment variables. If it finds these variables it will use them for connecting to AWS.

  • AWS_ACCESS_KEY_ID - The access key for your AWS account.
  • AWS_SECRET_ACCESS_KEY - The secret key for your AWS account.

Once you set these environment variables, you can directly create boto3 client or session for service. In the backend, boto3 will use these keys to communicate with AWS

With this approach, you can be sure that your access key is only used on your machine It is not easily visible to someone watching. You can also share your code on GitHub or to some person without any worries about exposing your user credentials.

One drawback of this method is when you have multiple AWS users (from different AWS accounts or for different AWS roles) then switching between them becomes difficult. You have to change environment variables each time when you want to use different users. And what happens when you want to use multiple users' credentials (like copying files from s3 bucket in one account to s3 bucket in another account) in one single session?

Using AWS CLI profile

This is similar to setting up Environment variables on your machine. In this case, Boto3 uses credentials that you have used when setting up a default profile while configuring AWS CLI. You can learn more about how to configure AWS CLI here.

Once you have configured AWS CLI, you can directly use boto3 to create a service client or resource.

But this approach has the same drawback, what if when we have multiple user profiles? Can we tell boto3 which profile to use when connecting to AWS? Well, of course, we can.

Using multiple AWS CLI profiles

There is a simple way to state profile name while initiating client in Boto3.

Conclusion

We have seen different ways to configure credentials with Boto3. Configuring AWS CLI profiles and using different profiles depending on our need is way to go!!! Hope this helps. If you have any questions please let me know.

AWS CLI profiles with Boto3

Mahesh Mogal

I am passionate about Cloud, Data Analytics, Machine Learning, and Artificial Intelligence. I like to learn and try out new things. I have started blogging about my experience while learning these exciting technologies.

Stay Updated with Latest Blogs

Get latest blogs delivered to your mail directly.

Recent Posts

Where and Filter in Spark Dataframes

In this blog, we will learn how to filter rows from spark dataframe using Where and Filter functions.

Where and Filter in Spark Dataframes
Read More
Distinct Rows and Distinct Count from Spark Dataframe

Getting distinct values from columns or rows is one of most used operations. We will learn how to get distinct values as well as count of distinct values.

Distinct Rows and Distinct Count from Spark Dataframe
Read More
Sorting in Spark Dataframe

In this blog, we will learn how to sort rows in spark dataframe based on some column values.

Sorting in Spark Dataframe
Read More

Leave a Reply

Your email address will not be published. Required fields are marked *

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram
Share via
Copy link
Powered by Social Snap