Veeam Amazing Object Storage Tips & Techniques Part 3

AWS CLI Profiles

In my previous articles I used the AWS CLI to interact with the object storage platforms for demonstration purposes.  This article will focus on a very powerful feature of the AWS CLI, the usage of profiles.

If you find yourself often interacting with several object storage platforms and/or use multiple object storage accounts, then using profiles for the AWS CLI will simplify your efforts.  It has for me.

Creating the first profile

For an example, let’s say you need to access three object storage platforms regularly.  For the remainder of this article, the storage platforms will be:

• platform1
• platform2
• platform3

You could use the “aws configure" command to reset your environment to use the credentials and region every time you switch to another object storage target.

My first step is to see what AWS cli profiles I have configured and set:

Let’s see what profiles are configured:

aws configure list-profiles

There are no existing profiles, so let’s create one up for object storage “platform1”:

aws configure --profile platform1
AWS Access Key ID [None]: platform1-access-key
AWS Secret Access Key [None]: platform1-secret-access-key
Default region name [None]: platform1-region
Default output format [None]:

Now we recheck to see if the profile was created using the “aws configure list-profiles” command we used earlier:

aws configure list-profiles
platform1

We now see “platform1” was created.  To view the details of the newly created profile we can use the “aws configure list” command:

aws configure list --profile platform1
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                platform1           manual    --profile
access_key     ****************-key shared-credentials-file
secret_key     ****************-key shared-credentials-file
    region         platform1-region      config-file    ~/.aws/config

Notice the entry “config-file” and its location of “~/aws.config”.  This is the file where the profiles that you configure are stored.  There are actually two files.  They are the “config” and “credentials” files.  Both files were automatically created when we used the “aws configure” command earlier to create the “platform1” profile.

You can view the contents with any text editor to see the contents of the files.

The “~/.aws/config” file for this demonstration looks like this:

[profile platform1]
region = platform1-region

It contains the profile name we used “platform1” and the region we specified “platform1-region”.

When we open up the “~/.aws/credentials” file, we see the following:

[platform1]
aws_access_key_id = platform1-access-key
aws_secret_access_key = platform1-secret-access-key

The name of the file sort of gives us a clue as to what is in the file 😀.  The credentials, in clear text, that we assigned to the “platform1” profile.

At the start of this article, I mentioned that we wanted to create 3 AWS CLI profiles to access our object storage platforms.  So, let’s create the remaining two.

Creating more profiles

There are two methods to create the remaining AWS CLI profiles.  The 1st is to use the “aws configure” command to create them.  The 2nd is we can use a text editor and add the profile information directly to the “~/.aws/config” and “~/.aws/credentials” files.

For demonstration purposes, I will use the text editor method:

Here is my updated “~/.aws/config” file:

[profile platform1]
region = platform1-region

[profile platform2]
region = platform2-region

[profile platform3]
region = platform3-region

Here is the updated “~/.aws/credentials” file:

[platform1]
aws_access_key_id = platform1-access-key
aws_secret_access_key = platform1-secret-access-key

[platform2]
aws_access_key_id = platform2-access-key
aws_secret_access_key = platform2-secret-access-key

[platform3]
aws_access_key_id = platform3-access-key
aws_secret_access_key = platform3-secret-access-key

We can verify that the profiles were created properly by using the “aws configure list-profiles” and “aws configure list” commands:

aws configure list-profiles
platform1
platform2
platform3

aws configure list --profile platform1
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                platform1           manual    --profile
access_key     ****************-key shared-credentials-file
secret_key     ****************-key shared-credentials-file
    region         platform1-region      config-file    ~/.aws/config


aws configure list --profile platform2
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                platform2           manual    --profile
access_key     ****************-key shared-credentials-file
secret_key     ****************-key shared-credentials-file
    region         platform2-region      config-file    ~/.aws/config


aws configure list --profile platform3
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                platform3           manual    --profile
access_key     ****************-key shared-credentials-file
secret_key     ****************-key shared-credentials-file
    region         platform3-region      config-file    ~/.aws/config

Using the AWS CLI profiles

Now that we have created the AWS CLI profiles, let’s see how we can use them.

First, let’s create a bucket using the “platform1” profile:

aws s3api create-bucket --bucket platform1-bucket1 --profile platform1 --endpoint-url <your.objectstorage1.endpoint-url>
{
    "Location": "/platform1-bucket1"
}

We can also use the same profile to see what buckets exist for that account:

aws s3 ls --profile platform1 --endpoint-url <your.objectstorage1.endpoint-url>
2022-12-23 11:40:51 platform1-bucket1

Now let’s repeat those steps, but this time using a different object storage platform and profile “platform2”.

aws s3api create-bucket --bucket platform2-bucket1 --profile platform2 --endpoint-url <your.objectstorage2.endpoint-url>
{
    "Location": "/platform2-bucket1"
}

aws s3 ls --profile platform2 --endpoint-url <your.objectstorage2.endpoint-url>2022-12-23 
11:45:29 platform2-bucket1

We could’ve run the “aws configure” command to set the environment to use the second object storage platform:

aws configure --profile platform2
AWS Access Key ID [None]: platform2-access-key
AWS Secret Access Key [None]: platform2-secret-access-key
Default region name [None]: platform2-region
Default output format [None]:

But that requires you to enter/re-enter the credentials and the other settings each time.  This can be time consuming and prone to errors.

Hopefully I successfully demonstrated that by using AWS CLI profiles, you can quickly switch between object storage platforms by simply changing the profile and endpoint-url.  I tried to cover the most important features of using profile, but for additional information please check out the official  AWS CLI profile documentation page.

In an upcoming article, I will show you how you can use the AWS CLI profile in a Python script and explain the benefits of doing so.