Bulk files/folder upload to S3 using AWS CLI 2

Applicable for Ubuntu 20.04 and its derivatives.

Creating a IAM User

To make any API call to AWS, we need a IAM user with appropriate permissions. According to the your purpose, create a IAM user and get the followings

You can download a csv file with these information provided by the AWS after creation of IAM user.

Install curl if it is not install in your machine. Also, you can use wget the default and preinstalled download manager for Ubuntu.

sudo apt-get install curl

Download the AWS installation file

curl "[https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip](https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip)" -o "awscliv2.zip"

Go to the downloaded directory and extract the files

unzip awscliv2.zip

Now, run the installation file

sudo ./aws/install

This will install the AWS CLI 2 in your system.

To verify the installation, use

aws --version

To configure your account

aws configure

Now insert the access key id, secret access key, default region name, your desired output format.

This should configure the CLI with your account credentials.

You can test the configuration by invoking the following commands

aws s3 ls

This should return all the buckets of your region.

If you do not have a S3 bucket create one.

To upload a single file we use the following

aws s3 cp local_file_path s3://bucket_name/

Here the local_file_path stands for the local file path we are uploading from our local machine.

The bucket_name stands for our desired cloud S3 Bucket Name.

Example:

aws s3 cp /home/nahid/Documents/data.json s3://test_bucket/

This will upload a file named data.json to a bucket named test_bucket.

To upload a folder along with all its files and folders, we can do the followings,

aws s3 cp local_folder_path s3://bucket_name/ --recursive

Here local_folder_path is the folder we are uploading.

The bucket_name is the desired cloud S3 bucket.

Example:

aws s3 cp /home/nahid/Documents/build s3://test_bucket/ --recursive

Here we are uploading a folder named build to the bucket named test_bucket.