Bulk files/folder upload to S3 using AWS CLI 2

Applicable for Ubuntu 20.04 and its derivatives.

Creating a IAM User

To make any API call to AWS, we need a IAM user with appropriate permissions. According to the your purpose, create a IAM user and get the followings

IAM user name
Access Key ID
Secret Access Key

You can download a csv file with these information provided by the AWS after creation of IAM user.

Install AWS CLI in Local Machine

Install AWS CLI

Install curl if it is not install in your machine. Also, you can use wget the default and preinstalled download manager for Ubuntu.

sudo apt-get install curl

Download the AWS installation file

curl "[https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip](https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip)" -o "awscliv2.zip"

Go to the downloaded directory and extract the files

unzip awscliv2.zip

Now, run the installation file

sudo ./aws/install

This will install the AWS CLI 2 in your system.

To verify the installation, use

aws --version

Configure Account

To configure your account

aws configure

Now insert the access key id, secret access key, default region name, your desired output format.

This should configure the CLI with your account credentials.

You can test the configuration by invoking the following commands

aws s3 ls

This should return all the buckets of your region.

Upload Files and Folders

If you do not have a S3 bucket create one.

Upload a Single File

To upload a single file we use the following

aws s3 cp local_file_path s3://bucket_name/

Here the local_file_path stands for the local file path we are uploading from our local machine.

The bucket_name stands for our desired cloud S3 Bucket Name.

Example:

aws s3 cp /home/nahid/Documents/data.json s3://test_bucket/

This will upload a file named data.json to a bucket named test_bucket.

Upload All Files and Folder Recursively

To upload a folder along with all its files and folders, we can do the followings,

aws s3 cp local_folder_path s3://bucket_name/ --recursive

Here local_folder_path is the folder we are uploading.

The bucket_name is the desired cloud S3 bucket.

Example:

aws s3 cp /home/nahid/Documents/build s3://test_bucket/ --recursive

Here we are uploading a folder named build to the bucket named test_bucket.

Reference

https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html

Upload files and folder to S3 from Ubuntu 20.04 using AWS CLI 2

Comments

AWS

Deploy Node.js Application To AWS Beanstalk With SSL and Load Balancer

More from this blog

The Comprehensive Guide to REST API Architecture and Conventions

JavaScript Loading Strategies in Web Page: async and defer

DOM Positioning

CSS Selector Specificity

Understanding CSS Display Property

Creating a IAM User

Install AWS CLI in Local Machine

Install AWS CLI

Configure Account

Upload Files and Folders

Upload a Single File

Upload All Files and Folder Recursively

Reference

Command Palette

Comments

AWS

Deploy Node.js Application To AWS Beanstalk With SSL and Load Balancer

More from this blog

Creating a IAM User

Install AWS CLI in Local Machine

Install AWS CLI

Configure Account

Upload Files and Folders

Upload a Single File

Upload All Files and Folder Recursively

Reference