A Beginner’s Guide to s3cmd

Fateh Ali Aamir
3 min readOct 2, 2024

--

Photo by From Marwool on Unsplash

The s3cmd tool is a versatile command-line interface for managing Amazon S3 objects and buckets. It allows users to create and delete buckets, upload, download, delete, and sync files, and manage S3 configurations. Here’s a quick overview of some common s3cmd usage scenarios and tips:

To install s3cmd, you can follow these steps based on your operating system:

For Linux or macOS:

  1. Install via Package Manager:
  2. On Ubuntu/Debian:
sudo apt-get update sudo apt-get install s3cmd
  1. On CentOS/RHEL:
sudo yum install s3cmd
  1. On macOS (using Homebrew):
brew install s3cm
  1. Install via pip (Python Package Manager):

If you have Python and pip installed, you can install s3cmd via pip:

pip install s3cmd

For Windows:

  1. Install using pip:
pip install s3cmd
  1. Alternative — Using Chocolatey: If you have Chocolatey installed, run:
choco install s3cmd

Configuration

To start using s3cmd, you need to configure it with your AWS credentials:

s3cmd --configure

You will be prompted to enter your AWS Access Key, Secret Key and some other parameters.

Example of Configuration:

New settings:
Access Key: 8d384-....-8b06f
Secret Key: GCOSU-....-b3P
Default Region: arc-is-haf-1
S3 Endpoint: s3.arc-is-haf-1.genesiscloudusercontent.com
DNS-style bucket+hostname:port template for accessing a bucket: s3.arc-is-haf-1.genesiscloudusercontent.com/%(bucket)
Encryption password:
Path to GPG program: /opt/homebrew/bin/gpg
Use HTTPS protocol: True
HTTP Proxy server name:
HTTP Proxy server port: 0

Common Commands

1. Create a Bucket:

Creates a new bucket with the specified name on S3.

s3cmd mb s3://your-bucket-name

2. Delete a Bucket:

Removes a bucket from S3 (the bucket must be empty to be deleted).

s3cmd rb s3://your-bucket-name

3. List Buckets and Objects:

List all buckets:

s3cmd ls

List objects in a specific bucket:

s3cmd ls s3://your-bucket-name/

4. Upload Files to S3:

Uploads a file from your local system to the specified S3 bucket.

s3cmd put your-file.txt s3://your-bucket-name/

5. Download Files from S3:

Downloads a file from the S3 bucket to a local directory.

s3cmd get s3://your-bucket-name/your-file.txt local-path/

6. Delete Files from S3:

Deletes a specific file from the S3 bucket.

s3cmd del s3://your-bucket-name/your-file.txt

7. Synchronize Local Directory with S3:

To sync a local directory to an S3 bucket:

s3cmd sync /local/directory/ s3://your-bucket-name/

To sync from S3 to your local directory:

s3cmd sync s3://your-bucket-name/ /local/directory/

You can learn more about s3cmd at https://s3tools.org/usage.

In conclusion, s3cmd is a powerful tool for anyone working with Amazon S3, offering an easy-to-use command-line interface for managing storage tasks. Whether you’re uploading, downloading, syncing, or managing buckets and files, s3cmd simplifies these operations with just a few commands. Once configured with your AWS credentials, it becomes a versatile utility for developers, system administrators, and cloud engineers. By mastering the common commands, you’ll be able to efficiently manage your S3 environment; making s3cmd an invaluable part of your toolkit.

--

--