Managing Files in Amazon S3 from Node.js

Adding and removing files from S3 is a breeze with the AWS JavaScript SDK.

Sam Magura

November 6, 2024

Amazon S3 is perhaps the most well-known service for storing data in the cloud. S3 enables many complex use cases, but at the most basic level, it allows you to upload files and then download them later.

In this post, we'll explore using the AWS SDK to add and remove S3 objects from a Node.js application. The AWS access credentials will be housed in Zero and retrieved using the Zero TypeScript SDK .

🔗 The full code for this example is available in the zerosecrets/examples GitHub repository.

Secure your secrets conveniently

Zero is a modern secrets manager built with usability at its core. Reliable and secure, it saves time and effort.

Getting the AWS Access Key

We'll assume you already have an AWS account — if not, you can create one for free. In the AWS Console, search for IAM. Then click the "My security credentials" link on the right-hand side:

Link to get to your access keys in AWS IAM

Then, click the "Create access key" button. You'll be presented with an access key ID and a secret access key. For now, save both of these values on your local PC.

The next step is to insert these values into the Zero secrets manager. In Zero, create a new project and save its token to a text file on your PC. Now, create an AWS secret and paste in the access key ID and secret access key. Now we're all set as far as secrets go, and can begin coding.

Creating the Node.js App

Now, create a simple Node.js application by creating a new directory and installing these basic packages:

Terminal

npm install --save-dev @types/node typescript

You can use npx tsc --init to create a new TSConfig file. Ensure you are using "module": "Node16" and "target": "es2022" or greater, since we'll be utilizing top-level await.

Finally, create an index.ts to house the code.

Initializing the S3 Client

The next step will use both the Zero TypeScript SDK and the @aws-sdk/client-s3 package, so install them both:

Terminal

npm install --save-dev @types/node typescript

In index.ts, we begin by fetching the secrets from Zero, using the Zero token that is passed in as an environment variable:

index.ts

if (!process.env.ZERO_TOKEN) {
  throw new Error('Did you forget to set the ZERO_TOKEN?')
}

const secrets = await zero({
  token: process.env.ZERO_TOKEN,
  pick: ['aws'],
  callerName: 'staging',
}).fetch()

if (!secrets.aws) {
  throw new Error('AWS secret was not present.')
}

Next, we can initialize the S3 client:

index.ts

const s3 = new S3({
  region: 'us-east-2', // Replace with your AWS region
  credentials: {
    accessKeyId: secrets.aws.access_key_id,
    secretAccessKey: secrets.aws.secret_access_key,
  },
})

Now we're ready to start managing S3 objects!

Adding an Object

Our program will be able to both add and remove objects, so we need to have some way of determining which action to take. We'll add a command-line argument for this purpose. At the top of the file, access the first command-line argument and validate that it is either add or remove:

index.ts

const command = process.argv[2]

if (command !== 'add' && command !== 'remove') {
  throw new Error('You must specify "add" or "remove" as a command-line argument.')
}

Then, later in the file, after initializing the S3 client, we can write the code for adding an object. The command for adding an object is PutObjectCommand. You can find the documentation on this command here . In our code, we set the content of the object to test, and name the object test-object.

index.ts

const BUCKET = 'YOUR_BUCKET_NAME' // TODO
const OBJECT_KEY = 'test-object'

if (command === 'add') {
  const s3Command = new PutObjectCommand({
    Body: 'test',
    Bucket: BUCKET,
    Key: OBJECT_KEY,
  })

  await s3.send(s3Command)

  console.log('Object added.')
} else if (command === 'remove') {
  // TODO
}

Now, run the program, passing in the Zero token as an environment variable:

Terminal

ZERO_TOKEN='YOUR_ZERO_TOKEN' npm start -- add

You should now see the object in your S3 bucket in the AWS Console:

The object is now visible in the S3 bucket

Removing the Object

Removing the object is similar to adding an object, except the command is DeleteObjectCommand . The only difference in the command arguments is that Body is not required.

index.ts

const s3Command = new DeleteObjectCommand({
  Bucket: BUCKET,
  Key: OBJECT_KEY,
})

await s3.send(s3Command)

console.log('Object removed.')

Run the program again, with remove as the command-line argument, and you will no longer see the object in the S3 UI.

Wrapping Up

This post showed you the basic steps to manage objects in an S3 bucket from a Node.js program using the @aws-sdk/client-s3 package. There is obviously much more that can be done with S3, but this should be enough to get you started. If you are learning AWS, a next step could be to use Amazon Cloudfront to serve a static website, with the HTML/CSS/JS files stored in S3.