AWS S3-Backed Node.js File System with s3fs

Ever wanted to use AWS S3 as a replacement for your native file system (fs) in Node.js?

There could be lots of cases in which you might want to do it like using it within a serverless environment (or even just for fun)

With s3fs library you can do it.

With this, you can just replace your existing fs with AWS S3 and it supports most of the common methods like:

How does it work?

Under the hood, it's a wrapper+helper for official AWS S3 SDK with a similar to that of fs module.

How to use it?

Install the library by running the following command

npm install @cyclic.sh/s3fs

Then, configure the following environment variables.

export AWS_REGION=us-east-1
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=

If those variables are not configured then it'll fall back to the native fs module instead (with a warning)

Writing & Reading a File

Here's how to read and write a file using s3fs

require("dotenv").config();
// I prefer to explictly configure it so that we don't get conflict from anything else
const config = {
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
};

const S3_BUCKET_NAME = process.env.S3_BUCKET_NAME;
const fs = require("@cyclic.sh/s3fs/promises")(S3_BUCKET_NAME, config);

const user = {
  name: "Ashik Nesin",
  website: "https://AshikNesin.com",
};

async function run() {
  await fs.writeFile("s3fs/user.json", JSON.stringify(user));

  console.log(`Saved json file to AWS S3`);

  const currentPath = `${process.cwd()}/s3fs/user.json`;

  console.log({ currentPath });

  const json = JSON.parse(await fs.readFile(currentPath));

  console.log(`Retrive json file from AWS S3`);

  console.log(JSON.stringify(json, null, 2));
}
run();

You can play around with this sample repo

Heads up!

• It seems like the library doesn't have official support for ES Modules.
• And it's not that configurable. For example, currently, it prepends the current working directory when you try to write a file (and there is no way to turn it off 😓) so using it for a production app might not be good choice at the moment.

Happy S3 files!