Published on

Create AWS Lambda Layer with Python 3 dependencies using Docker

Authors

If you want to use custom dependencies for your application or function that is hosted on AWS Lambda and also want to share those across multiple projects, then AWS Lambda Layers is a good option for that use case.

The way it works is basically that you build your dependencies, bundle them together, and upload them as Lambda layers.

And you can refer to AWS Lambda layers in your AWS functions.

That's pretty much it.

Let's see how to do that for Python 3 dependencies, and we'll be using Docker to bundle the required dependencies.

Step 1: Downloading the official build docker image

We'll be using the official build-python3.10 Docker image for building and bundling the dependencies.

docker pull public.ecr.aws/sam/build-python3.10:1.106.0-20240104015600

Step 2: Run the docker image

Run the docker image and make sure you're using the correct platform by configuring --platform option

Since I'm on a Apple M1 device but my AWS Lambda function is on x86, I'll be passing --platform=linux/amd64 to run the Docker image with the correct architecture.

And we'll be mounting the current working directory as well.

docker run -it -v $(pwd):/var/task --platform=linux/amd64 public.ecr.aws/sam/build-python3.10:1.106.0-20240104015600

Now you can install the dependencies that you want. In my case, I'll just install OpenAI-related sdk

pip install langchain openai tiktoken  -t  ./python

Step 3: Zip it and upload it in AWS Lambda layers

Now zip the dependencies and upload them to AWS Lambda layers

zip -r python.zip ./python

That's pretty much it.

Step 4: Use your dependencies

To use the dependencies for your Lambda function, you need to add the upload layer.

And post that, you should be able to use your dependencies without any issues

When we can't use AWS Layers?

  • If the total unzipped size of a function and ALL the layers together exceeds 250 MB
  • When a function can use up to 5 layers at a time.

Looking for other alternatives?

  • Do you know you can use container image in AWS Lambda? Meaning, you can just run any docker image within your AWS Lambda function.
  • AWS Fargate might be a good alternative for you as well.

Reference

Happy building and shipping 🎉