Fun Challenge: A Script To Handle Lambda Layer Building For Python And Node.js

For both Windows and MacOS, somewhat more convoluted than expected

Juggling pythons and green gems (kind of), image by Midjourney

AWS Cloud Development Kit makes it super easy to compress and deploy a Lambda layer:

const layer = new LayerVersion(stack, 'sample-layer', {
    code: Code.fromAsset('./layers/build/sample-layer'),
    ...
});

But how do we prepare the layer folder in the first place?

Building Binary Python Dependencies For AMD64 And ARM64 Lambda Layers

I recently needed to upload a tricky Python layer as part of a CDK deployment, and my original layer build script just wasn’t up for the task. That’s because it was written exclusively for Node.js layers, which aren’t quite as demanding.

Side note: when it comes to tooling, Python’s always messier than Node.js. Just sayin’.

First Iteration

For Node.js layers, all I needed were source folders that contain a package.json file and optional subfolders containing modules to be loaded directly by the Lambda functions. The build folders and node_modules folders are .gitignore'd so that I don’t include all the dependencies themselves in my repository.

An example source folder for a Node.js Lambda layer, from the aws-cdk-js-dev-guide

The build script’s functionality was very simple:

1. Remove any build folders without a matching source folder
2. For each source folder, get a checksum of the folder. If the folder has been updated, copy everything except the package-lock.json file* and node_modules folder into the build folder and run npm install from there.

* As I typed this, I realized I could improve this logic by using npm ci if the package-lock.json file exists and npm install if it doesn’t. I’ve just updated the aws-cdk-js-dev-guide accordingly.