CircleCI Hacks: Caching NPM Package Dependencies

CircleCI’s caching is wonky, here’s a way around the weirdness

A Caucus Race, image by Midjourney

Problem 1: Dynamic Cache Keys

Caching Dependencies - CircleCI

Let’s begin by reading the documentation on CircleCI caching keys and templates. This is where we learn about cache keys, eg. myapp-{{ checksum "package-lock.json" }}, which enable you to relate your cache to the specific contents of a file (amongst other things).

This is a fantastic idea, because the cache should only be updated when the dependencies change, and that’s what’s indicated by changes to the package.json and package-lock.jsonfiles.

There are two ways to install packages with npm:

npm install

The first, npm install, will install the dependencies listed in thepackage.json file. This allows for npm to select the optimal versions of the upstream dependencies based on the version specifications in package.json as well as those in the package.json files of the aforementioned upstream dependencies.

npm ci

npm ci — “ci” stands for “clean install” — ignores the contents of the package.json file and installs precisely the versions of the upstream dependencies as specified by the package-lock.json file. This means that your test and deployment behaviors are far more predictable between runs.

While it can be argued that more deterministic is generally better, both approaches have their place.

Back to the caching

The first thing to note is that cache keys are calculated dynamically. This means that a key provided as a default in the pipeline parameters, as in the following example, is re-calculated each and every time it’s referenced:

parameters:
  my-cache-key:
    type: string
    default: my-key-{{ checksum "package.json" }}

command:
  steps:
    - restore_cache:
        # here the checksum will be calculated on the original…