Verifying large files with node crypto

Node.js’s built-in crypto library lets you verify the signature of a file with very few lines of (Typescript) code:

verify(filePath: string, signatureBase64: string): boolean {
    const fileData = fs.readFileSync(filePath);
    const verifier = crypto.createVerify('sha256');
    verifier.update(fileData);
    return verifier.verify(this.publicKey, signatureBase64,'base64');
}

What if the file is large and/or you’re running on a low-memory device? This might be the case if you’re writing a firmware updater to run on an embedded device e.g. a Raspberry Pi Zero.

In this case, the above code could fail at runtime due to an out of memory error. This is because fs.readFileSync tries to read the entire contents of the file into memory, which might be more than you have available.

To avoid this, you can use streams. It turns out the Verify class extends <stream.Writable>, so you can pipe data from the input file to it like this:

verify(filePath: string, signatureBase64: string, callback: (err: any, result: boolean) => void) {
    const readStream = fs.createReadStream(filePath);
    const verifier = crypto.createVerify('sha256');
    const publicKey = this.publicKey;

    readStream.on('open', function () {
        readStream.pipe(verifier);

        readStream.on('end', () => {
            const result = verifier.verify(publicKey, signatureBase64,'base64');
            callback(undefined, result);
        });
    });

    readStream.on('error', function(err) {
        callback(err, false);
    });
}

Note that calling verify only after the readStream has ended will prevent you from getting verify.update Error: Not initialised.

Congrats, you can now verify multi-GB images on your 512MB RAM device! (Probably.)

If you have any corrections or improvements, drop me a comment below.

Publishing your node service with DNS-SD/mDNS from an Alpine Linux docker container

Multicast DNS service discovery, aka. Zeroconf or Bonjour, is a useful means of making your node app (e.g. multiplayer game or IoT project) easily discoverable to clients on the same local network.

The node_mdns module worked out-of-the-box on my Mac. Unfortunately things weren’t as straightforward on a node-alpine docker container running on Raspberry Pi Zero, evidenced by this error at runtime:

Error: dns service error: unknown
    at new Browser (/home/app/node_modules/mdns/lib/browser.js:86:10)
    at Object.create [as createBrowser] (/home/app/node_modules/mdns/lib/browser.js:114:10)

Here’s how I managed to solve this. The following was pieced together from a number of sources (linked at the end).

I’ll assume you have a node app using node_mdns to publish your service, and a Dockerfile based on alpine-linux to build your app into an image for running on the Pi.

Firstly, you’ll need to have the alpine packages to run the avahi daemon, along with its development headers and compat support for bonjour. I.e. in your Dockerfile:

FROM arm32v6/node:10-alpine3.9

# Avahi is for DNS-SD broadcasting on the local network; DBUS is how Avahi communicates with clients
RUN apk add python make gcc libc-dev g++ linux-headers dbus avahi avahi-dev avahi-compat-libdns_sd

You’ll need to make sure the DBus and Avahi daemons are started in your container before starting your node app. Since you can only execute a single startup command from your Dockerfile, we’ll need to bundle the commands into a startup script, and run that. In your Dockerfile:

ENTRYPOINT ["./startup.sh"]

And startup.sh:

#!/usr/bin/env sh

dbus-daemon --system
avahi-daemon --no-chroot &
node index.js    # your app script here

Note: --no-chroot is added to avoid this runtime error:

alpine linux netlink.c: send(): Not supported

Build your Docker image (since this is for a Pi Zero in my case, I’m using DockerX to build for the ARMv6 architecture on my Mac. I recommend this over waiting days or weeks for it to build on the Pi Zero):

docker buildx build -t myapp --platform linux/arm/v6 -o type=docker .

Now push then pull your Docker image onto your Raspberry Pi. If you don’t want to use a cloud-hosted registry, I’d recommend taking a look into setting up a local registry to push it directly to the Pi on your local network.

To run your docker image on the Pi, you’ll first need to disable the host OS’s avahi-daemon (if any) to prevent conflicts with the avahi-daemon that will be running inside your alpine-linux container. On Raspbian, you can disable avahi with:

# SSH into your Pi
sudo systemctl disable avahi-daemon

Then to run your docker image:

docker run -d --net=host localhost:5000/myapp

(localhost:5000 here refers to a local docker registry.) Using the host’s network (--net=host) seems to be necessary for mDNS advertisements to function. In theory you should be able to just map port 5353/udp from the container, but this didn’t work. (If you happen to know why, please drop a comment below).

That’s it. If all goes well you should be able to see your service advertised on the local network. E.g. from a Mac on the same network (the last line is our node app’s http service):

$ dns-sd -B _services._dns-sd._udp
Browsing for _services._dns-sd._udp
DATE: ---Fri 29 May 2020---
22:05:29.580  ...STARTING...
Timestamp     A/R    Flags  if Domain               Service Type         Instance Name
22:05:29.582  Add        3  10 .                    _tcp.local.          _hue
22:05:29.582  Add        3  10 .                    _tcp.local.          _hap
22:05:29.582  Add        3  10 .                    _tcp.local.          _workstation
22:05:29.582  Add        3  10 .                    _tcp.local.          _ssh
22:05:29.582  Add        3  10 .                    _tcp.local.          _sftp-ssh
22:05:29.582  Add        2  10 .                    _tcp.local.          _http

Full sample source code on github: https://github.com/kiwiandroiddev/node-alpine-docker-mdns

Credits/References

https://hub.docker.com/r/stanback/alpine-avahi

https://github.com/homebridge/homebridge/issues/613

https://github.com/joyent/smartos-live/issues/669

https://github.com/home-assistant/docker/issues/23

https://stackoverflow.com/questions/30646943/how-to-avahi-browse-from-a-docker-container