OpenZFS: adding support for Chacha20-Poly1305 encryption

In 2021, while at Fastmail, I oversaw rebuilding our mail storage architecture around OpenZFS. I rapidly became enamoured of it, and with over a year of serious experience now under my belt I’ve pretty much settled on it as the only option worth considering for serious storage.

In mid-2022 I was playing with it on some small devices I had at home, and noticed that enabling encryption was adding substantial overhead, to the point of being almost unusable on low-powered devices like the Raspberry Pi. I think these days encryption should be table stakes for any filesystem, so I set out to find out why.

My first observation was that OpenZFS did not have hardware acceleration for ARM CPUs, but through further investigation it turns out it wouldn’t have mattered, as the CPU in the Raspberry Pi does not have hardware encryption extensions. AES-GCM is well known to be hostile to software implementations, so I started looking for other options.

To anyone with cryptography experience the problem and solution would have been obvious, but that wasn’t me, so there were a lot of false starts and misunderstandings before I eventually learned that Chacha20-Poly1305 was the logical choice of algorithm when you don’t have supporting hardware.

Over a couple of months, I learned how the underlying primitives worked and were put together into a working cryptosystem, and then built a new encryption option for OpenZFS to support it. It achieved the goal I set, in that the overheads added by encryption have come down from ~70% to ~25% on a Raspberry Pi. Even on a high-end Intel CPU with full hardware support, a generic software version of Chacha20-Poly1305 is coming close to 80% throughput compared to accelerated AES-GCM, which is impressive.

At time of writing the change under review, but I hope to see it available in an OpenZFS release in due course.

I’m very pleased with all of this. I came in with an idea in mind, but definitely without the knowledge to put all the pieces together. But, I’ve done enough kernel and filesystem operations and development to have a sense of how it should hang together, and to understand existing code to see how to connect everything up, so to succeed and learn some new things felt like a great achievement.