TLDR: I learned the hard way that, at some point, my nightly backups were switched to being encrypted. As far as I can tell, it seems like it was part of an update and not a lot of users noticed it. Thankfully (and concerningly?), the key is stored in plain text if you can recover data from your SD card.
A storm came through my area yesterday and caused the power to drop multiple times. The first time, my HA Pi rebooted fine as it always does, but the second time it failed to boot because it couldn't mount the HassOS boot partition.
After work, I pulled the SD card and connected it to a linux VM on my laptop to see if I could recover the backups from it and just restore from a clean install because I didn't want to spend my night chasing down boot issues. The first thing I did was dd to dump the card to an image before I tried to mess with anything. I found the nightly backups, copied them over to my system, briefly tried to fix the boot issues, gave up, and then wiped and reinstalled Home Assistant OS with Raspberry Pi Imager.
Imagine my surprise when I go through the setup process, get to the "Restore from Backup" screen, and I am then prompted to enter the encryption key for my backup. My backup through Nabu Casa also conveniently had an error when I tried to load it, I assume that one is encrypted as well, but I didn't even get that far.
I have always been pretty good about copying encryption keys into my password manager when I get them, so I was really confused when that prompt came up. I tried my account password and a few others that I might have used if it was a key that I set manually, no luck. After some research, the keys are in the form of XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX, which I definitely did not have, and the SD card had already been wiped, so any hope of finding it died.
At that point, it was getting late and I was getting frustrated, so I threw in the towel and gave into doing a fresh Home Assistant, not looking forward to rebuilding my automations and all that fun stuff.
When I woke up this morning, I had the epiphany that I had dumped the SD card as my very first step, so I grabbed the dump, mounted it, did a few find and greps, and found my encryption key in a json file called <mount path>/supervisor/homeassistant/.storage/backup. I copied the key, pulled up the restore page on my blank Home Assistant, put everything in, and it immediately began the restore process and has been happy all day.
Lessons Learned:
- Save your encryption key and emergency kit at
<server>/config/backup/settings - Look into remote backups because next time the SD card might be totally dead and I had bad luck with my Nabu Casa backup, I am going to install this Google Drive add on as a simple start
- You can always trust that someone, somewhere will leave an encryption key in plain text