fuchsiaaaaaaaaaaaaaaaaa
woooo we're back online
after long debugging in the cold next to my windy front door; with emotional support from my blahaj and technical support from IRC #debian my homeserver (and pixie.town workhorse) is all working again after a system malfunction during an upgrade
#PostMortem:
during a system upgrade wireguard suddenly cut out (not sure if that was a homeserver or endpoint-vps error). Unable to reconnect, I asked someone at home to force-reboot, which resulted in a corrupt kernel.
that would've been easy to fix from a live-usb, which I tried, except the cryptsetup initramfs part complained with stuff being mounted wrong in the chroot. Those errors were hidden behind a flood of useless error messages (known bug), so I did not notice and tried various things
in the end mounting the disks in the liveusb as expected, then regenerating the initramfs fixed it
🖲️
@f0x I want to set up full disk encryption on my server, the main reason I haven't so far is because I want to make sure I can reboot remotely and decrypt it without issues. I haven't had to recover from a chroot in a long time, since the new version of proxmox includes ZFS right in the kernel, kernel updates can't break ZFS support anymore.
@f0x I set up dropbear on my desktop a while ago, and I can SSH into it, but for some reason it doesn't decrypt properly if I put the password in over SSH. I need to try setting that up again.
A remote KVM switch would be very handy. I found this guide on setting something like that up, but it seems a bit complicated: https://github.com/Fmstrat/diy-ipmi
@theonefreeman I used https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/ as a base I think, dropbear works pretty well as long as your system boots :P
currently looking into (DIY) IPMI/KVM controllers but shit's expensive or complicated