Not as much time to tinker this week, but I still had some fun and learned some things!
How to run a memory test using memtest86+
My error message is back, which means my nuke and pave approach didn’t solve the problem. So, yay to having a record of the error message?
Here’s the error:
echo 0 > / proc/sys/kernel/hung_task_timeout_secs" disables this message.
INFO: task txg_sync: 3557 blocked for more than 241 seconds.
Tainted: P 0 5.15.0-94-generic #104-Ubuntu
I decided to do a little more searching and found that the txg_sync is a zfs task. I know zfs uses a lot of RAM as part of it’s processing. As a result/starting point, I decided to do a memory test to see if I messed up any of my RAM modules when I knocked the shit out of my server.
Running a memory test was really easy. I downloaded the latest memtest86+ ISO, used balena etcher to flash it to a usb stick, booted from that stick, and let the test run.
I let it run for two full passes and got no errors.
So as of right now I know that the error is being caused by zfs writes/activity, but I don’t know why the error is happening, other than that I fucked something up when I knocked the shit out of my server.
How to set up a wireguard tunnel
This also has been on my list of things to figure out for quite awhile, and, turns out, with the wg-easy project, it is exactly as easy as the name implies. I found out about wg-easy through the Awesome Open Source YouTube channel. I’ve learned a lot from the guy that runs that channel, so I always check there when I want to learn more about something. Timing was fortuitous, since he had just dropped his video on wg-easy.
I’ve got wg-easy running on my vps, and I’m planning to connect my playground server to it so I can ssh in and play around during my breaks at work.
Grav CMS is pretty nice
I’ve kind of idly been looking for an alternative to Wordpress. I found out about Grav while bouncing around Linux YouTube looking for things to learn about/try. I’ve already tried 11ty and Hugo ssgs and neither worked for me.
Grav on the other hand was easy to get up and running, is easy to theme, (Theming was the problem I kept running into for both 11ty and Hugo), can be managed through cli or webui, and can have content added through the webui, or, more importantly for me, from markdown files on the server.
Whether or not I’ll actually use it to deploy a site remains to be seen. I’ll continue to tinker with it while I decide if I want to migrate my wordpress site over to it.
What module(s) is tainting your kernel, and is that what’s causing the txg_sync to hang?
After some reading and learning, I’ve gotten this output, which appears to indicate that the tainted modules are all zfs modules.
zfs 4112384 8 - Live 0x0000000000000000 (PO) zunicode 348160 1 zfs, Live 0x0000000000000000 (PO) zzstd 491520 1 zfs, Live 0x0000000000000000 (O) zlua 163840 1 zfs, Live 0x0000000000000000 (O) zavl 16384 1 zfs, Live 0x0000000000000000 (PO) icp 311296 1 zfs, Live 0x0000000000000000 (PO) zcommon 106496 2 zfs,icp, Live 0x0000000000000000 (PO) znvpair 98304 2 zfs,zcommon, Live 0x0000000000000000 (PO) spl 118784 6 zfs,zzstd,zavl,icp,zcommon,znvpair, Live 0x0000000000000000 (O)I’m still reading, and honestly, not sure what to do with this information at this point.
Turns out this is normal. OpenZFS is released under a different license that is not compatible with the GPL. As a result, ZFS modules are always considered out of tree/proprietary and will taint the kernel.
It also turns out that having a tainted kernel isn’t always a bad thing or a problem. Essentially using any proprietary software that loads kernel modules (Broadcom or Nvidia for example) means you’ve got a tainted kernel.
So while
txg_synchanging is also coincidentally an OpenZFS process, the ZFS modules tainting the kernel are not causing the hang, andtxg_synchanging is not the cause of the kernel being tainted.I don’t know, and to be honest, I also don’t know how I’d go about finding out. I’ll have to see if I can figure it out.