Before you go too far into it and spend lots of time, I think most VPS services let you installed a new OS on their admin site so you can start again from scratch. If you’re not sure that is the right linux flavour, go for something else more mainstream so you can find lots of support online. Looking at the OS, I’m sure it might be good but I’m also sure you can install all the features very easily yourself, especially if it’s just using docker mainly.
I second UFW. I found this guide useful: https://www.digitalocean.com/community/tutorials/ufw-essentials-common-firewall-rules-and-commands. You might want to try tailscale as others use it for easily setting up vpn access but not used it myself. Also go for fail2ban or, for more assurance but harder work, try crowdsec too.
You could also use cloudflare dns and add IP and/or country restrictions to block all traffic before it gets to your VPS. I have a country filter and it’s crazy how many bots get blocked from all over!
What difficulties are you finding with it and are you switching from Google? The results are as custom as Google given they haven’t scraped your life history so wondering if that’s it? I’ve been using DDG without any issues. About once every 6 months I struggle to find something so try the Google bang but have never found better results. In fact, I was shocked last time how crap the Google results were, just full of AI generated crap and SEO based crap.
To be honest, DDG is also struggling with that now as it’s based on Bing. I have been trying a public searxg but not found it very good so far.