Best practices to access docker containers?

I am a noob and I have been searching all over for weeks. I can’t find the proper answers or I simply don’t get it, but either way I am completely stuck and would really appreciate some input from others who have experience.

Basically, I have a bunch of docker containers running on my NAS. Things like Nextcloud (calendar & tasks on phone via webdav), paperlessngx, mealie, etc. I set up a reverse proxy for each container (e.g. as explained here under step 6), as it seems that this is probably the easiest way for me and my family members to access those docker containers.

I would like to enable a VPN connection on my router, simply to make sure that all family members are always accessing the internet with a VPN. My understanding is that this is generally a good thing to do.
But when I enable the VPN on the router, then I can no longer access my docker containers via reverse proxy.

So basically my question is: What is a best practice to have a VPN for general internet access enabled and still being able to conveniently access all my docker containers while ideally not compromising on security?

I hope it makes sense what I am saying and would really appreciate some help as I am completely stuck and can’t find an answer to this. Considering I haven’t found a straight forward answer to this makes me wonder if I am going about this completely wrong. Which in this case I would also like to know and learn what setup I should be using.
Thanks in advance.

Edit: I’ve also come across the setup of running all your docker containers through a VPN. Is this the better option then? I am so confused on what’s the best practice here in terms of security and convenience.