I use Authentik - works well - but I’m prepping to switch to Authelia for the config-based setup.

I updated Authentik across 2 versions (they did 2 month-long supported tags, missed one; now they do 3 month-long supported tags) and it destroyed itself. Had to recover the DB from a backup (and then step through the version I tried to skip), and while I was doing that I was like “wait why does this have only a DB? Should just be a config file cause that’s all the depth I do with it” and lo that’s what Authelia is.

Authentik is audited and Authelia has not been. Initially while I chose Authentik. But config-file robustness in the face of Authentik’s GUI-setup DB imploding swayed me not to care, it’d take so long and be so tedious to redo all my proxies and auths through the GUI. Plus I always forget what to click in the GUI when I come back to add some new program in 4 months.

I added examples so you don’t have to dig as far if you want to give it a try!

I approach it the same as I did with Docker, one user per container.

I started with rootless but networking within Podman is moot with multiple users per container. And one user for all containers to get networking has to lead to subUID clashes (and thus escape vectors) - unless someone can explain how not…

But root Podman is just as secure anyway, and easier, so I just roll with UserNS=auto and use idmap= on the volumes to enable writing as the specified user for the container. And networking in Podman works because it’s one user space. By default UserNS=auto gives 1024 subUIDs to a container. I had to up that to the expected max of 65535 for Frigate to work (example bit I use with idmap, UserNS with the size and group mapping, and GroupAdd):

Volume=/mnt/camraz:/media/frigate:rw,noexec,nosuid,nodev,Z,idmap=uids=@1111-0-1;gids=@1111-0-1#1020-1020-1  
UserNS=auto:size=65535,uidmapping=105:@105:1,gidmapping=105:@105:1  
GroupAdd=105  

So what’s going on here is on the Volume= I have the camera folder mounted to where Frigate wants it, and I specify that it accesses that volume with idmap= and frigate’s UID/GID 1111 and the camera group I made at GID 1020.

The order for the idmap is @${hostUID}-${containerUID}-${num2map}. The @ symbol means that 1111 is an absolute host UID; drop that and 1111 is pulled from the containers subUID/subGID list. Most containers run as root (0) so you just map to that user.

Side quest: For containers that have one of those mini-hypervisors like LinuxServers images and their “S6” thing, I worked around them starting with a root user but then requiring moving to a non-root user by specifying PUID=1 and PGID=1 then setting idmap=uids=@1999-0-2;gids=@1999-0-2 and reserving 1999 as username containerNameStartup and 2000 as containerName. This idmap= maps host 1999 to container 0 but maps two UID/GIDs consecutively, so it also maps host 2000 to container 1 keeping everything lined up perfectly - while LinuxServer images still don’t get host root like they so desperately want.

GroupAdd=105 adds in the group 105 to the container, which is notably not the host group 105. UserNS is needed to complete that. This just makes that group exist in the container.

UserNS=auto:size=65535,gidmapping=105:@105:1 accomplishes the subUID/subGID size allocation to the expected default max of 65535 (because Frigate needs it) and the group connection from host 105 to container 105. Note that this syntax helpfully uses the opposite order as idmap=. It goes ${containerUID}:@${hostUID}:${num2map} and uses colons instead of dashes.

Every other container is cool with the default 1024 (just UserNS=auto). The subUIDs are pulled from a non-existent user named containers that you need to enable for Podman root to work with UserNS, and it has like 2 million billion or something with their recommended setup, so it’s good on subUIDs/subGIDs.

Command to set up root Podman with UserNS=auto:

sudo echo "containers:2147483647:2147483648" >> /etc/subuid  
sudo echo "containers:2147483647:2147483648" >> /etc/subgid  

You can also have UserNS=auto and run the container as a specific host user directly instead of “nobody” with:

UserNS=auto:uidmapping=0:@1111:1,gidmapping=0:@1111:1

Same idea as how I mapped in the host group 105 above. Where the root user 0 is mapped to the host (via @) user 1111 so the container runs as the user. I don’t use that because the idmap= works perfectly and then the container doesn’t even run as the user that controls the files directly. But maybe there will be a use for that sometime(?).

And I do have a fuckton of users; Debian once complained it ran out of numbers or something after like 20 users, so I just ran the first thing I found to make the UID limit some really big number, and I never thought about it again! No idea what it was…

**Edit: ** I cleaned it up and added examples

Also I mostly work with UIDs not users per-say, so I specify UID 1111 has access to Frigate files. I just make a user named frigate to make it a bit easier for me. When you do ls -l it says frigate instead of 1111; life is easier.

Overall, it has been a lot of work to divine these things because the documentation and useful examples for Podman are confusing and non-existent. I think I’ve gotten most of this idmap= and UserNS= stuff working from reading GitHub issues that drop little tidbits. But after I get it working as I want it, it’s solid, and I can now rinse-and-repeat in the future. No Docker daemon to fuck with or get hacked, and it’s reliable since I just let systemd start these quadlets up.

I’m not 100% done, but I’m close. Outstanding issue right now is that on server restart sometimes some Podman networks don’t actually start up correctly and then containers that want them fail to start because of that (I have to restart the Podman network via systemctl to get them to exist in Podman). But eventually I’ll find the stupid GitHub issue where someone mentions something relevant for that, and then I’ll be golden!

Thanks for the write up, again! I def gotta up my NetworkManager game…

This is great, I’d love to see the config files - especially the WiFi pass through stuff. No experience with that! But the rest all fitting together would be rad as hell.

Peeps would be able to get the 4GB banana zero and just rock it up quick full service!

I know that’s a big ask, sharing your cursed config files is work… if you feel up for it tho - hell yes

I have this setup. Upfront, I would not recommend Proxmox, the update methods are annoying. The better way is straight Debian with Incus installed, then you get straightforward stable Debian updates automatically - they won’t break anything and you’re secure. Sometime I’ll redo it - I haven’t because, of course, it is my router and when its down I don’t have internet! So foreboding and on the back burner.

Also also Proxmox’s GUI leaves a lot to be desired (for me, it looks like ass and is confusing), Incus is nicer for VM control and Cockpit is nicer for host control. After typing all that I realize I’m a hater at this point

I haven’t really noticed downtime issues cause of Proxmox updates cause I just do it when nothing is happening. And Proxmox hasn’t bricked itself, though I am wary of it because that has happened to others due to their rolling release update style.

I’ve got a Dell Wyse 5070 Extended with a 2 port Intel NIC in it. I pass both ports through leaving the built-in port for managing Proxmox.

Here are my notes:

Set NIC PCIe Passthrough for Network Card

nano /etc/default/grub

• Edit this line by adding intel_iommu=on to get

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

update-grub

nano /etc/modules

• Add these lines

vfio  
vfio_iommu_type1  
vfio_pci  
vfio_virqfd  

update-initramfs -u -k all

reboot

Click on 2nd level thing named router on the left side vertical bar hierarchy thing and then click in the top right the blue Create VM button.

• General tab
  • Name: OPNsense
  • Start at boot: checked
  • Start/Shutdown order: 1
  • Startup delay: 15
• OS tab
  • Use media: DVD version (usb might work) of OPNsense.iso
• System tab
  • Machine: q35
  • Bios: OVMF (UEFI)
    • Storage: local-lvm
    • UNCHECK Pre-enroll Keys (HATE)
• Hard Disk tab
  • Disk size (GiB): 15
  • Discard: checked
  • SSD emulation: checked
• CPU tab
  • Cores: 4
  • Type: host {makes it not moveable between diff CPU types but will theoretically allow for more speed}
• Memory tab
  • Memory (MiB): 2048
  • Minimum memory (MiB): 512
• Network tab
  • No network device: checked
• Confirm tab
  • Do not start on creation
• After creation, go to Hardware tab in the 2nd left vertical list on the browser page and click add
• Click PCI Device
  • Device: ...01:00.0 I350 Gigabit... & ...01:00.1 I350 Gigabit... (1st & 2nd ones)
  • PCI-Express: checked

Go to the Console tab in the 2nd left vertical list on the browser page and hit enter to get to a command line in the OPNsense VM

!Add expand storage via command line!

And lastly, during setup I have these notes

It will choose wrong (WAN gets igb1 and LAN gets igb0 -> we want WAN gets igb0 and LAN gets igb1)  
Default User: root, PW: opnsense (they don't tell you anywhere, you don't have internet b/c this is your new router, fuck em)  
**Access at 192.168.1.1 via pluging an ethernet cable into the 1st port in a set of forwarded ports**  
*Note that we will move it so the 1st port is the WAN (can't access OPNsense from the WAN port for safety), so after following this you access via 2nd port*  

So watch out for those things. Not sure quite what I mean by the 1st and 2nd port things, may be related to on setup it had the order of the ports I wanted wrong so they’re switched till setup is complete and it reboots.

I don’t remember doing this at this point, but maybe this info dump will help!

Pangolin has a lot more going on than I expected, I thought it was just a mesh system. That might be a one-stop shop, thanks for sharing!!

I do see Authentik can apparently act as a reverse proxy, so it can sit at the very front. But I’d lose out on caddy + crowdsec then…

I’ll have to do some reading if caddy can actually just forward to caddy, and where the TLS is terminated and all that.

And I do use the internal setting now, but I need it off if I want to publish the port on the LAN so that the VM can see the ports on the LAN. But if I can have WAN caddy do the auth check and forward along good stuff to the LAN caddy, then I’ll only need to publish LAN caddy’s port and that’s not the worst at all.

Thanks for the ideas, I’ll try to cook “caddy (DMZ) -> auth OIDC (DMZ) -> caddy (LAN) -> services (LAN)”!

Rootful to get the cross-user networking
Edit: cross-user via running the containers with userNS & user or volume idmap from root Podman. Containers can be run by different users and share networks, but the containers don’t have root access

Containers lower the bar since the developer doesn’t need to make their program work on every system - just the container’s system.

Price we pay for more programs. And they bring boons like read-only, rootless, limited capabilities, and constrained perf limits (esp if you use Podman with Quadlets).

And don’t feel trapped - the Dockerfile is a recipe to build that program. Probably want to do it in an LXC container since it’ll want to use /data for its data or something. But the LXC container can also be run as a user but the program thinks it’s root. Plenty of security abounds!

I think it’s worth the price and you’re not trapped. They’re trapped with you and your robust Quadlet files