Persistent Linux 'jails' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts.
USING THIS SCRIPT IS AT YOUR OWN RISK! IT COMES WITHOUT WARRANTY AND IS NOT SUPPORTED BY IXSYSTEMS.
TrueNAS SCALE can create persistent Linux 'jails' with systemd-nspawn. This script helps with the following:
- Installing the systemd-container package (which includes systemd-nspawn)
- Setting up the jail so it won't be lost when you update SCALE
- Choosing a distro (Debian 12 strongly recommended, but Ubuntu, Arch Linux or Rocky Linux seem good choices too)
- Optional: configuring the jail so you can run Docker inside it
- Optional: GPU passthrough (including nvidia GPU with the drivers bind mounted from the host)
- Starting the jail with your config applied
Create a new dataset called
jailmaker with the default settings (from TrueNAS web interface). Then login as the root user and download
cd /mnt/mypool/jailmaker curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.py chmod +x jlmkr.py ./jlmkr.py install
jlmkr.py script (and the jails + config it creates) are now stored on the
jailmaker dataset and will survive updates of TrueNAS SCALE. Additionally a symlink has been created so you can call
jlmkr from anywhere.
After an update of TrueNAS SCALE the symlink will be lost and
systemd-nspawn (the core package which makes
jailmaker work) may be gone too. Not to worry, just run
./jlmkr.py install again or use the
./jlmkr.py startup command.
Creating a jail is interactive. You'll be presented with questions which guide you through the process.
jlmkr create myjail
After answering a few questions you should have your first jail up and running!
Startup Jails on Boot
# Best to call startup directly (not through the jlmkr symlink) /mnt/mypool/jailmaker/jlmkr.py startup # Can be called from the symlink too... # But this may not be available after a TrueNAS SCALE update jlmkr startup
In order to start jails automatically after TrueNAS boots, run
/mnt/mypool/jailmaker/jlmkr.py startup as Post Init Script with Type
Command from the TrueNAS web interface. This will automatically fix the installation of
systemd-nspawn and setup the
jlmkr symlink, as well as start all the jails with
startup=1 in the config file. Running the
startup command Post Init is recommended to keep
jailmaker working after a TrueNAS SCALE update.
jlmkr start myjail
Execute Command in Jail
You may want to execute a command inside a jail, for example from a shell script or a CRON job. The example below executes the
env command inside the jail.
jlmkr exec myjail env
This example executes bash inside the jail with a command as additional argument.
jlmkr exec myjail bash -c 'echo test; echo $RANDOM;'
Edit Jail Config
jlmkr edit myjail
Once you've created a jail, it will exist in a directory inside the
jails dir next to
jlmkr.py. For example
/mnt/mypool/jailmaker/jails/myjail if you've named your jail
myjail. You may edit the jail configuration file, e.g. using the
jlmkr edit myjail command (which uses the nano text editor). You'll have to stop the jail and start it again with
jlmkr for these changes to take effect.
jlmkr remove myjail
jlmkr stop myjail
jlmkr shell myjail
jlmkr status myjail
jlmkr log myjail
Expert users may use the following additional commands to manage jails directly:
jlmkr script uses these commands under the hood and implements a subset of their capabilities. If you use them directly you will bypass any safety checks or configuration done by
jlmkr and not everything will work in the context of TrueNAS SCALE.
By default the jail will have full access to the host network. No further setup is required. You may download and install additional packages inside the jail. Note that some ports are already occupied by TrueNAS SCALE (e.g. 443 for the web interface), so your jail can't listen on these ports. This is inconvenient if you want to host some services (e.g. traefik) inside the jail. To workaround this issue when using host networking, you may disable DHCP and add several static IP addresses (Aliases) through the TrueNAS web interface. If you setup the TrueNAS web interface to only listen on one of these IP addresses, the ports on the remaining IP addresses remain available for the jail to listen on.
See Advanced Networking for more.
jailmaker script won't install Docker for you, but it can setup the jail with the capabilities required to run docker. You can manually install Docker inside the jail using the official installation guide or use convenience script.
To make passthrough of the nvidia GPU work, you need to schedule a Pre Init command. The reason is that TrueNAS SCALE by default doesn't load the nvidia kernel modules (and
jailmaker doesn't do that either). This screenshot shows what the configuration should look like.
[ ! -f /dev/nvidia-uvm ] && modprobe nvidia-current-uvm && /usr/bin/nvidia-modprobe -c0 -u
Additional documentation contributed by the community can be found in the docs directory.
TODO: write comparison between systemd-nspawn (without
jailmaker), LXC, VMs, Docker (on the host).
The rootfs image
jlmkr.py downloads comes from the Linux Containers Image server. These images are made for LXC. We can use them with systemd-nspawn too, although not all of them work properly. For example, the
alpine image doesn't work well. If you stick with common systemd based distros (Debian, Ubuntu, Arch Linux...) you should be fine.