From 189a5d4b8f987a8e3114beee39e50805812251c5 Mon Sep 17 00:00:00 2001 From: Jip-Hop <2871973+Jip-Hop@users.noreply.github.com> Date: Sat, 28 Jan 2023 23:09:54 +0100 Subject: [PATCH] Add documentation --- NETWORKING.md | 41 +++++++++++++++++++++ README.md | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 139 insertions(+), 1 deletion(-) create mode 100644 NETWORKING.md diff --git a/NETWORKING.md b/NETWORKING.md new file mode 100644 index 0000000..fd62af3 --- /dev/null +++ b/NETWORKING.md @@ -0,0 +1,41 @@ +# Jailmaker + +## Advanced Networking + +These are notes on advanced networking setup you may want to try. Contributions are welcome! + +### Bridge Networking + +As an alternative to the default host networking mode, you may want to connect to a bridge interface instead and let the jail obtain its IP address via DHCP. + +TODO: explain how to setup the bridge interface using the TrueNAS web interface. May lock yourself out... May take several tries... TrueNAS is a bit picky when switching IP addresses and toggling DHCP. May be helpful to connect a monitor and keyboard to the NAS and use `/etc/netcli` to reset the networking interface. Kept bothering with "Register Default Gateway" warning... I just clicked Cancel. + +Add the `--network-bridge=br1 --resolv-conf=bind-host` systemd-nspawn flag when asked for `Additional flags` during jail creation, or set it post-creation by [editing](./README.md#edit-jail-config) the `SYSTEMD_NSPAWN_USER_ARGS` variable inside the `config` file. + +The TrueNAS host and the jail will be able to communicate with each other as if the jail was just another device on the LAN. It will use the same DNS servers as the TrueNAS host because the `--resolv-conf=bind-host` option bind mounts the `/etc/resolv.conf` file from the host inside the jail. If you want to use the DNS servers advertised via DHCP, then check [DNS via DHCP](#dns-via-dhcp). + +### Macvlan Networking + +To setup Macvlan Networking you may follow the [Bridge Networking](#bridge-networking) section, but skip the setup of a bridge interface and use these flags instead: `--network-macvlan=eno1 --resolv-conf=bind-host`. By default the TrueNAS host and jail will not be able to communicate with each other via the network if Macvlan Networking mode is used. If that's required it would be better to use [Bridge Networking](#bridge-networking). + +### DNS via DHCP + +If you're not using host networking, and you're not using the `--resolv-conf=` in case of bridge/macvlan networking, then you have to configure the DNS servers to use. + +To get DNS servers via DHCP install and enable `resolvconf`. + +```shell +# Only run this inside the jail! + +# Temporarily fix DNS resolution, +# otherwise we can't install packages +echo 'nameserver 8.8.8.8' > /etc/resolv.conf +# On debian based distro +apt update && apt -y install resolvconf +``` + +## References + +- [systemd-nspawn](https://manpages.debian.org/bullseye/systemd-container/systemd-nspawn.1.en.html)- [Setting up Systemd-nspawn](https://www.cocode.se/linux/systemd_nspawn.html#orge360318) +- [Debian Reference - Chapter 5. Network setup](https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution) +- [Disabling link-local addressing](https://jerrington.me/posts/2017-08-06-systemd-nspawn-disabling-link-local-addressing.html#disabling-link-local-addressing) \ No newline at end of file diff --git a/README.md b/README.md index 092c723..f0018ef 100644 --- a/README.md +++ b/README.md @@ -1 +1,98 @@ -# jailmaker \ No newline at end of file +# Jailmaker + +Persistent Linux 'jails' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts. Without modifying the host OS at all thanks to systemd-nspawn! + +## Disclaimer + +**USING THIS SCRIPT IS AT YOUR OWN RISK! IT COMES WITHOUT WARRANTY AND IS NOT SUPPORTED BY IXSYSTEMS.** + +The systemd-container package may be removed from a future release of TrueNAS SCALE without warning. If that happens, you may be unable to start jails create with `jlmkr.sh`. The jail itself and the files within it will not be lost, but in order to start your jail you'd have to reinstall systemd-container, roll back to the previous release or migrate to LXC [if iXsystems includes it](https://ixsystems.atlassian.net/browse/NAS-114193?focusedCommentId=175214). Since systemd-container comes by default with Debian on which SCALE is built, I don't think it will be removed. But there's no guarantee! + +## Summary + +TrueNAS SCALE already has everything onboard to create persistent Linux 'jails' with systemd-nspawn. This script helps with the following: + +- Setting up the jail so it won't be lost when you update SCALE +- Choosing a distro (Debian 11 strongly recommended, but Ubuntu, Arch Linux or Rocky Linux seem good choices too) +- Optional: configuring the jail so you can run Docker inside it +- Optional: GPU passthrough (including nvidia GPU with the drivers bind mounted from the host) +- Starting the jail with your config applied + +## Installation + +Create a new dataset called `jailmaker` with the default settings (from TrueNAS web interface). Then login as the root user and download `jlmkr.sh`. + +```shell +cd /mnt/mypool/jailmaker +curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.sh +chmod +x jlmkr.sh +``` + +The `jlmkr.sh` script (and the jails + config it creates) are now stored on the `jailmaker` dataset and will survive updates of TrueNAS SCALE. + +## Create Jail + +Creating a jail is interactive. You'll be presented with questions which guide you through the process. + +```shell +./jlmkr.sh create myjail +``` + +## Start Jail + +```shell +./jlmkr.sh start myjail +``` + +### Autostart Jail on Boot + +In order to start a jail automatically after TrueNAS boots, run `/mnt/mypool/jailmaker/jlmkr.sh start myjail` as Post Init Script with Type `Command` from the TrueNAS web interface. + +## Additional Commands + +For additional commands we can use `machinectl`, `systemctl` and `journalctl` directly. The `jlmkr.sh` script does not play a role here. + +### Stop Jail + +```shell +machinectl stop myjail +``` + +### Jail Shell + +```shell +machinectl shell myjail +``` + +### Jail Status + +```shell +systemctl status jlmkr-myjail +``` + +### Jail Logs + +```shell +journalctl -u jlmkr-myjail +``` + +## Edit Jail Config + +Once you've created a jail, it will exist in a directory inside the `jails` dir next to `jlmkr.sh`. For example `./jails/myjail` if you've named your jail `myjail`. You may edit the jail configuration file. You'll have to stop the jail and start it again with `jlmkr.sh` for these changes to take effect. + +## Networking + +By default the jail will have full access to the host network. No further setup is required. You may download and install additional packages inside the jail. Note that some ports are already occupied by TrueNAS SCALE (e.g. 443 for the web interface), so your jail can't listen on these ports. This is inconvenient if you want to host some services (e.g. traefik) inside the jail. To workaround this issue when using host networking, you may disable DHCP and add several static IP addresses (Aliases) through the TrueNAS web interface. If you setup the TrueNAS web interface to only listen on one of these IP addresses, the ports on the remaining IP addresses remain available for the jail to listen on. + +See [Advanced Networking](./NETWORKING.md) for more. + +## Comparison + +TODO: write comparison between systemd-nspawn (without jailmaker), LXC, VMs, docker (on the host). + +## References + +- [systemd-nspawn](https://manpages.debian.org/bullseye/systemd-container/systemd-nspawn.1.en.html) +- [machinectl](https://manpages.debian.org/bullseye/systemd-container/machinectl.1.en.html) +- [systemd-run](https://manpages.debian.org/bullseye/systemd/systemd-run.1.en.html) +- [Run docker in systemd-nspawn](https://wiki.archlinux.org/title/systemd-nspawn#Run_docker_in_systemd-nspawn) \ No newline at end of file