From 679bc6e3407f26431cb04691f315dda123d3df31 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:23:18 +1100 Subject: [PATCH 1/9] Create wikimain.md --- docs/wikimain.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/wikimain.md diff --git a/docs/wikimain.md b/docs/wikimain.md new file mode 100644 index 0000000..e556b83 --- /dev/null +++ b/docs/wikimain.md @@ -0,0 +1 @@ +w From 7bf3e04fd167ac272fb1d4472945340f4c21b5b7 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:23:52 +1100 Subject: [PATCH 2/9] Update and rename wikimain.md to .md --- docs/.md | 3 +++ docs/wikimain.md | 1 - 2 files changed, 3 insertions(+), 1 deletion(-) create mode 100644 docs/.md delete mode 100644 docs/wikimain.md diff --git a/docs/.md b/docs/.md new file mode 100644 index 0000000..0c8d31a --- /dev/null +++ b/docs/.md @@ -0,0 +1,3 @@ +Welcome to the jailmaker wiki! + +Use the sidebar to navigate the topics. diff --git a/docs/wikimain.md b/docs/wikimain.md deleted file mode 100644 index e556b83..0000000 --- a/docs/wikimain.md +++ /dev/null @@ -1 +0,0 @@ -w From 3944f365479ef1f34f49a2c0c1c1e363c8b775ae Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:24:11 +1100 Subject: [PATCH 3/9] Rename .md to wikimain.md --- docs/{.md => wikimain.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/{.md => wikimain.md} (100%) diff --git a/docs/.md b/docs/wikimain.md similarity index 100% rename from docs/.md rename to docs/wikimain.md From b4735ced6105d79af4e248adc1fb093c51b88557 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:24:43 +1100 Subject: [PATCH 4/9] Create basicconfig.md --- docs/basicconfig.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 docs/basicconfig.md diff --git a/docs/basicconfig.md b/docs/basicconfig.md new file mode 100644 index 0000000..e14435e --- /dev/null +++ b/docs/basicconfig.md @@ -0,0 +1,37 @@ +# User Management +The root user (also known as the superuser or su) can access any file, make system changes, and lots of room for security vulnerabilities. +For this reason you should aspire to run services as a non-root user. + +### Create a non-root user +`useradd USERNAME` + +Where username can be anything, but should reflect the service/jail's name for diagnostic. + +Then a password should be created as some commands require a non-blank password to be inserted: +`passwd USERNAME` + +If you want the ability to run commands as root, add the user to the sudo group +`usermod -aG sudo USERNAME` + +This WILL require a non-blank password, and any command run with sudo will be run as root not as the user. But it saves time compared to switching users to root to install/change things then switching back. + +### Switch to user +`su -l USERNAME` + +### Put a password on Root +While logged in as root run `passwd` + +# Common tweaks +### Update repository list +`sudo apt update` + +### Install common services +`sudo apt install nano wget curl git` + +### Set Static IP +See `Networking` + +### Install Docker +``` +apt install curl && cd /tmp && curl -fsSL https://get.docker.com -o get-docker.sh && sudo sh get-docker.sh && cd ~ && docker +``` From b81ecd7fedacaa37f70ae40eec8dfcc160f2775d Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:25:02 +1100 Subject: [PATCH 5/9] Create compatibility.md --- docs/compatibility.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 docs/compatibility.md diff --git a/docs/compatibility.md b/docs/compatibility.md new file mode 100644 index 0000000..b4ddb3a --- /dev/null +++ b/docs/compatibility.md @@ -0,0 +1,16 @@ +# TrueNAS Compatibility +TrueNAS Core ❌ +TrueNAS 22.12 ✅ +TrueNAS 23.10-BETA1 ✅ +TrueNAS 23.10-RC1 ✅ + +# Distro Compatibility +Debian 11 Bullseye ✅ +Debian 12 Bookworm ✅ +Arch 🟨 +Ubuntu 🟨 +Alpine ❌ + +✅ = Personally tested and working +🟨 = Haven't personally tested +❌ = Reported not working From 169329430855002d8393a91db6039e7296bb0315 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:25:19 +1100 Subject: [PATCH 6/9] Create commands.md --- docs/commands.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 docs/commands.md diff --git a/docs/commands.md b/docs/commands.md new file mode 100644 index 0000000..2852032 --- /dev/null +++ b/docs/commands.md @@ -0,0 +1,35 @@ +Create a jail +`jlmkr create JAILNAME` + +Start a jail +`jlmkr start JAILNAME` + +Stop a jail +`jlmkr stop JAILNAME` + +Check jail status +`jlmkr status JAILNAME` + +Delete a jail and remove it's files (requires confirmation) +`jlmkr remove JAILNAME` + +See list of jails (including running, non running, distro, startup state, and IP) +`jlmkr list` + +See list of running jails +`machinectl list` + +Execute a command inside a jail from the TrueNAS shell +`jlmkr exec JAILNAME COMMAND` + +Execute a bash command inside a jail from the TrueNAS shell +`jlmkr exec JAILNAME bash -c 'BASHCOMMAND'` + +Switch into the jail's shell +`machinectl shell JAILNAME` + +View a jail's logs +`jlmkr log JAILNAME` + +Edit a jail's config +`jlmkr edit JAILNAME` From 798d8a37f5537f3ddc7311a92000fc18d26403c8 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:25:45 +1100 Subject: [PATCH 7/9] Create network.md --- docs/network.md | 57 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 docs/network.md diff --git a/docs/network.md b/docs/network.md new file mode 100644 index 0000000..92a6d44 --- /dev/null +++ b/docs/network.md @@ -0,0 +1,57 @@ +# Host Passthrough (Default network configuration) +By default jails will use the same physical interface as the TrueNAS host. If a service attempts to bind to port 80 or 443, it will either fail or render both the service and TrueNAS unavailable. +### Flaws +Depending on the service this may be ok, for example Home Assistant will bind to port 8123, leaving the 80 and 443 ports free from clashes for the TrueNAS web interface. You can then either connect to the service with the port, or use a reverse proxy such as [nginx](https://www.nginx.com/#). +### Setup +No configuration is necessary + +# MAC VLAN Virtual Interface +Some services require the use of port 80 or 443, or would benefit from a separate IP. For these situations the easiest network configuration is the MAC VLAN configuration. This creates a virtual interface with its own separate randomly generated MAC address and IP. +The default config uses DHCP by default, but can easily be set to a Static IP. +### Flaws +Any services in the jail cannot communicate with the direct host (TrueNAS). The jail can communicate with any other jail or device on the network, besides TrueNAS. This may or not be a benefit (security) or disadvantage (no communication) depending on your service. +### Setup +Add the following argument to the "additional flags" prompt of jail creation or the "systemd_nspawn_user_arguments" line of the jail config file: +``` +--network-macvlan=eno1 --resolv-conf=bind-host +``` + +### Setting a Static IP +To set a Static IP you need to disable DHCP in the macvlan config file `/etc/systemd/network/mv-dhcp.network` +You can do this with a network client like WinSCP by navigating into the jail's filesystem then the path above, or by using a text editing program like nano by running `nano /etc/systemd/network/mv-dhcp.network' in the jail's shell. + +The DHCP in [Network] needs to be set to false, an Address (static IP) needs to be added, a Gateway needs to be defined (e.g your router such as 192.168.0.1) and the entire DHCP section needs to be removed. + +An example static IP configuration is as follows: +``` +[Match] +Virtualization=container +Name=mv-* + +[Network] +DHCP=false +Address=192.168.X.XXX/24 +Gateway=192.168.X.X +``` +Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `jlmkr stop JAILNAME && jlmkr start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP. + +# Passthrough a TrueNAS Bridge Interface +By creating a network bridge in the TrueNAS Network page you can bridge the incoming physical network interface to a virtual interface that can be passed to the jail. This type of interface has the benefits of a MAC VLAN interface without the flaws (host to jail networking). Once working the virtual interface can either be assigned a static IP or obtain one automatically via DHCP. +### Flaws +This type of interface takes much longer to set up both in complexity and wait time as there is a current flaw in which HDCP can take between 10 seconds and a minute. +Furthermore, if the configuration is not correct it can render your TrueNAS inaccessible via ssh, necessitating a reset using a keyboard and monitor plugged into the TrueNAS server. +### Setup +[TrueNAS Bridge interface guide](https://www.youtube.com/watch?v=7clQw132w58) +May be helpful to connect a monitor and keyboard to the NAS and use /etc/netcli to reset the networking interface. Kept bothering with "Register Default Gateway" warning... I just clicked Cancel. + +Add the `--network-bridge=br1 --resolv-conf=bind-host systemd-nspawn` flag when asked for additional flags during jail creation, or set it post-creation by editing the `SYSTEMD_NSPAWN_USER_ARGS` variable inside the config file. + +### Static IP +To configure a static IP with our bridge interface, we need to edit the `/etc/systemd/network/80-container-host0.network` file. Change the [Network] section to look like this: +``` +[Network] +DHCP=false +Address=192.168.X.XXX/24 +Gateway=192.168.X.X +``` +Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `jlmkr stop JAILNAME && jlmkr start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP. From 858f45514c2e625dde3a3e1286faa3fc926a6f52 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:26:47 +1100 Subject: [PATCH 8/9] Create performance.md --- docs/performance.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 docs/performance.md diff --git a/docs/performance.md b/docs/performance.md new file mode 100644 index 0000000..92c1e3a --- /dev/null +++ b/docs/performance.md @@ -0,0 +1,21 @@ +(Anecdotal from observations, actual measurements with resource monitor captures and wall power meter coming soon.) + +Kubernetes Server (TrueNAS Apps) with no apps installed: +* Idle on 7100T: ~20% / 15W +* Idle on 10600K: ~10% + +Kubernetes Server (TrueNAS Apps) with 10 apps installed: +* Idle on 7100T: ~26% / 18W +* Idle on 10600K: ~15% + +Systemd-nspawn container (jailmaker) with no apps installed: +* Idle on 7100T: ~1% / 6W +* Idle on 10600K: ~0% + +Systemd-nspawn container (jailmaker) with 10 apps installed: +* Idle on 7100T: ~4% / 8W +* Idle on 10600K: ~0% + + +Systemd-nspawn container (jailmaker) with 20 apps installed: +* Idle on 10600K: ~1% From 401700c29131371d65b5a5366aa03a0fd59ecc27 Mon Sep 17 00:00:00 2001 From: spusuf <86286368+spusuf@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:27:11 +1100 Subject: [PATCH 9/9] Create storage.md --- docs/storage.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 docs/storage.md diff --git a/docs/storage.md b/docs/storage.md new file mode 100644 index 0000000..b88754f --- /dev/null +++ b/docs/storage.md @@ -0,0 +1,28 @@ +# Default storage system +When creating a jail, an entire Linux filesystem is created in the 'rootfs' folder within the jail's folder of the jailmaker directory E.g `/mnt/tank/vault/jailmaker/jails/jailname/rootfs`. No files from the TrueNAS host will be available. + +Common locations for services are: +`/home` for user accessible files +`/var/www/` for webpages +`/tmp` for temporary application data such as build files + +# Linking folders to TrueNAS folders +To allow file access by either the jail, another jail, or TrueNAS a bind can be made. A bind creates a link between two locations. Think of this as a portal, anything that goes in one side is visible from the other side and vice versa. + +Note that creating a file in the jail or TrueNAS will reflect in both binded locations, so be careful of overwrites and corruption. + +### Setup +Add the following to your user arguments during setup or into the jail's config file, with your two linked locations separated by a colon: +``` +--bind='/host/path/to/:/jail/path/to' +``` +Where `/host/path/to/` is the folder on the TrueNAS filesystem you want shared. +And where `/jail/path/to/` is the folder you want those shared files accessible by the jail. + +### Example +A use of this is making files available in a jail for it to use or serve, such as media files in Plex/Jellyfin: +Example: `--bind='/mnt/tank/content/:/media'` will make any files inside the content dataset of the tank pool available inside the jail's /media folder. To visualise or test this you can copy some files to `/mnt/tank/content/` such as `media1.mp4`, `media2.mkv` and `photo.jpg`. Then change directory to that folder inside the jail `cd /media` and list files in that directory `ls -l` where those files should appear. + +### Warning +Do not bind your TrueNAS system directories (`/root` `/mnt` `/dev` `/bin` `/etc` `/home` `/var` `/usr` or anything else in the root directory) to your jail as this can cause TrueNAS to lose permissions and render your TrueNAS system unusable. +Best practice is to create a dataset in a pool which also allows zfs, raidz, permissions, and backups to function. E.g creating a `websites` dataset in a pool named `tank` then binding `--bind='/mnt/tank/websites/websitename/:/var/www/websitename/'`