Update docs
This commit is contained in:
Jip-Hop
parent
07f405ef73
commit
fbe988137f
54
README.md
54
README.md
|
|
@ -21,29 +21,9 @@ TrueNAS SCALE can create persistent Linux 'jails' with systemd-nspawn. This scri
|
|||
- Optional: GPU passthrough (including nvidia GPU with the drivers bind mounted from the host)
|
||||
- Starting the jail with your config applied
|
||||
|
||||
## Security
|
||||
|
||||
Despite what the word 'jail' implies, jailmaker's intended use case is to create one or more additional filesystems to run alongside SCALE with minimal isolation. By default the root user in the jail with uid 0 is mapped to the host's uid 0. This has [obvious security implications](https://linuxcontainers.org/lxc/security/#privileged-containers). If this is not acceptable to you, you may lock down the jails by [limiting capabilities](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Security_Options) and/or using [user namespacing](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#User_Namespacing_Options) or use a VM instead.
|
||||
|
||||
### Seccomp
|
||||
Seccomp is a Linux kernel feature that restricts programs from making unauthorized system calls. This means that when seccomp is enabled there can be times where a process run inside a jail will be killed with the error "Operation not permitted." In order to find out which syscall needs to be added to the `--system-call-filter=` configuration you can use `strace`.
|
||||
|
||||
For example:
|
||||
```
|
||||
# /usr/bin/intel_gpu_top
|
||||
Failed to initialize PMU! (Operation not permitted)
|
||||
|
||||
# strace /usr/bin/intel_gpu_top 2>&1 |grep Operation\ not\ permitted
|
||||
perf_event_open({type=0x10 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER7, config=0x100002, sample_period=0, sample_type=0, read_format=PERF_FORMAT_TOTAL_TIME_ENABLED|PERF_FORMAT_GROUP, precise_ip=0 /* arbitrary skid */, use_clockid=1, ...}, -1, 0, -1, 0) = -1 EPERM (Operation not permitted)
|
||||
write(2, "Failed to initialize PMU! (Opera"..., 52Failed to initialize PMU! (Operation not permitted)
|
||||
```
|
||||
The syscall that needs to be added to the `--system-call-filter` option in the jlmkr config in this case would be `perf_event_open`. You may need to run strace multiple times.
|
||||
|
||||
Seccomp is important for security, but as a last resort can be disabled by setting `seccomp=0` in the jail config.
|
||||
|
||||
## Installation
|
||||
|
||||
Beginning with 24.04 (Dragonfish), TrueNAS SCALE includes the systemd-nspawn containerization program in the base system. Technically there's nothing to install. You only need the `jlmkr.py` script file in the right place. [Instructions with screenshots](https://www.truenas.com/docs/scale/scaletutorials/apps/sandboxes/) are provided on the TrueNAS website. Start by creating a new dataset called `jailmaker` with the default settings (from TrueNAS web interface). Then login as the root user and download `jlmkr.py`. If you login as non-root user (e.g. as admin), **you must become root first** by executing `sudo su`.
|
||||
Beginning with 24.04 (Dragonfish), TrueNAS SCALE officially includes the systemd-nspawn containerization program in the base system. Technically there's nothing to install. You only need the `jlmkr.py` script file in the right place. [Instructions with screenshots](https://www.truenas.com/docs/scale/scaletutorials/apps/sandboxes/) are provided on the TrueNAS website. Start by creating a new dataset called `jailmaker` with the default settings (from TrueNAS web interface). Then login as the root user and download `jlmkr.py`.
|
||||
|
||||
```shell
|
||||
cd /mnt/mypool/jailmaker
|
||||
|
|
@ -53,6 +33,16 @@ chmod +x jlmkr.py
|
|||
|
||||
The `jlmkr.py` script (and the jails + config it creates) are now stored on the `jailmaker` dataset and will survive updates of TrueNAS SCALE. If the automatically created `jails` directory is also a ZFS dataset (which is true for new users), then the `jlmkr.py` script will automatically create a new dataset for every jail created. This allows you to snapshot individual jails. For legacy users (where the `jails` directory is not a dataset) each jail will be stored in a plain directory.
|
||||
|
||||
### Alias
|
||||
|
||||
Optionally you may create a shell alias for the currently logged in (admin) user to conveniently run `jlmkr.py` without having to change into the `jailmaker` directory or specify the full absolute path. I suggest to create the `jlmkr` alias like this:
|
||||
|
||||
```shell
|
||||
echo "alias jlmkr=\"sudo '/mnt/mypool/jailmaker/jlmkr.py'\"" > ~/.bashrc
|
||||
```
|
||||
|
||||
Please replace `/mnt/mypool/jailmaker/` with the actual path to where you stored `jlmkr.py`. If you're using zsh instead of bash, then you should replace `.bashrc` in the command above with `.zshrc`. If you've created the alias, you may use it instead of `./jlmkr.py`.
|
||||
|
||||
## Usage
|
||||
|
||||
### Create Jail
|
||||
|
|
@ -174,6 +164,26 @@ View a jail's logs.
|
|||
|
||||
Expert users may use the following additional commands to manage jails directly: `machinectl`, `systemd-nspawn`, `systemd-run`, `systemctl` and `journalctl`. The `jlmkr` script uses these commands under the hood and implements a subset of their functions. If you use them directly you will bypass any safety checks or configuration done by `jlmkr` and not everything will work in the context of TrueNAS SCALE.
|
||||
|
||||
## Security
|
||||
|
||||
By default the root user in the jail with uid 0 is mapped to the host's uid 0. This has [obvious security implications](https://linuxcontainers.org/lxc/security/#privileged-containers). If this is not acceptable to you, you may lock down the jails by [limiting capabilities](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Security_Options) and/or using [user namespacing](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#User_Namespacing_Options) or use a VM instead.
|
||||
|
||||
### Seccomp
|
||||
Seccomp is a Linux kernel feature that restricts programs from making unauthorized system calls. This means that when seccomp is enabled there can be times where a process run inside a jail will be killed with the error "Operation not permitted." In order to find out which syscall needs to be added to the `--system-call-filter=` configuration you can use `strace`.
|
||||
|
||||
For example:
|
||||
```
|
||||
# /usr/bin/intel_gpu_top
|
||||
Failed to initialize PMU! (Operation not permitted)
|
||||
|
||||
# strace /usr/bin/intel_gpu_top 2>&1 |grep Operation\ not\ permitted
|
||||
perf_event_open({type=0x10 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER7, config=0x100002, sample_period=0, sample_type=0, read_format=PERF_FORMAT_TOTAL_TIME_ENABLED|PERF_FORMAT_GROUP, precise_ip=0 /* arbitrary skid */, use_clockid=1, ...}, -1, 0, -1, 0) = -1 EPERM (Operation not permitted)
|
||||
write(2, "Failed to initialize PMU! (Opera"..., 52Failed to initialize PMU! (Operation not permitted)
|
||||
```
|
||||
The syscall that needs to be added to the `--system-call-filter` option in the `jailmaker` config in this case would be `perf_event_open`. You may need to run strace multiple times.
|
||||
|
||||
Seccomp is important for security, but as a last resort can be disabled by setting `seccomp=0` in the jail config.
|
||||
|
||||
## Networking
|
||||
|
||||
By default a jails will use the same networking namespace, with access to all (physical) interfaces the TrueNAS host has access to. No further setup is required. You may download and install additional packages inside the jail. Note that some ports are already occupied by TrueNAS SCALE (e.g. 443 for the web interface), so your jail can't listen on these ports.
|
||||
|
|
@ -202,7 +212,7 @@ The rootfs image `jlmkr.py` downloads comes from the [Linux Containers Image ser
|
|||
|
||||
## Filing Issues and Community Support
|
||||
|
||||
When in need of help or when you think you've found a bug in jailmaker, [please start with reading this](https://github.com/Jip-Hop/jailmaker/discussions/135).
|
||||
When in need of help or when you think you've found a bug in `jailmaker`, [please start with reading this](https://github.com/Jip-Hop/jailmaker/discussions/135).
|
||||
|
||||
## References
|
||||
|
||||
|
|
|
|||
|
|
@ -52,7 +52,7 @@ See [Networking](./network.md)
|
|||
|
||||
### Colorized bash prompt
|
||||
|
||||
To visually distinguish between a root shell inside the jail and a root shell outside the jail, it's possible to colorize the shell prompt. When using a debian jail with the bash shell, you may run the following command **inside the jail** to get a yellow prompt inside the jail (will be activated the next time you run `jlmkr shell myjail`):
|
||||
To visually distinguish between a root shell inside the jail and a root shell outside the jail, it's possible to colorize the shell prompt. When using a debian jail with the bash shell, you may run the following command **inside the jail** to get a yellow prompt inside the jail (will be activated the next time you run `./jlmkr.py shell myjail`):
|
||||
|
||||
```bash
|
||||
echo "PS1='${debian_chroot:+($debian_chroot)}\[\033[01;33m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '" >> ~/.bashrc
|
||||
|
|
|
|||
|
|
@ -91,7 +91,7 @@ DHCP=false
|
|||
Address=192.168.X.XXX/24
|
||||
Gateway=192.168.X.X
|
||||
```
|
||||
Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `jlmkr stop JAILNAME && jlmkr start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP.
|
||||
Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `./jlmkr.py stop JAILNAME && ./jlmkr.py start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP.
|
||||
|
||||
## DNS via DHCP
|
||||
|
||||
|
|
|
|||
|
|
@ -11,9 +11,9 @@ Jailmaker operates in dual-mode: it supports using both directories and datasets
|
|||
|
||||
#### Stop all jails
|
||||
|
||||
`jlmkr stop jail1`
|
||||
`./jlmkr.py stop jail1`
|
||||
|
||||
`jlmkr stop jail2`
|
||||
`./jlmkr.py stop jail2`
|
||||
etc..
|
||||
|
||||
#### Move/rename the 'jailmaker' directory
|
||||
|
|
@ -56,6 +56,6 @@ Warning! It's important that both directories have the `/` at the end to make su
|
|||
|
||||
#### Test everything works
|
||||
|
||||
If everything works, you should be able to use the `jlmkr` command directly. Try doing a `jlmkr list` to check if the jails are correctly recognized
|
||||
If everything works, you should be able to use the `./jlmkr.py` command directly. Try doing a `./jlmkr.py list` to check if the jails are correctly recognized
|
||||
|
||||
You can also try creating a new jail and see that the dataset is created automatically.
|
||||
|
|
@ -2,4 +2,4 @@
|
|||
|
||||
## Setup
|
||||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/docker/config mydockerjail`. If you want the `nvidia-container-toolkit` to be installed, ensure you set `gpu_passthrough_nvidia=1` when creating the jail.
|
||||
Check out the [config](./config) template file. You may provide it when asked during `./jlmkr.py create` or, if you have the template file stored on your NAS, you may provide it directly by running `./jlmkr.py create --start --config /mnt/tank/path/to/docker/config mydockerjail`. If you want the `nvidia-container-toolkit` to be installed, ensure you set `gpu_passthrough_nvidia=1` when creating the jail.
|
||||
|
|
@ -6,17 +6,17 @@
|
|||
|
||||
## Setup
|
||||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/incus/config myincusjail`.
|
||||
Check out the [config](./config) template file. You may provide it when asked during `./jlmkr.py create` or, if you have the template file stored on your NAS, you may provide it directly by running `./jlmkr.py create --start --config /mnt/tank/path/to/incus/config myincusjail`.
|
||||
|
||||
We manually finish the setup by running the following after creating and starting the jail:
|
||||
|
||||
```bash
|
||||
jlmkr exec myincusjail bash -c 'incus admin init'
|
||||
./jlmkr.py exec myincusjail bash -c 'incus admin init'
|
||||
```
|
||||
|
||||
Follow [First steps with Incus](https://linuxcontainers.org/incus/docs/main/tutorial/first_steps/).
|
||||
|
||||
Then visit the Incus GUI inside the browser https://0.0.0.0:8443. To find out which IP address to use instead of 0.0.0.0, check the IP address for your jail with `jlmkr list`.
|
||||
Then visit the Incus GUI inside the browser https://0.0.0.0:8443. To find out which IP address to use instead of 0.0.0.0, check the IP address for your jail with `./jlmkr.py list`.
|
||||
|
||||
## Known Issues
|
||||
|
||||
|
|
|
|||
|
|
@ -6,17 +6,17 @@
|
|||
|
||||
## Setup
|
||||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/lxd/config mylxdjail`.
|
||||
Check out the [config](./config) template file. You may provide it when asked during `./jlmkr.py create` or, if you have the template file stored on your NAS, you may provide it directly by running `./jlmkr.py create --start --config /mnt/tank/path/to/lxd/config mylxdjail`.
|
||||
|
||||
We manually finish the setup by running the command below after creating and starting the jail. Choose the `dir` storage backend during `lxd init` and answer `yes` to "Would you like the LXD server to be available over the network?"
|
||||
|
||||
```bash
|
||||
jlmkr exec mylxdjail bash -c 'lxd init &&
|
||||
./jlmkr.py exec mylxdjail bash -c 'lxd init &&
|
||||
snap set lxd ui.enable=true &&
|
||||
systemctl reload snap.lxd.daemon'
|
||||
```
|
||||
|
||||
Then visit the `lxd` GUI inside the browser https://0.0.0.0:8443. To find out which IP address to use instead of 0.0.0.0, check the IP address for your jail with `jlmkr list`.
|
||||
Then visit the `lxd` GUI inside the browser https://0.0.0.0:8443. To find out which IP address to use instead of 0.0.0.0, check the IP address for your jail with `./jlmkr.py list`.
|
||||
|
||||
## Known Issues
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Setup
|
||||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/podman/config mypodmanjail`.
|
||||
Check out the [config](./config) template file. You may provide it when asked during `./jlmkr.py create` or, if you have the template file stored on your NAS, you may provide it directly by running `./jlmkr.py create --start --config /mnt/tank/path/to/podman/config mypodmanjail`.
|
||||
|
||||
## Rootless
|
||||
|
||||
|
|
@ -14,11 +14,11 @@ Check out the [config](./config) template file. You may provide it when asked du
|
|||
|
||||
Prerequisites: created a jail using the [config](./config) template file.
|
||||
|
||||
Run `jlmkr edit mypodmanjail` and add `--private-users=524288:65536 --private-users-ownership=chown` to `systemd_nspawn_user_args`. We start at UID 524288, as this is the [systemd range used for containers](https://github.com/systemd/systemd/blob/main/docs/UIDS-GIDS.md#summary).
|
||||
Run `./jlmkr.py edit mypodmanjail` and add `--private-users=524288:65536 --private-users-ownership=chown` to `systemd_nspawn_user_args`. We start at UID 524288, as this is the [systemd range used for containers](https://github.com/systemd/systemd/blob/main/docs/UIDS-GIDS.md#summary).
|
||||
|
||||
The `--private-users-ownership=chown` option will ensure the rootfs ownership is corrected.
|
||||
|
||||
After the jail has started run `jlmkr stop mypodmanjail && jlmkr edit mypodmanjail`, remove `--private-users-ownership=chown` and increase the UID range to `131072` to double the number of UIDs available in the jail. We need more than 65536 UIDs available in the jail, since rootless podman also needs to be able to map UIDs. If I leave the `--private-users-ownership=chown` option I get the following error:
|
||||
After the jail has started run `./jlmkr.py stop mypodmanjail && ./jlmkr.py edit mypodmanjail`, remove `--private-users-ownership=chown` and increase the UID range to `131072` to double the number of UIDs available in the jail. We need more than 65536 UIDs available in the jail, since rootless podman also needs to be able to map UIDs. If I leave the `--private-users-ownership=chown` option I get the following error:
|
||||
|
||||
> systemd-nspawn[678877]: Automatic UID/GID adjusting is only supported for UID/GID ranges starting at multiples of 2^16 with a range of 2^16
|
||||
|
||||
|
|
@ -31,7 +31,7 @@ systemd_nspawn_user_args=--network-macvlan=eno1
|
|||
--private-users=524288:131072
|
||||
```
|
||||
|
||||
Start the jail with `jlmkr start mypodmanjail` and open a shell session inside the jail (as the remapped root user) with `jlmkr shell mypodmanjail`.
|
||||
Start the jail with `./jlmkr.py start mypodmanjail` and open a shell session inside the jail (as the remapped root user) with `./jlmkr.py shell mypodmanjail`.
|
||||
|
||||
Then inside the jail setup the new rootless user:
|
||||
|
||||
|
|
@ -61,7 +61,7 @@ exit
|
|||
From the TrueNAS host, open a shell as the rootless user inside the jail.
|
||||
|
||||
```bash
|
||||
jlmkr shell --uid 1000 mypodmanjail
|
||||
./jlmkr.py shell --uid 1000 mypodmanjail
|
||||
```
|
||||
|
||||
Run rootless podman as user 1000.
|
||||
|
|
@ -97,7 +97,7 @@ Add `sysctl net.ipv4.ip_unprivileged_port_start=23` to the `pre_start_hook` insi
|
|||
Install and enable cockpit:
|
||||
|
||||
```bash
|
||||
jlmkr exec mypodmanjail bash -c "dnf -y install cockpit cockpit-podman && \
|
||||
./jlmkr.py exec mypodmanjail bash -c "dnf -y install cockpit cockpit-podman && \
|
||||
systemctl enable --now cockpit.socket && \
|
||||
ip a &&
|
||||
ip route | awk '/default/ { print \$9 }'"
|
||||
|
|
@ -108,7 +108,7 @@ Check the IP address of the jail and access the Cockpit web interface at https:/
|
|||
If you've setup the `rootless` user, you may login with the password you've created earlier. Otherwise you'd have to add an admin user first:
|
||||
|
||||
```bash
|
||||
jlmkr exec podmantest bash -c 'adduser admin
|
||||
./jlmkr.py exec podmantest bash -c 'adduser admin
|
||||
passwd admin
|
||||
usermod -aG wheel admin'
|
||||
```
|
||||
|
|
|
|||
Loading…
Reference in New Issue