Jip-Hop
GitHub
commit
bc8284edb8
|
|
@ -1,61 +0,0 @@
|
|||
# Jailmaker
|
||||
|
||||
## Advanced Networking
|
||||
|
||||
These are notes on advanced networking setup you may want to try. Contributions are welcome!
|
||||
|
||||
### Bridge Networking
|
||||
|
||||
As an alternative to the default host networking mode, you may want to connect to a bridge interface instead and let the jail obtain its IP address via DHCP (although you may have to be patient for up to 20 seconds after the jail started for networking to work, [assigning the IP address is somehow slow](https://github.com/Jip-Hop/jailmaker/issues/7)).
|
||||
|
||||
[This YouTube video](https://www.youtube.com/watch?v=7clQw132w58) may be helpful when setting up the bridge interface. Note: You may lock yourself out... It may take several tries... TrueNAS is a bit picky when switching IP addresses and toggling DHCP. May be helpful to connect a monitor and keyboard to the NAS and use `/etc/netcli` to reset the networking interface. Kept bothering with "Register Default Gateway" warning... I just clicked Cancel.
|
||||
|
||||
Add the `--network-bridge=br1 --resolv-conf=bind-host` systemd-nspawn flag when asked for `Additional flags` during jail creation, or set it post-creation by [editing](./README.md#edit-jail-config) the `SYSTEMD_NSPAWN_USER_ARGS` variable inside the `config` file.
|
||||
|
||||
The TrueNAS host and the jail will be able to communicate with each other as if the jail was just another device on the LAN. It will use the same DNS servers as the TrueNAS host because the `--resolv-conf=bind-host` option bind mounts the `/etc/resolv.conf` file from the host inside the jail. If you want to use the DNS servers advertised via DHCP, then check [DNS via DHCP](#dns-via-dhcp).
|
||||
|
||||
To configure a **static IP** with our bridge interface, we need to edit the `80-container-host0.network` file located in `/etc/systemd/network`. Change the `[Network]` section to look like this:
|
||||
|
||||
```ini
|
||||
[Network]
|
||||
DHCP=false
|
||||
Address=192.168.0.12/24
|
||||
Gateway=192.168.0.1
|
||||
LinkLocalAddressing=no
|
||||
LLDP=yes
|
||||
EmitLLDP=customer-bridge
|
||||
```
|
||||
Then restart the `systemd-networkd` service and check your network configuration.
|
||||
|
||||
```shell
|
||||
systemctl restart systemd-networkd
|
||||
systemctl status systemd-networkd
|
||||
ifconfig
|
||||
```
|
||||
|
||||
|
||||
### Macvlan Networking
|
||||
|
||||
To setup Macvlan Networking you may follow the [Bridge Networking](#bridge-networking) section, but skip the setup of a bridge interface and use these flags instead: `--network-macvlan=eno1 --resolv-conf=bind-host`. By default the TrueNAS host and jail will not be able to communicate with each other via the network if Macvlan Networking mode is used. If that's required it would be better to use [Bridge Networking](#bridge-networking).
|
||||
|
||||
### DNS via DHCP
|
||||
|
||||
If you're not using host networking, and you're not using the `--resolv-conf=` in case of bridge/macvlan networking, then you have to configure the DNS servers to use.
|
||||
|
||||
To get DNS servers via DHCP install and enable `resolvconf`.
|
||||
|
||||
```shell
|
||||
# Only run this inside the jail!
|
||||
|
||||
# Temporarily fix DNS resolution,
|
||||
# otherwise we can't install packages
|
||||
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
|
||||
# On debian based distro
|
||||
apt update && apt -y install resolvconf
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [systemd-nspawn](https://manpages.debian.org/bullseye/systemd-container/systemd-nspawn.1.en.html)- [Setting up Systemd-nspawn](https://www.cocode.se/linux/systemd_nspawn.html#orge360318)
|
||||
- [Debian Reference - Chapter 5. Network setup](https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution)
|
||||
- [Disabling link-local addressing](https://jerrington.me/posts/2017-08-06-systemd-nspawn-disabling-link-local-addressing.html#disabling-link-local-addressing)
|
||||
32
README.md
32
README.md
|
|
@ -90,13 +90,15 @@ jlmkr start myjail
|
|||
|
||||
### List Jails
|
||||
|
||||
See list of jails (including running, startup state, GPU passthrough, distro, and IP).
|
||||
|
||||
```shell
|
||||
jlmkr list
|
||||
```
|
||||
|
||||
### Execute Command in Jail
|
||||
|
||||
You may want to execute a command inside a jail, for example from a shell script or a CRON job. The example below executes the `env` command inside the jail.
|
||||
You may want to execute a command inside a jail, for example manually from the TrueNAS shell, a shell script or a CRON job. The example below executes the `env` command inside the jail.
|
||||
|
||||
```shell
|
||||
jlmkr exec myjail env
|
||||
|
|
@ -118,6 +120,8 @@ Once you've created a jail, it will exist in a directory inside the `jails` dir
|
|||
|
||||
### Remove Jail
|
||||
|
||||
Delete a jail and remove it's files (requires confirmation).
|
||||
|
||||
```shell
|
||||
jlmkr remove myjail
|
||||
```
|
||||
|
|
@ -136,6 +140,8 @@ jlmkr restart myjail
|
|||
|
||||
### Jail Shell
|
||||
|
||||
Switch into the jail's shell.
|
||||
|
||||
```shell
|
||||
jlmkr shell myjail
|
||||
```
|
||||
|
|
@ -148,6 +154,8 @@ jlmkr status myjail
|
|||
|
||||
### Jail Logs
|
||||
|
||||
View a jail's logs.
|
||||
|
||||
```shell
|
||||
jlmkr log myjail
|
||||
```
|
||||
|
|
@ -158,17 +166,21 @@ Expert users may use the following additional commands to manage jails directly:
|
|||
|
||||
## Networking
|
||||
|
||||
By default the jail will have full access to the host network. No further setup is required. You may download and install additional packages inside the jail. Note that some ports are already occupied by TrueNAS SCALE (e.g. 443 for the web interface), so your jail can't listen on these ports. This is inconvenient if you want to host some services (e.g. traefik) inside the jail. To workaround this issue when using host networking, you may disable DHCP and add several static IP addresses (Aliases) through the TrueNAS web interface. If you setup the TrueNAS web interface to only listen on one of these IP addresses, the ports on the remaining IP addresses remain available for the jail to listen on.
|
||||
By default a jails will use the same networking namespace, with access to all (physical) interfaces the TrueNAS host has access to. No further setup is required. You may download and install additional packages inside the jail. Note that some ports are already occupied by TrueNAS SCALE (e.g. 443 for the web interface), so your jail can't listen on these ports.
|
||||
|
||||
See [Advanced Networking](./NETWORKING.md) for more.
|
||||
Depending on the service this may be o.k. For example Home Assistant will bind to port 8123, leaving the 80 and 443 ports free from clashes for the TrueNAS web interface. You can then either connect to the service on 8123, or use a reverse proxy such as traefik.
|
||||
|
||||
But clashes may happen if you want some services (e.g. traefik) inside the jail to listen on port 443. To workaround this issue when using host networking, you may disable DHCP and add several static IP addresses (Aliases) through the TrueNAS web interface. If you setup the TrueNAS web interface to only listen on one of these IP addresses, the ports on the remaining IP addresses remain available for the jail to listen on.
|
||||
|
||||
See [the networking docs](./docs/network.md) for more advanced options (bridge and macvlan networking).
|
||||
|
||||
## Docker
|
||||
|
||||
Using the [docker config template](./templates/docker/README.md) is recommended if you want to run docker inside the jail. You may of course manually install docker inside a jail. But keep in mind that you need to add `--system-call-filter='add_key keyctl bpf'` (or disable seccomp filtering). It is [not recommended to use host networking for a jail in which you run docker](https://github.com/Jip-Hop/jailmaker/issues/119). Docker needs to manage iptables rules, which it can safely do in its own networking namespace (when using [bridge or macvlan networking](./NETWORKING.md) for the jail).
|
||||
Using the [docker config template](./templates/docker/README.md) is recommended if you want to run docker inside the jail. You may of course manually install docker inside a jail. But keep in mind that you need to add `--system-call-filter='add_key keyctl bpf'` (or disable seccomp filtering). It is [not recommended to use host networking for a jail in which you run docker](https://github.com/Jip-Hop/jailmaker/issues/119). Docker needs to manage iptables rules, which it can safely do in its own networking namespace (when using [bridge or macvlan networking](./docs/network.md) for the jail).
|
||||
|
||||
## Documentation
|
||||
|
||||
Additional documentation contributed by the community can be found in [the docs directory](./docs/).
|
||||
Additional documentation can be found in [the docs directory](./docs/) (contributions are welcome!).
|
||||
|
||||
## Comparison
|
||||
|
||||
|
|
@ -178,16 +190,6 @@ TODO: write comparison between systemd-nspawn (without `jailmaker`), LXC, VMs, D
|
|||
|
||||
The rootfs image `jlmkr.py` downloads comes from the [Linux Containers Image server](https://images.linuxcontainers.org). These images are made for LXC. We can use them with systemd-nspawn too, although not all of them work properly. For example, the `alpine` image doesn't work well. If you stick with common systemd based distros (Debian, Ubuntu, Arch Linux...) you should be fine.
|
||||
|
||||
## Tips & Tricks
|
||||
|
||||
### Colorized bash prompt
|
||||
|
||||
To visually distinguish between a root shell inside the jail and a root shell outside the jail, it's possible to colorize the shell prompt. When using a debian jail with the bash shell, you may run the following command to get a yellow prompt inside the jail (will be activated the next time you run `jlmkr shell myjail`):
|
||||
|
||||
```bash
|
||||
echo "PS1='${debian_chroot:+($debian_chroot)}\[\033[01;33m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '" >> ~/.bashrc
|
||||
```
|
||||
|
||||
## Filing Issues and Community Support
|
||||
|
||||
When in need of help or when you think you've found a bug in jailmaker, [please start with reading this](https://github.com/Jip-Hop/jailmaker/discussions/135).
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
Welcome to the jailmaker wiki!
|
||||
# Jailmaker Docs
|
||||
|
||||
Welcome to the Jailmaker Docs!
|
||||
|
||||
Use the sidebar to navigate the topics.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,8 @@
|
|||
# User Management
|
||||
# Jailmaker Docs
|
||||
|
||||
Anything described on this page is completely optional. You do NOT need to do anything of this in order to start using jailmaker.
|
||||
|
||||
## User Management
|
||||
The root user (also known as the superuser or su) can access any file, make system changes, and lots of room for security vulnerabilities.
|
||||
For this reason you should aspire to run services as a non-root user.
|
||||
|
||||
|
|
@ -10,28 +14,54 @@ Where username can be anything, but should reflect the service/jail's name for d
|
|||
Then a password should be created as some commands require a non-blank password to be inserted:
|
||||
`passwd USERNAME`
|
||||
|
||||
If you want the ability to run commands as root, add the user to the sudo group
|
||||
`usermod -aG sudo USERNAME`
|
||||
If you want the ability to run commands as root, add the user to the sudo group:
|
||||
|
||||
```sh
|
||||
usermod -aG sudo USERNAME
|
||||
```
|
||||
|
||||
This WILL require a non-blank password, and any command run with sudo will be run as root not as the user. But it saves time compared to switching users to root to install/change things then switching back.
|
||||
|
||||
### Switch to user
|
||||
`su -l USERNAME`
|
||||
|
||||
```sh
|
||||
su -l USERNAME
|
||||
```
|
||||
|
||||
### Put a password on Root
|
||||
While logged in as root run `passwd`
|
||||
|
||||
# Common tweaks
|
||||
While logged in as root run `passwd`.
|
||||
|
||||
## Common Tweaks
|
||||
|
||||
### Update repository list
|
||||
`sudo apt update`
|
||||
|
||||
```sh
|
||||
sudo apt update
|
||||
```
|
||||
|
||||
### Install common services
|
||||
`sudo apt install nano wget curl git`
|
||||
|
||||
```sh
|
||||
sudo apt install nano wget curl git
|
||||
```
|
||||
|
||||
### Set Static IP
|
||||
See `Networking`
|
||||
|
||||
See [Networking](./network.md)
|
||||
|
||||
### Colorized bash prompt
|
||||
|
||||
To visually distinguish between a root shell inside the jail and a root shell outside the jail, it's possible to colorize the shell prompt. When using a debian jail with the bash shell, you may run the following command **inside the jail** to get a yellow prompt inside the jail (will be activated the next time you run `jlmkr shell myjail`):
|
||||
|
||||
```bash
|
||||
echo "PS1='${debian_chroot:+($debian_chroot)}\[\033[01;33m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '" >> ~/.bashrc
|
||||
```
|
||||
|
||||
### Install Docker
|
||||
```
|
||||
|
||||
It's advised to use the [docker config template](../templates/docker/README.md). But you can install it manually like this as well:
|
||||
|
||||
```sh
|
||||
apt install curl && cd /tmp && curl -fsSL https://get.docker.com -o get-docker.sh && sudo sh get-docker.sh && cd ~ && docker
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,35 +0,0 @@
|
|||
Create a jail
|
||||
`jlmkr create JAILNAME`
|
||||
|
||||
Start a jail
|
||||
`jlmkr start JAILNAME`
|
||||
|
||||
Stop a jail
|
||||
`jlmkr stop JAILNAME`
|
||||
|
||||
Check jail status
|
||||
`jlmkr status JAILNAME`
|
||||
|
||||
Delete a jail and remove it's files (requires confirmation)
|
||||
`jlmkr remove JAILNAME`
|
||||
|
||||
See list of jails (including running, non running, distro, startup state, and IP)
|
||||
`jlmkr list`
|
||||
|
||||
See list of running jails
|
||||
`machinectl list`
|
||||
|
||||
Execute a command inside a jail from the TrueNAS shell
|
||||
`jlmkr exec JAILNAME COMMAND`
|
||||
|
||||
Execute a bash command inside a jail from the TrueNAS shell
|
||||
`jlmkr exec JAILNAME bash -c 'BASHCOMMAND'`
|
||||
|
||||
Switch into the jail's shell
|
||||
`machinectl shell JAILNAME`
|
||||
|
||||
View a jail's logs
|
||||
`jlmkr log JAILNAME`
|
||||
|
||||
Edit a jail's config
|
||||
`jlmkr edit JAILNAME`
|
||||
|
|
@ -1,12 +1,14 @@
|
|||
# TrueNAS Compatibility
|
||||
# Jailmaker Docs
|
||||
|
||||
## TrueNAS Compatibility
|
||||
| | |
|
||||
|---|---|
|
||||
|TrueNAS Core|❌|
|
||||
|TrueNAS 22.12|✅|
|
||||
|TrueNAS 23.10|✅|
|
||||
|TrueNAS 24.04 nightly|✅|
|
||||
|TrueNAS 24.04|✅|
|
||||
|
||||
# Distro Compatibility
|
||||
## Distro Compatibility
|
||||
| | |
|
||||
|---|---|
|
||||
|Debian 11 Bullseye|✅|
|
||||
|
|
|
|||
105
docs/network.md
105
docs/network.md
|
|
@ -1,24 +1,60 @@
|
|||
# Host Passthrough (Default network configuration)
|
||||
By default jails will use the same physical interface as the TrueNAS host. If a service attempts to bind to port 80 or 443, it will either fail or render both the service and TrueNAS unavailable.
|
||||
### Flaws
|
||||
Depending on the service this may be ok, for example Home Assistant will bind to port 8123, leaving the 80 and 443 ports free from clashes for the TrueNAS web interface. You can then either connect to the service with the port, or use a reverse proxy such as [nginx](https://www.nginx.com/#).
|
||||
### Setup
|
||||
No configuration is necessary
|
||||
# Jailmaker Docs
|
||||
|
||||
# MAC VLAN Virtual Interface
|
||||
Some services require the use of port 80 or 443, or would benefit from a separate IP. For these situations the easiest network configuration is the MAC VLAN configuration. This creates a virtual interface with its own separate randomly generated MAC address and IP.
|
||||
The default config uses DHCP by default, but can easily be set to a Static IP.
|
||||
### Flaws
|
||||
Any services in the jail cannot communicate with the direct host (TrueNAS). The jail can communicate with any other jail or device on the network, besides TrueNAS. This may or not be a benefit (security) or disadvantage (no communication) depending on your service.
|
||||
### Setup
|
||||
Add the following argument to the "additional flags" prompt of jail creation or the "systemd_nspawn_user_arguments" line of the jail config file:
|
||||
## Host Networking
|
||||
|
||||
[Notes on the default host networking are in the main README.md file](../README.md#networking).
|
||||
|
||||
## Bridge Networking
|
||||
|
||||
As an alternative to the default host networking mode, you may want to connect to a bridge interface instead and let the jail obtain its IP address via DHCP.
|
||||
|
||||
[<br>Watch on YouTube](https://www.youtube.com/watch?v=uPkoeWUfiHU "TrueNAS Scale: Setting up a Static IP and Network Bridge // Access NAS host from VM - YouTube Video")
|
||||
|
||||
The above YouTube video may be helpful when setting up the bridge interface.
|
||||
|
||||
### Bridge Flaws
|
||||
|
||||
This type of interface takes much longer to set up both in complexity and wait time (you may have to be patient for up to 60 seconds after the jail started for networking to work, [assigning the IP address via DHCP is somehow slow](https://github.com/Jip-Hop/jailmaker/issues/7)). Furthermore, if the configuration is not correct it can render your TrueNAS inaccessible via ssh or the web interface, necessitating a reset using a keyboard and monitor plugged into the TrueNAS server and use `/etc/netcli` to reset the networking interface.
|
||||
|
||||
### Bridge Setup
|
||||
|
||||
Add the `--network-bridge=br1 --resolv-conf=bind-host` systemd-nspawn flag when asked for `Additional flags` during jail creation, or set it post-creation by [editing](./README.md#edit-jail-config) the `SYSTEMD_NSPAWN_USER_ARGS` variable inside the `config` file.
|
||||
|
||||
The TrueNAS host and the jail will be able to communicate with each other as if the jail was just another device on the LAN. It will use the same DNS servers as the TrueNAS host because the `--resolv-conf=bind-host` option bind mounts the `/etc/resolv.conf` file from the host inside the jail. If you want to use the DNS servers advertised via DHCP, then check [DNS via DHCP](#dns-via-dhcp).
|
||||
|
||||
### Bridge Static IP
|
||||
To configure a static IP with our bridge interface, we need to edit the `/etc/systemd/network/80-container-host0.network` file. Change the [Network] section to look like this:
|
||||
|
||||
```ini
|
||||
[Network]
|
||||
DHCP=false
|
||||
Address=192.168.0.12/24
|
||||
Gateway=192.168.0.1
|
||||
LinkLocalAddressing=no
|
||||
LLDP=yes
|
||||
EmitLLDP=customer-bridge
|
||||
```
|
||||
--network-macvlan=eno1 --resolv-conf=bind-host
|
||||
Then restart the `systemd-networkd` service and check your network configuration.
|
||||
|
||||
```shell
|
||||
systemctl restart systemd-networkd
|
||||
systemctl status systemd-networkd
|
||||
ifconfig
|
||||
```
|
||||
|
||||
### Setting a Static IP
|
||||
To set a Static IP you need to disable DHCP in the macvlan config file `/etc/systemd/network/mv-dhcp.network`
|
||||
You can do this with a network client like WinSCP by navigating into the jail's filesystem then the path above, or by using a text editing program like nano by running `nano /etc/systemd/network/mv-dhcp.network` in the jail's shell.
|
||||
## Macvlan Networking
|
||||
|
||||
Some services require the use of port 80 or 443, or would benefit from a separate IP. For these situations the easiest network configuration is the MAC VLAN configuration. This creates a virtual interface with its own separate randomly generated MAC address and IP. The default config uses DHCP by default, but can easily be set to a Static IP.
|
||||
|
||||
### Macvlan Flaws
|
||||
Any services in the jail cannot communicate with the direct host (TrueNAS). The jail can communicate with any other jail or device on the network, besides TrueNAS or VMs hosted on TrueNAS. This may be a benefit (security) or disadvantage (no communication) depending on your service. If that's required it would be better to use [Bridge Networking](#bridge-networking).
|
||||
|
||||
### Macvlan Setup
|
||||
|
||||
Add the following argument to the "additional flags" prompt of jail creation or the "systemd_nspawn_user_arguments" line of the jail config file: `--network-macvlan=eno1 --resolv-conf=bind-host`. Where eno1 is the name of your physical network interface.
|
||||
|
||||
### Macvlan Static IP
|
||||
To set a Static IP you need to disable DHCP in the macvlan config file `/etc/systemd/network/mv-dhcp.network`. You can do this with a network client like WinSCP by navigating into the jail's filesystem then the path above, or by using a text editing program like nano by running `nano /etc/systemd/network/mv-dhcp.network` in the jail's shell.
|
||||
|
||||
The DHCP in [Network] needs to be set to false, an Address (static IP) needs to be added, a Gateway needs to be defined (e.g your router such as 192.168.0.1) and the entire DHCP section needs to be removed.
|
||||
|
||||
|
|
@ -35,23 +71,24 @@ Gateway=192.168.X.X
|
|||
```
|
||||
Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `jlmkr stop JAILNAME && jlmkr start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP.
|
||||
|
||||
# Passthrough a TrueNAS Bridge Interface
|
||||
By creating a network bridge in the TrueNAS Network page you can bridge the incoming physical network interface to a virtual interface that can be passed to the jail. This type of interface has the benefits of a MAC VLAN interface without the flaws (host to jail networking). Once working the virtual interface can either be assigned a static IP or obtain one automatically via DHCP.
|
||||
### Flaws
|
||||
This type of interface takes much longer to set up both in complexity and wait time as there is a current flaw in which HDCP can take between 10 seconds and a minute.
|
||||
Furthermore, if the configuration is not correct it can render your TrueNAS inaccessible via ssh, necessitating a reset using a keyboard and monitor plugged into the TrueNAS server.
|
||||
### Setup
|
||||
[TrueNAS Bridge interface guide](https://www.youtube.com/watch?v=7clQw132w58)
|
||||
May be helpful to connect a monitor and keyboard to the NAS and use /etc/netcli to reset the networking interface. Kept bothering with "Register Default Gateway" warning... I just clicked Cancel.
|
||||
## DNS via DHCP
|
||||
|
||||
Add the `--network-bridge=br1 --resolv-conf=bind-host` flag when asked for additional flags during jail creation, or set it post-creation by editing the `SYSTEMD_NSPAWN_USER_ARGS` variable inside the config file.
|
||||
If you're not using host networking, and you're not using the `--resolv-conf=` in case of bridge/macvlan networking, then you have to configure the DNS servers to use.
|
||||
|
||||
### Static IP
|
||||
To configure a static IP with our bridge interface, we need to edit the `/etc/systemd/network/80-container-host0.network` file. Change the [Network] section to look like this:
|
||||
To get DNS servers via DHCP install and enable `resolvconf`.
|
||||
|
||||
```shell
|
||||
# Only run this inside the jail!
|
||||
|
||||
# Temporarily fix DNS resolution,
|
||||
# otherwise we can't install packages
|
||||
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
|
||||
# On debian based distro
|
||||
apt update && apt -y install resolvconf
|
||||
```
|
||||
[Network]
|
||||
DHCP=false
|
||||
Address=192.168.X.XXX/24
|
||||
Gateway=192.168.X.X
|
||||
```
|
||||
Then restart the network interface inside the jail `systemctl restart systemd-networkd` or restart the jail by running `jlmkr stop JAILNAME && jlmkr start JAILNAME` from the TrueNAS shell. Use `ifconfig` to verify the interface is up and has the correct IP.
|
||||
|
||||
## References
|
||||
|
||||
- [systemd-nspawn](https://manpages.debian.org/bullseye/systemd-container/systemd-nspawn.1.en.html)- [Setting up Systemd-nspawn](https://www.cocode.se/linux/systemd_nspawn.html#orge360318)
|
||||
- [Debian Reference - Chapter 5. Network setup](https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution)
|
||||
- [Disabling link-local addressing](https://jerrington.me/posts/2017-08-06-systemd-nspawn-disabling-link-local-addressing.html#disabling-link-local-addressing)
|
||||
|
|
@ -1,3 +1,5 @@
|
|||
# Jailmaker Docs
|
||||
|
||||
(Anecdotal from observations, actual measurements with resource monitor captures and wall power meter coming soon.)
|
||||
|
||||
Kubernetes Server (TrueNAS Apps) with no apps installed:
|
||||
|
|
@ -16,6 +18,5 @@ Systemd-nspawn container (jailmaker) with 10 apps installed:
|
|||
* Idle on 7100T: ~4% / 8W
|
||||
* Idle on 10600K: ~0%
|
||||
|
||||
|
||||
Systemd-nspawn container (jailmaker) with 20 apps installed:
|
||||
* Idle on 10600K: ~1%
|
||||
|
|
|
|||
|
|
@ -1,12 +1,14 @@
|
|||
# Default storage system
|
||||
When creating a jail, an entire Linux filesystem is created in the 'rootfs' folder within the jail's folder of the jailmaker directory E.g `/mnt/tank/vault/jailmaker/jails/jailname/rootfs`. No files from the TrueNAS host will be available.
|
||||
# Jailmaker Docs
|
||||
|
||||
## Default storage system
|
||||
When creating a jail, an entire Linux filesystem is created in the 'rootfs' folder within the jail's folder of the jailmaker directory. E.g. `/mnt/tank/vault/jailmaker/jails/jailname/rootfs`. No files from the TrueNAS host will be available.
|
||||
|
||||
Common locations for services are:
|
||||
`/home` for user accessible files
|
||||
`/var/www/` for webpages
|
||||
`/tmp` for temporary application data such as build files
|
||||
|
||||
# Linking folders to TrueNAS folders
|
||||
## Linking folders to TrueNAS folders
|
||||
To allow file access by either the jail, another jail, or TrueNAS a bind can be made. A bind creates a link between two locations. Think of this as a portal, anything that goes in one side is visible from the other side and vice versa.
|
||||
|
||||
Note that creating a file in the jail or TrueNAS will reflect in both binded locations, so be careful of overwrites and corruption.
|
||||
|
|
@ -21,7 +23,7 @@ And where `/jail/path/to/` is the folder you want those shared files accessible
|
|||
|
||||
### Example
|
||||
A use of this is making files available in a jail for it to use or serve, such as media files in Plex/Jellyfin:
|
||||
Example: `--bind='/mnt/tank/content/:/media'` will make any files inside the content dataset of the tank pool available inside the jail's /media folder. To visualise or test this you can copy some files to `/mnt/tank/content/` such as `media1.mp4`, `media2.mkv` and `photo.jpg`. Then change directory to that folder inside the jail `cd /media` and list files in that directory `ls -l` where those files should appear.
|
||||
Example: `--bind='/mnt/tank/content/:/media'` will make any files inside the content dataset of the tank pool available inside the jail's /media folder. To visualize or test this you can copy some files to `/mnt/tank/content/` such as `media1.mp4`, `media2.mkv` and `photo.jpg`. Then change directory to that folder inside the jail `cd /media` and list files in that directory `ls -l` where those files should appear.
|
||||
|
||||
### Warning
|
||||
Do not bind your TrueNAS system directories (`/root` `/mnt` `/dev` `/bin` `/etc` `/home` `/var` `/usr` or anything else in the root directory) to your jail as this can cause TrueNAS to lose permissions and render your TrueNAS system unusable.
|
||||
|
|
|
|||
|
|
@ -1,24 +1,26 @@
|
|||
# ZFS Datasets Migration
|
||||
# Jailmaker Docs
|
||||
|
||||
## ZFS Datasets Migration
|
||||
|
||||
From version 1.1.4 ZFS Datasets support was added to jailmaker.
|
||||
By default starting in v1.1.4, jailmaker will create a separate dataset for each jail if possible. This allows the user to configure snapshots, rollbacks, replications etc.
|
||||
|
||||
Jailmaker operates in dual-mode: it supports using both directories and datasets. If the 'jailmaker' directory is a dataset, it will use datasets, if it is a directory, it will use directories.
|
||||
___
|
||||
## Procedure to migrate from directories to ZFS Datasets
|
||||
|
||||
### Stop all jails
|
||||
### Procedure to migrate from directories to ZFS Datasets
|
||||
|
||||
#### Stop all jails
|
||||
|
||||
`jlmkr stop jail1`
|
||||
|
||||
`jlmkr stop jail2`
|
||||
etc..
|
||||
|
||||
### Move/rename the 'jailmaker' directory
|
||||
#### Move/rename the 'jailmaker' directory
|
||||
|
||||
`mv jailmaker orig_jailmaker`
|
||||
|
||||
### Create the ZFS datasets for jailmaker
|
||||
#### Create the ZFS datasets for jailmaker
|
||||
|
||||
Create all the required datasets via GUI or CLI.
|
||||
|
||||
|
|
@ -44,8 +46,7 @@ zfs create mypool/jailmaker/jails/jail1
|
|||
zfs create mypool/jailmaker/jails/jail2
|
||||
```
|
||||
|
||||
|
||||
### Move the existing jail data into the newly created datasets
|
||||
#### Move the existing jail data into the newly created datasets
|
||||
|
||||
Now move all the jail data:
|
||||
|
||||
|
|
@ -53,7 +54,7 @@ Now move all the jail data:
|
|||
|
||||
Warning! It's important that both directories have the `/` at the end to make sure contents are copied correctly. Otherwise you may end up with `jailmaker/jailmaker`
|
||||
|
||||
### Test everything works
|
||||
#### Test everything works
|
||||
|
||||
If everything works, you should be able to use the `jlmkr` command directly. Try doing a `jlmkr list` to check if the jails are correctly recognized
|
||||
|
||||
|
|
|
|||
186
jlmkr.py
186
jlmkr.py
|
|
@ -4,7 +4,7 @@
|
|||
with full access to all files via bind mounts, \
|
||||
thanks to systemd-nspawn!"""
|
||||
|
||||
__version__ = "1.3.0"
|
||||
__version__ = "1.4.0"
|
||||
|
||||
__disclaimer__ = """USE THIS SCRIPT AT YOUR OWN RISK!
|
||||
IT COMES WITHOUT WARRANTY AND IS NOT SUPPORTED BY IXSYSTEMS."""
|
||||
|
|
@ -43,15 +43,13 @@ docker_compatible=0
|
|||
# Turning off seccomp filtering improves performance at the expense of security
|
||||
seccomp=1
|
||||
|
||||
# Add additional systemd-nspawn flags
|
||||
# E.g. to mount host storage in the jail (--bind-ro for readonly):
|
||||
# --bind='/mnt/pool/dataset:/home' --bind-ro=/etc/certificates
|
||||
# E.g. macvlan networking:
|
||||
# --network-macvlan=eno1 --resolv-conf=bind-host
|
||||
# E.g. bridge networking:
|
||||
# --network-bridge=br1 --resolv-conf=bind-host
|
||||
# E.g. allow syscalls required by docker:
|
||||
# --system-call-filter='add_key keyctl bpf'
|
||||
# Below you may add additional systemd-nspawn flags behind systemd_nspawn_user_args=
|
||||
# To mount host storage in the jail, you may add: --bind='/mnt/pool/dataset:/home'
|
||||
# To readonly mount host storage, you may add: --bind-ro=/etc/certificates
|
||||
# To use macvlan networking add: --network-macvlan=eno1 --resolv-conf=bind-host
|
||||
# To use bridge networking add: --network-bridge=br1 --resolv-conf=bind-host
|
||||
# Ensure to change eno1/br1 to the interface name you want to use
|
||||
# To allow syscalls required by docker add: --system-call-filter='add_key keyctl bpf'
|
||||
systemd_nspawn_user_args=
|
||||
|
||||
# Specify command/script to run on the HOST before starting the jail
|
||||
|
|
@ -73,10 +71,8 @@ post_stop_hook=
|
|||
distro=debian
|
||||
release=bookworm
|
||||
|
||||
# Specify command/script to run IN THE JAIL before the first start
|
||||
# Specify command/script to run IN THE JAIL on the first start (once networking is ready in the jail)
|
||||
# Useful to install packages on top of the base rootfs
|
||||
# NOTE: this script will run in the host networking namespace and
|
||||
# ignores all systemd_nspawn_user_args such as bind mounts
|
||||
initial_setup=
|
||||
# initial_setup=bash -c 'apt-get update && apt-get -y upgrade'
|
||||
|
||||
|
|
@ -545,55 +541,6 @@ def start_jail(jail_name):
|
|||
|
||||
seccomp = config.my_getboolean("seccomp")
|
||||
|
||||
# Handle initial setup
|
||||
initial_setup = config.my_get("initial_setup")
|
||||
|
||||
# Alternative method to setup on first boot:
|
||||
# https://www.undrground.org/2021/01/25/adding-a-single-run-task-via-systemd/
|
||||
# If there's no machine-id, then this the first time the jail is started
|
||||
if initial_setup and not os.path.exists(
|
||||
os.path.join(jail_rootfs_path, "etc/machine-id")
|
||||
):
|
||||
initial_setup_file = None
|
||||
|
||||
if initial_setup.startswith("#!"):
|
||||
# Write a script file and call that
|
||||
initial_setup_file = os.path.abspath(
|
||||
os.path.join(jail_path, ".initial_setup")
|
||||
)
|
||||
print(initial_setup, file=open(initial_setup_file, "w"))
|
||||
stat_chmod(initial_setup_file, 0o700)
|
||||
cmd = [
|
||||
"systemd-nspawn",
|
||||
"-q",
|
||||
"-D",
|
||||
jail_rootfs_path,
|
||||
f"--bind-ro={initial_setup_file}:/root/initial_startup",
|
||||
"/root/initial_startup",
|
||||
]
|
||||
else:
|
||||
# Run the command directly if it doesn't start with a shebang
|
||||
cmd = [
|
||||
"systemd-nspawn",
|
||||
"-q",
|
||||
"-D",
|
||||
jail_rootfs_path,
|
||||
*shlex.split(initial_setup),
|
||||
]
|
||||
|
||||
returncode = subprocess.run(cmd).returncode
|
||||
|
||||
# Cleanup the initial_setup_file
|
||||
if initial_setup_file:
|
||||
Path(initial_setup_file).unlink(missing_ok=True)
|
||||
|
||||
if returncode != 0:
|
||||
eprint("Failed to run initial setup:")
|
||||
eprint(initial_setup)
|
||||
eprint()
|
||||
eprint("Abort starting jail.")
|
||||
return returncode
|
||||
|
||||
systemd_run_additional_args = [
|
||||
f"--unit={SYMLINK_NAME}-{jail_name}",
|
||||
f"--working-directory=./{jail_path}",
|
||||
|
|
@ -605,6 +552,24 @@ def start_jail(jail_name):
|
|||
f"--directory={JAIL_ROOTFS_NAME}",
|
||||
]
|
||||
|
||||
# The systemd-nspawn manual explicitly mentions:
|
||||
# Device nodes may not be created
|
||||
# https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
|
||||
# This means docker images containing device nodes can't be pulled
|
||||
# https://github.com/moby/moby/issues/35245
|
||||
#
|
||||
# The solution is to use DevicePolicy=auto
|
||||
# https://github.com/kinvolk/kube-spawn/pull/328
|
||||
#
|
||||
# DevicePolicy=auto is the default for systemd-run and allows access to all devices
|
||||
# as long as we don't add any --property=DeviceAllow= flags
|
||||
# https://manpages.debian.org/bookworm/systemd/systemd.resource-control.5.en.html
|
||||
#
|
||||
# We can now successfully run:
|
||||
# mknod /dev/port c 1 4
|
||||
# Or pull docker images containing device nodes:
|
||||
# docker pull oraclelinux@sha256:d49469769e4701925d5145c2676d5a10c38c213802cf13270ec3a12c9c84d643
|
||||
|
||||
if config.my_getboolean("docker_compatible"):
|
||||
eprint("WARNING: DEPRECATED OPTION")
|
||||
eprint(
|
||||
|
|
@ -704,23 +669,28 @@ def start_jail(jail_name):
|
|||
"--setenv=SYSTEMD_SECCOMP=0",
|
||||
]
|
||||
|
||||
# The systemd-nspawn manual explicitly mentions:
|
||||
# Device nodes may not be created
|
||||
# https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
|
||||
# This means docker images containing device nodes can't be pulled
|
||||
# https://github.com/moby/moby/issues/35245
|
||||
#
|
||||
# The solution is to use DevicePolicy=auto
|
||||
# https://github.com/kinvolk/kube-spawn/pull/328
|
||||
#
|
||||
# DevicePolicy=auto is the default for systemd-run and allows access to all devices
|
||||
# as long as we don't add any --property=DeviceAllow= flags
|
||||
# https://manpages.debian.org/bookworm/systemd/systemd.resource-control.5.en.html
|
||||
#
|
||||
# We can now successfully run:
|
||||
# mknod /dev/port c 1 4
|
||||
# Or pull docker images containing device nodes:
|
||||
# docker pull oraclelinux@sha256:d49469769e4701925d5145c2676d5a10c38c213802cf13270ec3a12c9c84d643
|
||||
initial_setup = False
|
||||
|
||||
# If there's no machine-id, then this the first time the jail is started
|
||||
if not os.path.exists(os.path.join(jail_rootfs_path, "etc/machine-id")) and (
|
||||
initial_setup := config.my_get("initial_setup")
|
||||
):
|
||||
if not initial_setup.startswith("#!"):
|
||||
initial_setup = "#!/bin/sh\n" + initial_setup
|
||||
|
||||
initial_setup_file_jailed_path = "/root/jlmkr-initial-setup"
|
||||
initial_setup_file_host_path = os.path.abspath(
|
||||
jail_rootfs_path + initial_setup_file_jailed_path
|
||||
)
|
||||
|
||||
# Write a script file to call during initial setup
|
||||
print(initial_setup, file=open(initial_setup_file_host_path, "w"))
|
||||
stat_chmod(initial_setup_file_host_path, 0o700)
|
||||
|
||||
# Ensure the jail init system is ready before we start the initial_setup
|
||||
systemd_nspawn_additional_args += [
|
||||
"--notify-ready=yes",
|
||||
]
|
||||
|
||||
cmd = [
|
||||
"systemd-run",
|
||||
|
|
@ -755,6 +725,44 @@ def start_jail(jail_name):
|
|||
)
|
||||
)
|
||||
|
||||
return returncode
|
||||
|
||||
# Handle initial setup after jail is up and running (for the first time)
|
||||
if initial_setup:
|
||||
print("About to run the initial setup.")
|
||||
print("Waiting for networking in the jail to be ready.")
|
||||
print("Please wait (this may take 90s in case of bridge networking)...")
|
||||
returncode = exec_jail(
|
||||
jail_name,
|
||||
[
|
||||
"--",
|
||||
"systemd-run",
|
||||
f"--unit={os.path.basename(initial_setup_file_jailed_path)}",
|
||||
"--quiet",
|
||||
"--pipe",
|
||||
"--wait",
|
||||
"--service-type=exec",
|
||||
"--property=After=network-online.target",
|
||||
"--property=Wants=network-online.target",
|
||||
initial_setup_file_jailed_path,
|
||||
],
|
||||
)
|
||||
|
||||
# Cleanup the initial_setup_file_host_path
|
||||
if initial_setup_file_host_path:
|
||||
Path(initial_setup_file_host_path).unlink(missing_ok=True)
|
||||
|
||||
if returncode != 0:
|
||||
eprint("Tried to run the following commands inside the jail:")
|
||||
eprint(initial_setup)
|
||||
eprint()
|
||||
eprint(
|
||||
f"""{RED}{BOLD}Failed to run initial setup... you may want to stop and remove the jail and try again.{NORMAL}"""
|
||||
)
|
||||
return returncode
|
||||
else:
|
||||
print(f"Done with initial setup of jail {jail_name}!")
|
||||
|
||||
return returncode
|
||||
|
||||
|
||||
|
|
@ -922,11 +930,12 @@ def get_zfs_dataset(path):
|
|||
"""
|
||||
Get ZFS dataset path.
|
||||
"""
|
||||
|
||||
def clean_field(field):
|
||||
# Put back spaces which were encoded
|
||||
# https://github.com/openzfs/zfs/issues/11182
|
||||
return field.replace('\\040', ' ')
|
||||
|
||||
return field.replace("\\040", " ")
|
||||
|
||||
path = os.path.realpath(path)
|
||||
with open("/proc/mounts", "r") as f:
|
||||
for line in f:
|
||||
|
|
@ -1025,11 +1034,14 @@ def get_text_editor():
|
|||
if editor := os.environ.get(key):
|
||||
return shutil.which(editor)
|
||||
|
||||
return get_from_environ("VISUAL") \
|
||||
or get_from_environ("EDITOR") \
|
||||
or shutil.which("editor") \
|
||||
or shutil.which("/usr/bin/editor") \
|
||||
return (
|
||||
get_from_environ("VISUAL")
|
||||
or get_from_environ("EDITOR")
|
||||
or shutil.which("editor")
|
||||
or shutil.which("/usr/bin/editor")
|
||||
or "nano"
|
||||
)
|
||||
|
||||
|
||||
def interactive_config():
|
||||
config = KeyValueParser()
|
||||
|
|
@ -1517,9 +1529,7 @@ def edit_jail(jail_name):
|
|||
|
||||
jail_config_path = get_jail_config_path(jail_name)
|
||||
|
||||
returncode = subprocess.run(
|
||||
[get_text_editor(), jail_config_path]
|
||||
).returncode
|
||||
returncode = subprocess.run([get_text_editor(), jail_config_path]).returncode
|
||||
|
||||
if returncode != 0:
|
||||
eprint(f"An error occurred while editing {jail_config_path}.")
|
||||
|
|
|
|||
|
|
@ -2,4 +2,4 @@
|
|||
|
||||
## Setup
|
||||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/docker/config mydockerjail`.
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/docker/config mydockerjail`. If you want the `nvidia-container-toolkit` to be installed, ensure you set `gpu_passthrough_nvidia=1` when creating the jail.
|
||||
|
|
@ -6,10 +6,10 @@ seccomp=1
|
|||
|
||||
# Use macvlan networking to provide an isolated network namespace,
|
||||
# so docker can manage firewall rules
|
||||
# Alternatively use --network-bridge=br1 instead of --network-macvlan
|
||||
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
|
||||
# Ensure to change eno1/br1 to the interface name you want to use
|
||||
# You may want to add additional options here, e.g. bind mounts
|
||||
systemd_nspawn_user_args=--network-macvlan=eno1
|
||||
systemd_nspawn_user_args=--network-bridge=br1
|
||||
--resolv-conf=bind-host
|
||||
--system-call-filter='add_key keyctl bpf'
|
||||
|
||||
|
|
@ -29,8 +29,8 @@ release=bookworm
|
|||
|
||||
# Install docker inside the jail:
|
||||
# https://docs.docker.com/engine/install/debian/#install-using-the-repository
|
||||
# NOTE: this script will run in the host networking namespace and ignores
|
||||
# all systemd_nspawn_user_args such as bind mounts
|
||||
# Will also install the NVIDIA Container Toolkit if gpu_passthrough_nvidia=1 during initial setup
|
||||
# https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
|
||||
initial_setup=#!/usr/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
|
|
@ -43,8 +43,26 @@ initial_setup=#!/usr/bin/bash
|
|||
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
|
||||
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
|
||||
tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
|
||||
apt-get update
|
||||
apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||
|
||||
# The /usr/bin/nvidia-smi will be present when gpu_passthrough_nvidia=1
|
||||
if [ -f /usr/bin/nvidia-smi ]; then
|
||||
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o /etc/apt/keyrings/nvidia.asc
|
||||
chmod a+r /etc/apt/keyrings/nvidia.asc
|
||||
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
|
||||
sed 's#deb https://#deb [signed-by=/etc/apt/keyrings/nvidia.asc] https://#g' | \
|
||||
tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
|
||||
|
||||
apt-get update
|
||||
apt-get install -y nvidia-container-toolkit
|
||||
|
||||
nvidia-ctk runtime configure --runtime=docker
|
||||
systemctl restart docker
|
||||
fi
|
||||
|
||||
docker info
|
||||
|
||||
# You generally will not need to change the options below
|
||||
systemd_run_default_args=--property=KillMode=mixed
|
||||
|
|
|
|||
|
|
@ -8,11 +8,10 @@
|
|||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/incus/config myincusjail`.
|
||||
|
||||
Unfortunately incus doesn't want to install from the `initial_setup` script inside the config file. So we manually finish the setup by running the following after creating and starting the jail:
|
||||
We manually finish the setup by running the following after creating and starting the jail:
|
||||
|
||||
```bash
|
||||
jlmkr exec myincusjail bash -c 'apt-get -y install incus incus-ui-canonical &&
|
||||
incus admin init'
|
||||
jlmkr exec myincusjail bash -c 'incus admin init'
|
||||
```
|
||||
|
||||
Follow [First steps with Incus](https://linuxcontainers.org/incus/docs/main/tutorial/first_steps/).
|
||||
|
|
|
|||
|
|
@ -3,15 +3,16 @@ startup=0
|
|||
gpu_passthrough_intel=0
|
||||
gpu_passthrough_nvidia=0
|
||||
# Turning off seccomp filtering improves performance at the expense of security
|
||||
seccomp=1
|
||||
# TODO: don't disable seccomp but specify which syscalls should be allowed
|
||||
seccomp=0
|
||||
|
||||
# Use macvlan networking to provide an isolated network namespace,
|
||||
# so incus can manage firewall rules
|
||||
# Alternatively use --network-bridge=br1 instead of --network-macvlan
|
||||
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
|
||||
# Ensure to change eno1/br1 to the interface name you want to use
|
||||
# You may want to add additional options here, e.g. bind mounts
|
||||
# TODO: don't use --capability=all but specify only the required capabilities
|
||||
systemd_nspawn_user_args=--network-macvlan=eno1
|
||||
systemd_nspawn_user_args=--network-bridge=br1
|
||||
--resolv-conf=bind-host
|
||||
--capability=all
|
||||
--bind=/dev/fuse
|
||||
|
|
@ -36,8 +37,6 @@ release=bookworm
|
|||
|
||||
# Install incus according to:
|
||||
# https://github.com/zabbly/incus#installation
|
||||
# NOTE: this script will run in the host networking namespace and ignores
|
||||
# all systemd_nspawn_user_args such as bind mounts
|
||||
initial_setup=#!/usr/bin/bash
|
||||
set -euo pipefail
|
||||
apt-get update && apt-get -y install curl
|
||||
|
|
@ -54,6 +53,7 @@ initial_setup=#!/usr/bin/bash
|
|||
|
||||
EOF'
|
||||
apt-get update
|
||||
apt-get -y install incus incus-ui-canonical
|
||||
|
||||
# You generally will not need to change the options below
|
||||
systemd_run_default_args=--property=KillMode=mixed
|
||||
|
|
|
|||
|
|
@ -8,20 +8,7 @@
|
|||
|
||||
Check out the [config](./config) template file. You may provide it when asked during `jlmkr create` or, if you have the template file stored on your NAS, you may provide it directly by running `jlmkr create --start --config /mnt/tank/path/to/lxd/config mylxdjail`.
|
||||
|
||||
Unfortunately snapd doesn't want to install from the `initial_setup` script inside the config file. So we manually finish the setup by running the following after creating and starting the jail:
|
||||
|
||||
```bash
|
||||
# Repeat listing the jail until you see it has an IPv4 address
|
||||
jlmkr list
|
||||
|
||||
# Install packages
|
||||
jlmkr exec mylxdjail bash -c 'apt-get update &&
|
||||
apt-get install -y --no-install-recommends snapd &&
|
||||
snap install lxd'
|
||||
|
||||
```
|
||||
|
||||
Choose the `dir` storage backend during `lxd init` and answer `yes` to "Would you like the LXD server to be available over the network?"
|
||||
We manually finish the setup by running the command below after creating and starting the jail. Choose the `dir` storage backend during `lxd init` and answer `yes` to "Would you like the LXD server to be available over the network?"
|
||||
|
||||
```bash
|
||||
jlmkr exec mylxdjail bash -c 'lxd init &&
|
||||
|
|
|
|||
|
|
@ -3,11 +3,12 @@ startup=0
|
|||
gpu_passthrough_intel=0
|
||||
gpu_passthrough_nvidia=0
|
||||
# Turning off seccomp filtering improves performance at the expense of security
|
||||
seccomp=1
|
||||
# TODO: don't disable seccomp but specify which syscalls should be allowed
|
||||
seccomp=0
|
||||
|
||||
# Use macvlan networking to provide an isolated network namespace,
|
||||
# so lxd can manage firewall rules
|
||||
# Alternatively use --network-bridge=br1 instead of --network-macvlan
|
||||
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
|
||||
# Ensure to change eno1/br1 to the interface name you want to use
|
||||
# You may want to add additional options here, e.g. bind mounts
|
||||
# TODO: don't use --capability=all but specify only the required capabilities
|
||||
|
|
@ -34,12 +35,13 @@ pre_start_hook=#!/usr/bin/bash
|
|||
distro=ubuntu
|
||||
release=jammy
|
||||
|
||||
# NOTE: this script will run in the host networking namespace and ignores
|
||||
# all systemd_nspawn_user_args such as bind mounts
|
||||
initial_setup=#!/usr/bin/bash
|
||||
set -euo pipefail
|
||||
# https://discuss.linuxcontainers.org/t/snap-inside-privileged-lxd-container/13691/8
|
||||
ln -sf /bin/true /usr/local/bin/udevadm
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends snapd
|
||||
snap install lxd
|
||||
|
||||
# You generally will not need to change the options below
|
||||
systemd_run_default_args=--property=KillMode=mixed
|
||||
|
|
|
|||
|
|
@ -46,6 +46,10 @@ usermod --del-subuids 0-4294967295 --del-subgids 0-4294967295 rootless
|
|||
# Set a specific range, so it fits inside the number of available UIDs
|
||||
usermod --add-subuids 65536-131071 --add-subgids 65536-131071 rootless
|
||||
|
||||
# Add the required capabilities to the `newuidmap` and `newgidmap` binaries
|
||||
setcap cap_setuid+eip /usr/bin/newuidmap
|
||||
setcap cap_setgid+eip /usr/bin/newgidmap
|
||||
|
||||
# Check the assigned range
|
||||
cat /etc/subuid
|
||||
# Check the available range
|
||||
|
|
@ -120,4 +124,7 @@ Resources mentioning `add_key keyctl bpf`
|
|||
Resources mentioning `@keyring`
|
||||
- https://github.com/systemd/systemd/issues/17606
|
||||
- https://github.com/systemd/systemd/blob/1c62c4fe0b54fb419b875cb2bae82a261518a745/src/shared/seccomp-util.c#L604
|
||||
`@keyring` also includes `request_key` but doesn't include `bpf`
|
||||
`@keyring` also includes `request_key` but doesn't include `bpf`
|
||||
Resources mentioning `cap_setuid+eip`, `cap_setgid+eip`, `newuidmap` and `newgidmap`
|
||||
- https://github.com/containers/podman/issues/2788#issuecomment-1016301663
|
||||
- https://github.com/containers/podman/issues/12637#issuecomment-996524341
|
||||
|
|
@ -6,10 +6,10 @@ seccomp=1
|
|||
|
||||
# Use macvlan networking to provide an isolated network namespace,
|
||||
# so podman can manage firewall rules
|
||||
# Alternatively use --network-bridge=br1 instead of --network-macvlan
|
||||
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
|
||||
# Ensure to change eno1/br1 to the interface name you want to use
|
||||
# You may want to add additional options here, e.g. bind mounts
|
||||
systemd_nspawn_user_args=--network-macvlan=eno1
|
||||
systemd_nspawn_user_args=--network-bridge=br1
|
||||
--resolv-conf=bind-host
|
||||
--system-call-filter='add_key keyctl bpf'
|
||||
|
||||
|
|
@ -28,16 +28,9 @@ distro=fedora
|
|||
release=39
|
||||
|
||||
# Install podman inside the jail
|
||||
# NOTE: this script will run in the host networking namespace and ignores
|
||||
# all systemd_nspawn_user_args such as bind mounts
|
||||
initial_setup=#!/usr/bin/bash
|
||||
set -euo pipefail
|
||||
dnf -y install podman
|
||||
# Add the required capabilities to the `newuidmap` and `newgidmap` binaries
|
||||
# https://github.com/containers/podman/issues/2788#issuecomment-1016301663
|
||||
# https://github.com/containers/podman/issues/12637#issuecomment-996524341
|
||||
setcap cap_setuid+eip /usr/bin/newuidmap
|
||||
setcap cap_setgid+eip /usr/bin/newgidmap
|
||||
|
||||
# You generally will not need to change the options below
|
||||
systemd_run_default_args=--property=KillMode=mixed
|
||||
|
|
|
|||
Loading…
Reference in New Issue