Cloned2Preserve
/
jailmaker
mirror of https://github.com/Jip-Hop/jailmaker.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove deprecated config options and commands 

Removed commands:
- install
Removed config options:
- gpu_passthrough
- docker_compatible
This commit is contained in:
Jip-Hop 2024-05-28 19:44:54 +02:00
parent 6c1d49cf77
commit 8c58673692
3 changed files with 36 additions and 243 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
README.md
View File

@ -37,7 +37,7 @@ Failed to initialize PMU! (Operation not permitted)
perf_event_open({type=0x10 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER7, config=0x100002, sample_period=0, sample_type=0, read_format=PERF_FORMAT_TOTAL_TIME_ENABLED|PERF_FORMAT_GROUP, precise_ip=0 /* arbitrary skid */, use_clockid=1, ...}, -1, 0, -1, 0) = -1 EPERM (Operation not permitted)
write(2, "Failed to initialize PMU! (Opera"..., 52Failed to initialize PMU! (Operation not permitted)
```
The syscall that needs to be added to the `--system-call-filter` option in the jlmkr config in this case would be `perf_event_open`. You may need to run strace multiple times.
The syscall that needs to be added to the `--system-call-filter` option in the jlmkr config in this case would be `perf_event_open`. You may need to run strace multiple times.
Seccomp is important for security, but as a last resort can be disabled by setting `seccomp=0` in the jail config.
@ -49,41 +49,36 @@ Beginning with 24.04 (Dragonfish), TrueNAS SCALE includes the systemd-nspawn con
cd /mnt/mypool/jailmaker
curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.py
chmod +x jlmkr.py
./jlmkr.py install
```
The `jlmkr.py` script (and the jails + config it creates) are now stored on the `jailmaker` dataset and will survive updates of TrueNAS SCALE. If the automatically created `jails` directory is also a ZFS dataset (which is true for new users), then the `jlmkr.py` script will automatically create a new dataset for every jail created. This allows you to snapshot individual jails. For legacy users (where the `jails` directory is not a dataset) each jail will be stored in a plain directory.
A symlink has been created so you can call `jlmkr` from anywhere (unless the boot pool is readonly, which is the default since SCALE 24.04). Additionally shell aliases have been setup, so you can still call `jlmkr` in an interactive shell (even if the symlink couldn't be created).
After an update of TrueNAS SCALE the symlink will be lost (but the shell aliases will remain). To restore the symlink, just run `./jlmkr.py install` again or use [the `./jlmkr.py startup` command](#startup-jails-on-boot).
## Usage
### Create Jail
Creating jail with the default settings is as simple as:
Creating a jail with the default settings is as simple as:
```shell
jlmkr create --start myjail
./jlmkr.py create --start myjail
```
You may also specify a path to a config template, for a quick and consistent jail creation process.
```shell
jlmkr create --start --config /path/to/config/template myjail
./jlmkr.py create --start --config /path/to/config/template myjail
```
Or you can override the default config by using flags. See `jlmkr create --help` for the available options. Anything passed after the jail name will be passed to `systemd-nspawn` when starting the jail. See the `systemd-nspawn` manual for available options, specifically [Mount Options](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Mount_Options) and [Networking Options](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Networking_Options) are frequently used.
Or you can override the default config by using flags. See `./jlmkr.py create --help` for the available options. Anything passed after the jail name will be passed to `systemd-nspawn` when starting the jail. See the `systemd-nspawn` manual for available options, specifically [Mount Options](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Mount_Options) and [Networking Options](https://manpages.debian.org/bookworm/systemd-container/systemd-nspawn.1.en.html#Networking_Options) are frequently used.
```shell
jlmkr create --start --distro=ubuntu --release=jammy myjail --bind-ro=/mnt
./jlmkr.py create --start --distro=ubuntu --release=jammy myjail --bind-ro=/mnt
```
If you omit the jail name, the create process is interactive. You'll be presented with questions which guide you through the process.
```shell
jlmkr create
./jlmkr.py create
```
After answering some questions you should have created your first jail (and it should be running if you chose to start it after creating)!
@ -92,16 +87,15 @@ After answering some questions you should have created your first jail (and it s
```shell
# Call startup using the absolute path to jlmkr.py
# The jlmkr shell alias doesn't work in Init/Shutdown Scripts
/mnt/mypool/jailmaker/jlmkr.py startup
```
In order to start jails automatically after TrueNAS boots, run `/mnt/mypool/jailmaker/jlmkr.py startup` as Post Init Script with Type `Command` from the TrueNAS web interface. This creates the `jlmkr` symlink (if possible), as well as start all the jails with `startup=1` in the config file.
In order to start jails automatically after TrueNAS boots, run `/mnt/mypool/jailmaker/jlmkr.py startup` as Post Init Script with Type `Command` from the TrueNAS web interface. This will start all the jails with `startup=1` in the config file.
### Start Jail
```shell
jlmkr start myjail
./jlmkr.py start myjail
```
### List Jails
@ -109,7 +103,7 @@ jlmkr start myjail
See list of jails (including running, startup state, GPU passthrough, distro, and IP).
```shell
jlmkr list
./jlmkr.py list
```
### Execute Command in Jail
@ -117,41 +111,41 @@ jlmkr list
You may want to execute a command inside a jail, for example manually from the TrueNAS shell, a shell script or a CRON job. The example below executes the `env` command inside the jail.
```shell
jlmkr exec myjail env
./jlmkr.py exec myjail env
```
This example executes bash inside the jail with a command as additional argument.
```shell
jlmkr exec myjail bash -c 'echo test; echo $RANDOM;'
./jlmkr.py exec myjail bash -c 'echo test; echo $RANDOM;'
```
### Edit Jail Config
```shell
jlmkr edit myjail
./jlmkr.py edit myjail
```
Once you've created a jail, it will exist in a directory inside the `jails` dir next to `jlmkr.py`. For example `/mnt/mypool/jailmaker/jails/myjail` if you've named your jail `myjail`. You may edit the jail configuration file using the `jlmkr edit myjail` command. This opens the config file in your favorite editor, as determined by following [Debian's guidelines](https://www.debian.org/doc/debian-policy/ch-customized-programs.html#editors-and-pagers) on the matter. You'll have to stop the jail and start it again with `jlmkr` for these changes to take effect.
Once you've created a jail, it will exist in a directory inside the `jails` dir next to `jlmkr.py`. For example `/mnt/mypool/jailmaker/jails/myjail` if you've named your jail `myjail`. You may edit the jail configuration file using the `./jlmkr.py edit myjail` command. This opens the config file in your favorite editor, as determined by following [Debian's guidelines](https://www.debian.org/doc/debian-policy/ch-customized-programs.html#editors-and-pagers) on the matter. You'll have to stop the jail and start it again with `jlmkr` for these changes to take effect.
### Remove Jail
Delete a jail and remove it's files (requires confirmation).
```shell
jlmkr remove myjail
./jlmkr.py remove myjail
```
### Stop Jail
```shell
jlmkr stop myjail
./jlmkr.py stop myjail
```
### Restart Jail
```shell
jlmkr restart myjail
./jlmkr.py restart myjail
```
### Jail Shell
@ -159,13 +153,13 @@ jlmkr restart myjail
Switch into the jail's shell.
```shell
jlmkr shell myjail
./jlmkr.py shell myjail
```
### Jail Status
```shell
jlmkr status myjail
./jlmkr.py status myjail
```
### Jail Logs
@ -173,7 +167,7 @@ jlmkr status myjail
View a jail's logs.
```shell
jlmkr log myjail
./jlmkr.py log myjail
```
### Additional Commands

2
docs/compatibility.md
View File

@ -4,7 +4,7 @@
| | |
|---|---|
|TrueNAS Core|❌|
|TrueNAS 22.12|✅|
|TrueNAS 22.12|❌|
|TrueNAS 23.10|✅|
|TrueNAS 24.04|✅|

231
jlmkr.py
View File

@ -13,8 +13,6 @@ import argparse
import configparser
import contextlib
import ctypes
import errno
import glob
import hashlib
import io
import json
@ -38,8 +36,6 @@ from textwrap import dedent
DEFAULT_CONFIG = """startup=0
gpu_passthrough_intel=0
gpu_passthrough_nvidia=0
# The docker_compatible option is deprecated and will be removed in a future release
docker_compatible=0
# Turning off seccomp filtering improves performance at the expense of security
seccomp=1
@ -128,7 +124,7 @@ SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_NAME = os.path.basename(SCRIPT_PATH)
SCRIPT_DIR_PATH = os.path.dirname(SCRIPT_PATH)
COMMAND_NAME = os.path.basename(__file__)
SYMLINK_NAME = "jlmkr"
SHORTNAME = "jlmkr"
# Only set a color if we have an interactive tty
if sys.stdout.isatty():
@ -337,7 +333,7 @@ def passthrough_nvidia(
):
jail_rootfs_path = get_jail_rootfs_path(jail_name)
ld_so_conf_path = Path(
os.path.join(jail_rootfs_path), f"etc/ld.so.conf.d/{SYMLINK_NAME}-nvidia.conf"
os.path.join(jail_rootfs_path), f"etc/ld.so.conf.d/{SHORTNAME}-nvidia.conf"
)
if not gpu_passthrough_nvidia:
@ -480,7 +476,7 @@ def status_jail(jail_name):
"""
# Alternatively `machinectl status jail_name` could be used
return subprocess.run(
["systemctl", "status", f"{SYMLINK_NAME}-{jail_name}"]
["systemctl", "status", f"{SHORTNAME}-{jail_name}"]
).returncode
@ -489,7 +485,7 @@ def log_jail(jail_name):
Show the log file of the jail with given name.
"""
return subprocess.run(
["journalctl", "-u", f"{SYMLINK_NAME}-{jail_name}"]
["journalctl", "-u", f"{SHORTNAME}-{jail_name}"]
).returncode
@ -574,7 +570,7 @@ def start_jail(jail_name):
seccomp = config.my_getboolean("seccomp")
systemd_run_additional_args = [
f"--unit={SYMLINK_NAME}-{jail_name}",
f"--unit={SHORTNAME}-{jail_name}",
f"--working-directory=./{jail_path}",
f"--description=My nspawn jail {jail_name} [created with jailmaker]",
]
@ -602,47 +598,6 @@ def start_jail(jail_name):
# Or pull docker images containing device nodes:
# docker pull oraclelinux@sha256:d49469769e4701925d5145c2676d5a10c38c213802cf13270ec3a12c9c84d643
if config.my_getboolean("docker_compatible"):
eprint("WARNING: DEPRECATED OPTION")
eprint(
"The `docker_compatible` option is deprecated and will be removed in a future release."
)
eprint("Please refer to the recommended way to run docker in a jail:")
eprint("https://github.com/Jip-Hop/jailmaker/tree/main/templates/docker")
# Enable ip forwarding on the host (docker needs it)
print(1, file=open("/proc/sys/net/ipv4/ip_forward", "w"))
# Load br_netfilter kernel module and enable bridge-nf-call to fix warning when running docker info:
# WARNING: bridge-nf-call-iptables is disabled
# WARNING: bridge-nf-call-ip6tables is disabled
#
# If we are using Apps then this should already be enabled
# May cause "guest container traffic to be blocked by iptables rules that are intended for the host"
# https://unix.stackexchange.com/q/720105/477308
# https://github.com/moby/moby/issues/24809
# https://docs.oracle.com/en/operating-systems/oracle-linux/docker/docker-KnownIssues.html#docker-issues
# https://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf
# https://serverfault.com/questions/963759/docker-breaks-libvirt-bridge-network
if subprocess.run(["modprobe", "br_netfilter"]).returncode == 0:
print(1, file=open("/proc/sys/net/bridge/bridge-nf-call-iptables", "w"))
print(1, file=open("/proc/sys/net/bridge/bridge-nf-call-ip6tables", "w"))
else:
eprint(
dedent(
"""
Failed to load br_netfilter kernel module."""
)
)
print("The `docker_compatible` option disables seccomp filtering...")
seccomp = False
# Add additional flags required for docker
systemd_nspawn_additional_args += [
"--capability=all",
]
# Add hooks to execute commands on the host before/after starting and after stopping a jail
add_hook(
jail_path,
@ -665,20 +620,8 @@ def start_jail(jail_name):
"ExecStopPost",
)
# Legacy gpu_passthrough config setting
if config.my_getboolean("gpu_passthrough", False):
eprint("WARNING: DEPRECATED OPTION")
eprint(
"The `gpu_passthrough` option is deprecated and will be removed in a future release."
)
eprint(
"Please use `gpu_passthrough_intel` and/or `gpu_passthrough_nvidia` instead."
)
gpu_passthrough_intel = True
gpu_passthrough_nvidia = True
else:
gpu_passthrough_intel = config.my_getboolean("gpu_passthrough_intel")
gpu_passthrough_nvidia = config.my_getboolean("gpu_passthrough_nvidia")
gpu_passthrough_intel = config.my_getboolean("gpu_passthrough_intel")
gpu_passthrough_nvidia = config.my_getboolean("gpu_passthrough_nvidia")
passthrough_intel(gpu_passthrough_intel, systemd_nspawn_additional_args)
passthrough_nvidia(
@ -759,7 +702,7 @@ def start_jail(jail_name):
f"""
Failed to start jail {jail_name}...
In case of a config error, you may fix it with:
{SYMLINK_NAME} edit {jail_name}
{COMMAND_NAME} edit {jail_name}
"""
)
)
@ -1253,7 +1196,7 @@ def interactive_config():
print(
dedent(
f"""
The `{COMMAND_NAME} startup` command can automatically ensure {COMMAND_NAME} is installed properly and start a selection of jails.
The `{COMMAND_NAME} startup` command can automatically start a selection of jails.
This comes in handy when you want to automatically start multiple jails after booting TrueNAS SCALE (e.g. from a Post Init Script).
"""
)
@ -1324,7 +1267,7 @@ def create_jail(**kwargs):
# TODO: fallback to default values for e.g. distro and release if they are not in the config file
print(f"Creating jail {jail_name} from config template {jail_config_path}.")
if jail_config_path not in config.read(jail_config_path):
eprint(f"Failed to read config config template {jail_config_path}.")
eprint(f"Failed to read config template {jail_config_path}.")
return 1
else:
print(f"Creating jail {jail_name} with default config.")
@ -1334,7 +1277,6 @@ def create_jail(**kwargs):
for option in [
"distro",
"docker_compatible",
"gpu_passthrough_intel",
"gpu_passthrough_nvidia",
"release",
@ -1358,9 +1300,9 @@ def create_jail(**kwargs):
print(
dedent(
f"""
TIP: Run `{SYMLINK_NAME} create` without any arguments for interactive config.
TIP: Run `{COMMAND_NAME} create` without any arguments for interactive config.
Or use CLI args to override the default options.
For more info, run: `{SYMLINK_NAME} create --help`
For more info, run: `{COMMAND_NAME} create --help`
"""
)
)
@ -1739,14 +1681,8 @@ def list_jails():
config = parse_config_file(get_jail_config_path(jail_name))
if config:
jail["startup"] = config.my_getboolean("startup")
# TODO: remove gpu_passthrough in future release
if config.my_getboolean("gpu_passthrough", False):
jail["gpu_intel"] = True
jail["gpu_nvidia"] = True
else:
jail["gpu_intel"] = config.my_getboolean("gpu_passthrough_intel")
jail["gpu_nvidia"] = config.my_getboolean("gpu_passthrough_nvidia")
jail["gpu_intel"] = config.my_getboolean("gpu_passthrough_intel")
jail["gpu_nvidia"] = config.my_getboolean("gpu_passthrough_nvidia")
if jail_name in running_machines:
machine = running_machines[jail_name]
@ -1795,133 +1731,7 @@ def list_jails():
return 0
def replace_or_add_string(file_path, regex, replacement_string):
"""
Replace all occurrences of a regular expression in a file with a given string.
Add the string to the end of the file if regex doesn't match.
Args:
file_path (str): The path to the file.
regex (str): The regular expression to search for.
replacement_string (str): The string to replace the matches with.
"""
with open(file_path, "a+") as f:
f.seek(0)
updated = False
found = False
new_text = ""
replacement_line = f"{replacement_string}\n"
for line in f:
if not re.match(regex, line):
new_text += line
continue
found = True
new_text += replacement_line
if replacement_line != line:
updated = True
if not new_text.strip():
# In case of an empty file just write the replacement_string
new_text = replacement_line
updated = True
elif not found:
# Add a newline to the end of the file in case it's not there
if not new_text.endswith("\n"):
new_text += "\n"
# Then add our replacement_string to the end of the file
new_text += replacement_line
updated = True
# Only overwrite in case there are change to the file
if updated:
f.seek(0)
f.truncate()
f.write(new_text)
return True
return False
def install_jailmaker():
# Check if command exists in path
if shutil.which("systemd-nspawn"):
print("systemd-nspawn is already installed.")
else:
print("Installing jailmaker dependencies...")
original_permissions = {}
print(
"Temporarily enable apt and dpkg (if not already enabled) to install systemd-nspawn."
)
# Make /bin/apt* and /bin/dpkg* files executable
for file in glob.glob("/bin/apt*") + (glob.glob("/bin/dpkg*")):
original_permissions[file] = os.stat(file).st_mode
stat_chmod(file, 0o755)
subprocess.run(["apt-get", "update"], check=True)
subprocess.run(["apt-get", "install", "-y", "systemd-container"], check=True)
# Restore original permissions
print("Restore permissions of apt and dpkg.")
for file, original_permission in original_permissions.items():
stat_chmod(file, original_permission)
symlink = f"/usr/local/sbin/{SYMLINK_NAME}"
if os.path.lexists(symlink) and not os.path.islink(symlink):
print(
f"Unable to create symlink at {symlink}. File already exists but is not a symlink."
)
# Check if the symlink is already pointing to the desired destination
elif os.path.realpath(symlink) != SCRIPT_PATH:
try:
Path(symlink).unlink(missing_ok=True)
os.symlink(SCRIPT_PATH, symlink)
print(f"Created symlink {symlink} to {SCRIPT_PATH}.")
except OSError as e:
if e.errno != errno.EROFS:
raise e
print(
f"Cannot create symlink because {symlink} is on a readonly filesystem."
)
alias = f"alias jlmkr='\"{SCRIPT_PATH}\"' # managed by jailmaker"
alias_regex = re.compile(r"^\s*alias jlmkr=.*# managed by jailmaker\s*")
shell_env = os.getenv("SHELL")
for shell_type in ["bash", "zsh"]:
file = "/root/.bashrc" if shell_type == "bash" else "/root/.zshrc"
if replace_or_add_string(file, alias_regex, alias):
print(f"Created {shell_type} alias {SYMLINK_NAME}.")
if shell_env.endswith(shell_type):
print(
f"Please source {file} manually for the {SYMLINK_NAME} alias to become effective immediately."
)
else:
print(f"The {shell_type} alias {SYMLINK_NAME} is already present.")
print("Done installing jailmaker.")
return 0
def startup_jails():
returncode = install_jailmaker()
if returncode != 0:
eprint("Failed to install jailmaker. Abort startup.")
return returncode
start_failure = False
for jail_name in get_all_jail_names():
config = parse_config_file(get_jail_config_path(jail_name))
@ -2010,11 +1820,6 @@ def main():
help="list available images to create jails from",
func=run_lxc_download_script,
),
dict(
name="install",
help="install jailmaker dependencies and create symlink",
func=install_jailmaker,
),
dict(
name="list", #
help="list jails",
@ -2048,7 +1853,7 @@ def main():
),
dict(
name="startup",
help=f"install {SYMLINK_NAME} and startup selected jails",
help="startup selected jails",
func=startup_jails,
),
dict(
@ -2097,12 +1902,6 @@ def main():
choices=[0, 1],
help=f"start this jail when running: {SCRIPT_NAME} startup",
)
commands["create"].add_argument(
"--docker_compatible", #
type=int,
choices=[0, 1],
help="DEPRECATED",
)
commands["create"].add_argument(
"--seccomp", #
type=int,