Adding k3s example (#197)

* Adding k3s example

---------

Co-authored-by: Javy de Koning <javydekoning@gmail.com>
Co-authored-by: Jip-Hop <2871973+Jip-Hop@users.noreply.github.com>

README.md

@@ -1,6 +1,6 @@
 # Jailmaker
 
-Persistent Linux 'jails' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts.
+Persistent Linux 'jails' on TrueNAS SCALE to install software (k3s, docker, portainer, podman, etc.) with full access to all files via bind mounts.
 
 ## Video Tutorial
 

templates/k3s/README.md

@@ -0,0 +1,5 @@
+# Debian k3s Jail Template
+
+## Setup
+
+Check out the [config](./config) template file. You may provide it when asked during `./jlmkr.py create` or, if you have the template file stored on your NAS, you may provide it directly by running `./jlmkr.py create --start --config /mnt/tank/path/to/k3s/config myk3sjail`.

templates/k3s/config

@@ -0,0 +1,85 @@
+startup=0
+gpu_passthrough_intel=0
+gpu_passthrough_nvidia=0
+# Turning off seccomp filtering improves performance at the expense of security
+seccomp=1
+
+# Use macvlan networking to provide an isolated network namespace,
+# so kubernetes can manage firewall rules
+# Alternatively use --network-macvlan=eno1 instead of --network-bridge
+# Ensure to change eno1/br1 to the interface name you want to use
+# You may want to add additional options here, e.g. bind mounts
+# For k3s we allow the use of keyrings and cgroups, 
+# You should add capability perf_event_open for tools like intel_gpu_top.
+# A bind mount is used to give k3s access to circular message buffer (/dev/kmsg)
+systemd_nspawn_user_args=--network-bridge=br1
+    --resolv-conf=bind-host
+    --system-call-filter='add_key keyctl bpf'
+    --bind=/dev/kmsg
+    # You can mount additional paths/devices like this:
+    # --bind=/dev/ttyUSB0
+
+# Script to run on the HOST before starting the jail
+# Load kernel module and config kernel settings required for k8s/containerd
+pre_start_hook=#!/usr/bin/bash
+    set -euo pipefail
+    echo 'PRE_START_HOOK'
+    # Set kernel parameters
+    # Enable IP forwarding
+    echo 1 > /proc/sys/net/ipv4/ip_forward
+    # Ensure that bridge traffic is processed by iptables (if using br nw)
+    echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
+    echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
+    # Set memory overcommit - needed for k3s kubelet
+    echo 1 > /proc/sys/vm/overcommit_memory
+    # Optional, increase inotify instances and watches. May be needed when
+    # running many apps
+    echo 1280 > /proc/sys/fs/inotify/max_user_instances
+    echo 655360 > /proc/sys/fs/inotify/max_user_watches
+    # Increase max tracked connections in conntrack
+    echo 196608 > /proc/sys/net/netfilter/nf_conntrack_max
+    # required for bridging and filtering network traffic
+    modprobe br_netfilter
+    # used for container storage
+    modprobe overlay
+    # enable nat and packet filter modules
+    modprobe iptable_nat
+    modprobe iptable_filter
+
+# Only used while creating the jail
+distro=debian
+release=bookworm
+
+# Install k3s, dependencies, helm inside jail 
+# https://docs.k3s.io/quick-start
+initial_setup=#!/usr/bin/bash
+    set -euo pipefail
+
+    # Install dependencies
+    apt-get update && apt-get install curl jq git -y
+
+    # Setup helm and k3s
+    curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
+    curl -sfL https://get.k3s.io | \
+    INSTALL_K3S_CHANNEL=latest sh -s - server --cluster-init
+    kubectl version
+
+    # Optional: Enable Intel HW Acceleration for Plex, Jellyfin, Frigate etc. 
+    # Uncomment below line 
+    # apt-get -y install {va-driver-all,ocl-icd-libopencl1,intel-opencl-icd,vainfo,intel-gpu-tools}
+
+# You generally will not need to change the options below
+systemd_run_default_args=--property=KillMode=mixed
+    --property=Type=notify
+    --property=RestartForceExitStatus=133
+    --property=SuccessExitStatus=133
+    --property=Delegate=yes
+    --property=TasksMax=infinity
+    --collect
+    --setenv=SYSTEMD_NSPAWN_LOCK=0
+
+systemd_nspawn_default_args=--keep-unit
+    --quiet
+    --boot
+    --bind-ro=/sys/module
+    --inaccessible=/sys/module/apparmor