b849cb787c
Updated config templates to benefit from initial_setup running after jail has fully started. Added conditional nvidia-container-toolkit install during initial_setup. Config templates now default to bridge networking. |
||
|---|---|---|
| .. | ||
| README.md | ||
| config | ||
README.md
Ubuntu LXD Jail Template
Disclaimer
Experimental. Using LXD in this setup hasn't been extensively tested and has known issues.
Setup
Check out the config template file. You may provide it when asked during jlmkr create or, if you have the template file stored on your NAS, you may provide it directly by running jlmkr create --start --config /mnt/tank/path/to/lxd/config mylxdjail.
We manually finish the setup by running the command below after creating and starting the jail. Choose the dir storage backend during lxd init and answer yes to "Would you like the LXD server to be available over the network?"
jlmkr exec mylxdjail bash -c 'lxd init &&
snap set lxd ui.enable=true &&
systemctl reload snap.lxd.daemon'
Then visit the lxd GUI inside the browser https://0.0.0.0:8443. To find out which IP address to use instead of 0.0.0.0, check the IP address for your jail with jlmkr list.
Known Issues
Instance creation failed
LXD no longer has access to the LinuxContainers image server.
Failed getting remote image info: Failed getting image: The requested image couldn't be found for fingerprint "ubuntu/focal/desktop"
SCALE Virtual Machines
Using LXD in the jail will cause the following error when starting a VM from the TrueNAS SCALE web GUI:
[EFAULT] internal error: process exited while connecting to monitor: Could not access KVM kernel module: Permission denied 2024-02-16T14:40:14.886658Z qemu-system-x86_64: -accel kvm: failed to initialize kvm: Permission denied
A reboot will resolve the issue (until you start the LXD jail again).
ZFS Issues
If you create a new dataset on your pool (e.g. tank) called lxd from the TrueNAS SCALE web GUI and tell LXD to use it during lxd init, then you will run into issues. Firstly you'd have to run apt-get install -y --no-install-recommends zfsutils-linux inside the jail to install the ZFS userspace utils and you've have to add --bind=/dev/zfs to the systemd_nspawn_user_args in the jail config. By mounting /dev/zfs into this jail, it will have total control of the storage on the host!
But then SCALE doesn't seem to like the ZFS datasets created by LXD. I get the following errors when browsing the sub-datasets:
[EINVAL] legacy: path must be absolute
[EFAULT] Failed retreiving USER quotas for tank/lxd/virtual-machines
As long as you don't operate on these datasets in the SCALE GUI this may not be a real problem...
However, creating an LXD VM doesn't work with the ZFS storage backend (creating a container works though):
Failed creating instance from image: Could not locate a zvol for tank/lxd/images/1555b13f0e89bfcf516bd0090eee6f73a0db5f4d0d36c38cae94316de82bf817.block
Could this be the same issue as Instance creation failed?
More info
Refer to the Incus README as a lot of it applies to LXD too.