For some time now I have being using KVM to test Ansible playbooks I develop to automate some machine bootstrapping and other routine tasks, in a “infrastructure as a code all the things” mindset. But reading some posts on the Reddit self-host forum I have being reading many people mentioning a container technology called LXD, and I kind of ignored it since I already use Docker for my container needs. As a curiosity I have researched a little more on it and discovered LXD can be good not as a replacement for Docker, but for KVM all along! With LXD, I can simply use full-blown linux distributions as I used on KVM, with the added benefits of performance an isolated process has. Think on Docker as for application containers and LXD for system (SO) containers. The official website of the project is https://linuxcontainers.org/. The LXD archwiki page has many useful tips, including how to use graphical (Xorg/Wayland) apps inside LXD containers.

Features

• They “weight” between VMs and Application Containers (e.g. Docker) - VMs are heavier and Application Containers the lightest. LXC/LXD are the middle ground.
• “Machine containers” - A full operating system containerized.
• Can have access to hardware on the containerized OS
• VMs reserve memory for a VM on boot. On LXC, they allocate resources dinamically, but you can configure limits.memory and limits.cpu (if you do not do that, all the machine resources are available to them).
• You can make clusters of LXD hosts (you can move a container from one host to another)
• You can snapshot/restore containers
• You can nest LXD (have LXD inside the containerized OS) - this can be useful e.g. on VPSs
• They use the kernel of the LXD daemon host
• You can ping between 2 or more containers using their DNS name (which is container-name.lxd)
• Network aware and all interactions go through a simple REST API, making it possible to remotely interact with containers on remote systems.

How to install on Arch Linux

1. Install snap support: $ yay -S snapd –noconfirm $ sudo systemctl start snapd.socket $ sudo systemctl enable snapd.socket $ sudo ln -s /var/lib/snapd/snap /snap # this to enable classic snap support Then, the machine has to be rebooted for snap packages to be found on the system.
2. Install LXD: $ sudo snap install lxd Then, add your user to the lxd linux group (/etc/group)
3. Run the script to create the lxd profiles, so that the container use cloud-init to import ssh keys and make other setups: $ cd /storage/src/devops/shellscripts/lxd/centos &&
   sudo ./setup-lxd-centos-guest-profile-with-cloud-init.sh
4. Create a bridge network to be used by all lxd containers: $ lxc network create lxdbr0 $ lxc network edit lxdbr0 # then, change to the appropriate range you want e.g. 10.0.10.1/24 Then, attach the created network to all containers: $ lxc network attach-profile lxdbr0 default eth0
5. Create a storage pool to hold your images: $ lxc storage create storage_pool dir source=/lxd
6. Finish configuration: $ lxd init
7. Add a repo containing distros with cloud images: $ lxc remote add cloud-images-from-nlogn images.nlogn.org –public
8. Launch a centos7 cloud-init enabled container: $ lxc launch cloud-images-from-nlogn:centos/7 my-centos -p centos.cloud-init.config -t c2-m2 -s storage_pool -n lxdbr0 -v –debug That will launch a centos 7 container with 2 cpus and 2 GB of RAM. To launch a ubuntu 18.04 cloud-init enabled container (remember to lxc copy the centos cloud-init config to a ubuntu one): $ lxc launch ubuntu:7b5 my-ubuntu-01 -p ubuntu.cloud-init.config -t c2-m2 -s storage_pool -n lxdbr0 -v

Create a image based on an existing container:

E.g., in the case of a CentOS 7 container, enter it, install cloud-init and generate another image with those changes: $ lxc exec my-centos /bin/bash $ yum upgrade &&
yum install cloud-init -y &&
systemctl enable cloud-init-local.service &&
systemctl enable cloud-init.service &&
systemctl enable cloud-config.service &&
systemctl enable cloud-final.service; $ # go to another terminal and run the commands below $ lxc stop my-centos $ lxc publish my-centos –alias centos-7-with-cloud-init $ lxc launch centos-7-with-cloud-init my-centos-ci -p centos.cloud-init.config -t c2-m2 -s storage_pool -n lxdbr0 -v –debug $ lxc exec my-centos-ci /bin/bash $ # check the logs to see if all went OK $ more /var/log/cloud-init.log

Random tidbits about LXD

• To convert a qcow2 image (or even physical machine) to LXD, it is enough to extract the disk rootfs.
• LXD 3.0 supports clustering machines having the LXD daemon (which allow, between other benefits, moving a container from one to another).
• When using cloud provider, e.g. DigitalOcean, I can setup an LXD daemon there on a machine and have many containerized machines on it (so I can play with 3 clusters Kubernetes without needing to spin 3 droplets, e.g.).
• It is best installed through Ubuntu’s snaps (note that Arch Linux e.g. supports snaps). A snap is the best (sometimes only) way to get the most recent version of LXD.
• You can configure a machine exactly like a AWS EC2 instance, or GCP ones. You basically say the amount or cpu or memory it will use. More on that here .
• There are arch and alpine linux container images available. You can get them here: http://us.images.linuxcontainers.org/

Cheatsheet

my navi cheatsheet

References

https://stgraber.org/2016/03/30/lxd-2-0-image-management-512/ https://blog.ubuntu.com/2017/02/25/howto-automatically-import-your-public-ssh-keys-into-lxd-instances https://github.com/lxc/lxd/blob/master/doc/image-handling.md https://blog.ubuntu.com/2017/02/14/network-management-with-lxd-2-3 https://github.com/felix-engelmann/lxd-image-converter/blob/master/README.md