(reference: https://wiki.archlinux.org/index.php/Libvirt)

1. Installing qemu supporting KVM and libvirt on Arch Linux

yay -Syy --noconfirm qemu qemu-arch-extra qemu-block-gluster qemu-block-iscsi qemu-block-rbd libvirt gnu-netcat dnsmasq ebtables bridge-utils virt-manager virt-viewer gnome-boxes ovmf create_ap haveged libvirt provides a CLI client called virsh, which is useful to create and configure new VMs.

2. Concepts (from libvirt / KVM)

• Domain: it is a virtual machine.
• Storage pool: it is where storage volumes (virtual disks, virtual machine images) are kept. On a system-level session (sudo virsh) they are at /var/lib/libvirt/images. On a user session (virsh) they are kept at $HOME/VirtualMachines.
• Volume: it is a virtual machine disk. They must be created inside a storage pool.

3. Configuring

3.1. Adding your user to the libvirt group, so it can manage VMs:

Add your user to the libvirt and kvm groups at /etc/group and reboot the machine.

3.2. Load the appropriate kernel modules:

Remove:

$ sudo rmmod kvm kvm_intel virtio

Add:

$ sudo modprobe kvm kvm_intel virtio

Add this modules to be started on boot:

$ sudo vim /etc/modules-load.d/custom.conf
kvm
kvm_intel
virtio

Then, reboot the machine, to make sure it worked: $ sudo lsmod | sort

3.3. Start the apropriate systemd services:

$ sudo systemctl enable libvirtd virtlogd virtlockd The command above will enable all related libvirt services at boot. If you don’t want to restart the machine yet, you must start them separatedly: $ sudo systemctl start libvirtd virtlogd virtlockd

3.4. Edit qemu.conf, specifying the right group for it:

$ sudo vim /etc/libvirt/qemu.conf There, change group = "78" to group = "kvm".

3.5. Testing

To test if libvirt is working properly on a system level: $ virsh -c qemu:///system This mode is the priviledged mode. It allows you to create bridges, e.g., which normal mode does not. To create vms or manage networks on this mode, use sudo virsh. This document will from now on assume this mode. To test if libvirt is working properly for a user-session: $ virsh -c qemu:///session This mode is the normal mode. It does NOT allow you to create bridges, e.g. To create vms or manage networks on this mode, use virsh only.

3.6 Networking

For networking, create a bridge network with NAT, to allow a different range of IPs from the host network for the VMs: (reference: https://jmutai.com/2017/02/14/managing-kvm-network-interfaces-with-virsh-nmcli-and-brctl-in-linux/)

• Create a xml containing the bridge configuration:

$ vim kvm-nat-network.xml
<network>
<name>br1</name>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='br1' stp='on' delay='0'/>
<ip address='10.0.5.1' netmask='255.255.255.0'>
<dhcp>
<range start='10.0.5.10' end='10.0.5.100'/>
</dhcp>
</ip>
</network>

• Define the network from the XML file, without starting it: $ sudo virsh net-define kvm-nat-network.xml
• To see all available networks: $ sudo virsh net-list –all
• To start a network (by name, which is the same as on the config file above: $ sudo virsh net-start br1 You should then be able to ping the host with ping 10.0.5.1.
• To auto-start a network (every time with kvm): $ sudo virsh net-autostart br1
• To get network UUID from its name: $ sudo virsh net-uuid br1
• Confirm the bridge was successfully created: $ sudo brctl show br1
• Show the ip assigned to the interface: $ sudo ip addr show dev br1
• Attach the interface to a running domain: $ sudo virsh attach-interface –domain [vm-name] –type bridge –source [network-interface] –model virtio –config –live e.g. $ sudo virsh attach-interface –domain pxe –type bridge –source br1 –model virtio –config –live
• List interfaces attached to a domain:

$ sudo virsh domiflist pxe
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      bridge     virbr0     virtio      52:54:00:e9:ad:17
vnet1      bridge     br1        virtio      52:54:00:47:2f:eb

• Dettach interface from a running domain: $ sudo virsh detach-interface –domain [vm-name] –type bridge –mac [network-interface-mac-address-as-shown-by-virsh-domiflist] –config e.g. $ sudo virsh detach-interface –domain pxe –type bridge –mac 52:54:00:47:2f:eb –config
• Removing a network: $ sudo virsh net-destroy br1 $ sudo virsh net-undefine br1 After those commands, it should not appear anymore at sudo virsh net-list --all. You can also use sudo brctl show br1 to confirm it does not exist anymore. :)

4. Cheatsheet

Domains:

List all domains:

$ sudo virsh list –all

Create new domain:

From an ISO file

$ sudo virt-install  \
--name arch-linux_testing \
--memory 1024             \
--vcpus=2,maxvcpus=4      \
--cpu host                \
--cdrom $HOME/Downloads/arch-linux_install.iso \
--disk size=2,format=qcow2  \
--network user            \
--virt-type kvm
##### From a previous created volume on a pool
$ sudo virt-install  \
--name fedora-testing \
--memory 2048         \
--vcpus=2             \
--cpu=host            \
--cdrom /tmp/fedora20_x84-64.iso      \
--os-type=linux --os-variant=fedora20 \
--vol poolname/volname \
--network bridge=br0                  \
--graphics=vnc                        \
--noautoconsole
--virt-type kvm

Import an existing volume:

$ sudo virt-install
–name demo
–memory 512
–disk /home/user/VMs/mydisk.img
–import

Manage domains

Start:

$ sudo virsh start domain

Gracefully attempt to shutdown a domain; force off a domain:

$ sudo virsh shutdown domain $ sudo virsh destroy domain

Pause/Resume a domain:

$ sudo virsh suspend domain $ sudo virsh resume domain

Autostart domain on libvirtd start:

$ sudo virsh autostart domain $ sudo virsh autostart domain –disable

Shutdown domain on host shutdown:

Running domains can be automatically suspended/shutdown at host shutdown using the libvirt-guests.service systemd service. This same service will resume/startup the suspended/shutdown domain automatically at host startup. Read /etc/conf.d/libvirt-guests for service options.

Edit a domain’s XML configuration:

$ sudo virsh edit domain

Connect to the QEMU hypervisor over SSH; and the same with logging:

$ sudo virsh –connect qemu+ssh://username@host/system $ sudo LIBVIRT_DEBUG=1 virsh –connect qemu+ssh://username@host/system

Connect a graphic console over SSH:

$ sudo virt-viewer –connect qemu+ssh://username@host/system domain $ sudo virt-manager –connect qemu+ssh://username@host/system domain

Clone a domain

NOTES:

• first, pause or shutdown the domain you wish to clone from.

Clone an identical VM (it only duplicates disks and does host side changes)

virt-clone does not change anything inside the guest OS, it only duplicates disks and does host side changes. So things like changing passwords, changing static IP address, ssh-keys, hostnames etc are outside the scope of this tool. If you need those changes, use virt-sysprep instead. $ sudo virt-clone –original [Domain-Vm-Name-Here] –auto-clone OR $ sudo virt-clone –original [Domain-Vm-Name-Here] –name [New-Domain-Vm-Name-Here] After that, you can resume or start the original domain (the one used to create the clone).

Clone resetting everything inside the guest OS:

We use virt-sysprep instead of virt-clone if we want to clone the VM and make/reset anything inside the guest OS.

Other useful operations:

Get information on a domain:

$ sudo virsh domiflist centos07-test

Get the list of all MAC adresses (and corresponding IPs) from a network:

$ sudo virsh net-dhcp-leases br1  #  - you must find the record containg the MAC adress of the domain you want to chech the IP from

Remove a domain:

$ sudo virsh undefine centos07-test

5. Quickstart

1. Follow steps here of the session “Networking” (3.6)
2. My github repository has a Makefile to ease the daily tasks on KVM. Here is a quick link to it: https://github.com/tiagoprn/devops/tree/master/shellscripts/kvm

6. Storage

Storage pools

List all (active/inactive):

$ sudo virsh pool-list –all

Add a new storage pool:

NOTE: a pool location can be a directory, a network filesystem, or a partition (LVM included).

1) Define the pool:

$ sudo virsh pool-define-as name type [source-host] [source-path] [source-dev] [source-name] [] [–source-format format] e.g. (directory) $ sudo virsh pool-define-as my_first_storage_pool dir - - - - /storage/kvm/storage_pools/001

2) To check it was defined:

$ sudo virsh pool-list --all
Nome                 Estado     Auto-iniciar
-------------------------------------------
my_first_storage_pool inativo    não

3) Build and start the pool:

$ sudo virsh pool-build my_first_storage_pool
$ sudo virsh pool-start my_first_storage_pool
$ sudo virsh pool-autostart my_first_storage_pool

4) Check the pool was built, it started and will auto-start:

$ sudo virsh pool-list --all
Nome                 Estado     Auto-iniciar
-------------------------------------------
my_first_storage_pool ativo      sim

Remove a storage pool:

$ sudo virsh pool-undefine my_first_storage_pool

Storage volumes:

After creating a pool, you can create volumes inside of it.

$ sudo virsh vol-create-as      poolname volumename 10GiB --format aw|bochs|raw|qcow|qcow2|vmdk
$ sudo virsh vol-upload  --pool poolname volumename volumepath
$ sudo virsh vol-list           poolname
$ sudo virsh vol-resize  --pool poolname volumename 12GiB
$ sudo virsh vol-delete  --pool poolname volumename
$ sudo virsh vol-dumpxml --pool poolname volumename  # for details.

Storage volumes howto

• Create a volume to upload the image you wish to use:

$ stat -Lc%s ~/distros/images/CentOS-7-x86_64-GenericCloud.qcow2
854851584

$ sudo virsh vol-create-as my_first_storage_pool centos7 854851584 --format qcow2
O volume centos7 foi criado

• Upload the image to the created volume

$ sudo virsh vol-upload --pool my_first_storage_pool --vol centos7 ~/distros/images/CentOS-7-x86_64-GenericCloud.qcow2
$ sudo virsh vol-list my_first_storage_pool
Nome                 Caminho
------------------------------------------------------------------------------
centos7              /storage/kvm/storage_pools/001/centos7