Remote support

Libvirt allows you to access hypervisors running on remote machines through authenticated and encrypted connections.

• Basic usage
• Transports
• Remote URIs
  • Extra parameters
• Generating TLS certificates
  • Public Key Infrastructure set up
  • Background to TLS certificates
  • Setting up a Certificate Authority (CA)
  • Issuing server certificates
  • Issuing client certificates
  • Troubleshooting TLS certificate problems
• libvirtd configuration file
• IPv6 support
• Limitations

Basic usage ¶

On the remote machine, libvirtd should be running in general. See the section on configuring libvirtd for more information.

Not all hypervisors supported by libvirt require a running libvirtd. If you want to connect to a VMware ESX/ESXi or GSX server then libvirtd is not necessary. See the VMware ESX page for details.

To tell libvirt that you want to access a remote resource, you should supply a hostname in the normal URI that is passed to virConnectOpen (or virsh -c ...). For example, if you normally use qemu:///system to access the system-wide QEMU daemon, then to access the system-wide QEMU daemon on a remote machine called compute1.libvirt.org you would use qemu://compute1.libvirt.org/system.

The section on remote URIs describes in more detail these remote URIs.

From an API point of view, apart from the change in URI, the API should behave the same. For example, ordinary calls are routed over the remote connection transparently, and values or errors from the remote side are returned to you as if they happened locally. Some differences you may notice:

• Additional errors can be generated, specifically ones relating to failures in the remote transport itself.
• Remote calls are handled synchronously, so they will be much slower than, say, direct hypervisor calls.

Transports ¶

Remote libvirt supports a range of transports:

tls

TLS 1.0 (SSL 3.1) authenticated and encrypted TCP/IP socket, usually listening on a public port number. To use this you will need to generate client and server certificates. The standard port is 16514.

unix

Unix domain socket. Since this is only accessible on the local machine, it is not encrypted, and uses Unix permissions or SELinux for authentication. The standard socket names are /var/run/libvirt/libvirt-sock and /var/run/libvirt/libvirt-sock-ro (the latter for read-only connections).

ssh

Transported over an ordinary ssh (secure shell) connection. Requires Netcat (nc) installed and libvirtd should be running on the remote machine. You should use some sort of ssh key management (eg. ssh-agent) otherwise programs which use this transport will stop to ask for a password.

ext

Any external program which can make a connection to the remote machine by means outside the scope of libvirt.

tcp

Unencrypted TCP/IP socket. Not recommended for production use, this is normally disabled, but an administrator can enable it for testing or use over a trusted network. The standard port is 16509.

libssh2

Transport over the SSH protocol using libssh2 instead of the OpenSSH binary. This transport uses the libvirt authentication callback for all ssh authentication calls and therefore supports keyboard-interactive authentication even with graphical management applications. As with the classic ssh transport netcat is required on the remote side.

libssh

Transport over the SSH protocol using libssh instead of the OpenSSH binary. This transport uses the libvirt authentication callback for all ssh authentication calls and therefore supports keyboard-interactive authentication even with graphical management applications. As with the classic ssh transport netcat is required on the remote side.

The default transport, if no other is specified, is tls.

Remote URIs ¶

See also: documentation on ordinary ("local") URIs.

Remote URIs have the general form ("[...]" meaning an optional part):

driver[+transport]://[username@][hostname][:port]/[path][?extraparameters]

Either the transport or the hostname must be given in order to distinguish this from a local URI.

Some examples:

• xen+ssh://rjones@towada/system
  — Connect to a remote Xen hypervisor on host towada using ssh transport and ssh username rjones.
• xen://towada/system
  — Connect to a remote Xen hypervisor on host towada using TLS.
• xen://towada/system?no_verify=1
  — Connect to a remote Xen hypervisor on host towada using TLS. Do not verify the server's certificate.
• qemu+unix:///system?socket=/opt/libvirt/run/libvirt/libvirt-sock
  — Connect to the local qemu instances over a non-standard Unix socket (the full path to the Unix socket is supplied explicitly in this case).
• test+tcp://localhost:5000/default
  — Connect to a libvirtd daemon offering unencrypted TCP/IP connections on localhost port 5000 and use the test driver with default settings.
• qemu+libssh2://user@host/system?known_hosts=/home/user/.ssh/known_hosts
  — Connect to a remote host using a ssh connection with the libssh2 driver and use a different known_hosts file.
• qemu+libssh://user@host/system?known_hosts=/home/user/.ssh/known_hosts
  — Connect to a remote host using a ssh connection with the libssh driver and use a different known_hosts file.

Extra parameters ¶

Extra parameters can be added to remote URIs as part of the query string (the part following ?). Remote URIs understand the extra parameters shown below. Any others are passed unmodified through to the back end. Note that parameter values must be URI-escaped.

command -p port [-l username] hostname netcat -U socket

Generating TLS certificates ¶

Public Key Infrastructure set up ¶

If you are unsure how to create TLS certificates, skip to the next section.

If 'pkipath' is specified in URI, then all the client certificates must be found in the path specified, otherwise the connection will fail with a fatal error. If 'pkipath' is not specified:

• For a non-root user, libvirt tries to find the certificates in $HOME/.pki/libvirt first. If the required CA certificate cannot be found, then the global default location (/etc/pki/CA/cacert.pem) will be used. Likewise, if either the client certificate or the client key cannot be found, then the global default locations (/etc/pki/libvirt/clientcert.pem, /etc/pki/libvirt/private/clientkey.pem) will be used.
• For the root user, the global default locations will always be used.

Background to TLS certificates ¶

Libvirt supports TLS certificates for verifying the identity of the server and clients. There are two distinct checks involved:

• The client should know that it is connecting to the right server. Checking done by client by matching the certificate that the server sends to the server's hostname. May be disabled by adding ?no_verify=1 to the remote URI.
• The server should know that only permitted clients are connecting. This can be done based on client's IP address, or on client's IP address and client's certificate. Checking done by the server. May be enabled and disabled in the libvirtd.conf file.

For full certificate checking you will need to have certificates issued by a recognised Certificate Authority (CA) for your server(s) and all clients. To avoid the expense of getting certificates from a commercial CA, you can set up your own CA and tell your server(s) and clients to trust certificates issues by your own CA. Follow the instructions in the next section.

Be aware that the default configuration for libvirtd allows any client to connect provided they have a valid certificate issued by the CA for their own IP address. You may want to change this to make it less (or more) permissive, depending on your needs.

Setting up a Certificate Authority (CA) ¶

You will need the GnuTLS certtool program documented here. In Fedora, it is in the gnutls-utils package.

Create a private key for your CA:

certtool --generate-privkey > cakey.pem

and self-sign it by creating a file with the signature details called ca.info containing:

cn = Name of your organization
ca
cert_signing_key

certtool --generate-self-signed --load-privkey cakey.pem \
  --template ca.info --outfile cacert.pem

(You can delete ca.info file now if you want).

Now you have two files which matter:

• cakey.pem - Your CA's private key (keep this very secret!)
• cacert.pem - Your CA's certificate (this is public).

cacert.pem has to be installed on clients and server(s) to let them know that they can trust certificates issued by your CA.

The normal installation directory for cacert.pem is /etc/pki/CA/cacert.pem on all clients and servers.

To see the contents of this file, do:

certtool -i --infile cacert.pem

X.509 certificate info:

Version: 3
Serial Number (hex): 00
Subject: CN=Libvirt Project
Issuer: CN=Libvirt Project
Signature Algorithm: RSA-SHA
Validity:
        Not Before: Mon Jun 18 16:22:18 2007
        Not After: Tue Jun 17 16:22:18 2008
[etc]

This is all that is required to set up your CA. Keep the CA's private key carefully as you will need it when you come to issue certificates for your clients and servers.

Issuing server certificates ¶

For each server (libvirtd) you need to issue a certificate containing one or more hostnames and/or IP addresses. Historically the CommonName (CN) field would contain the hostname of the server and would match the hostname used in the URI that clients pass to libvirt. In most TLS implementations the CN field is considered legacy data. The preferential mechanism is to use Subject Alt Name (SAN) extension fields to validate against. In the future use of the CN field for validation may be discontinued entirely, so it is strongly recommended to include the SAN fields.

In the example below, clients will be connecting to the server using a URI of qemu://compute1.libvirt.org/system, so the CN must be "compute1.libvirt.org".

Make a private key for the server:

certtool --generate-privkey > serverkey.pem

and sign that key with the CA's private key by first creating a template file called server.info. The template file will contain a number of fields to define the server as follows:

organization = Name of your organization
cn = compute1.libvirt.org
dns_name = compute1
dns_name = compute1.libvirt.org
ip_address = 10.0.0.74
ip_address = 192.168.1.24
ip_address = 2001:cafe::74
ip_address = fe20::24
tls_www_server
encryption_key
signing_key

The 'cn' field should refer to the fully qualified public hostname of the server. For the SAN extension data, there must also be one or more 'dns_name' fields that contain all possible hostnames that can be reasonably used by clients to reach the server, both with and without domain name qualifiers. If clients are likely to connect to the server by IP address, then one or more 'ip_address' fields should also be added.

Use the template file as input to a certtool command to sign the server certificate:

certtool --generate-certificate --load-privkey serverkey.pem \
  --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \
  --template server.info --outfile servercert.pem

This gives two files:

• serverkey.pem - The server's private key.
• servercert.pem - The server's public key.

We can examine this certificate and its signature:

certtool -i --infile servercert.pem
X.509 certificate info:

Version: 3
Serial Number (hex): 00
Subject: O=Libvirt Project,CN=compute1.libvirt.org
Issuer: CN=Libvirt Project
Signature Algorithm: RSA-SHA
Validity:
        Not Before: Wed Oct 04 09:09:44 UTC 2017
        Not After: Thu Oct 04 09:09:44 UTC 2018
Extensions:
        Basic Constraints (critical):
                Certificate Authority (CA): FALSE
        Subject Alternative Name (not critical):
                DNSname: compute1
                DNSname: compute1.libvirt.org
                IPAddress: 10.0.0.74
                IPAddress: 192.168.1.24
                IPAddress: 2001:cafe::74
                IPAddress: fe20::24

Note the "Issuer" CN is "Libvirt Project" (the CA) and the "Subject" CN is "compute1.libvirt.org" (the server). Notice that the hostname listed in the CN must also be duplicated as a DNSname entry

Finally we have two files to install:

• serverkey.pem is the server's private key which should be copied to the server only as /etc/pki/libvirt/private/serverkey.pem.
• servercert.pem is the server's certificate which can be installed on the server as /etc/pki/libvirt/servercert.pem.

Issuing client certificates ¶

For each client (ie. any program linked with libvirt, such as virt-manager) you need to issue a certificate with the X.509 Distinguished Name (DN) set to a suitable name. You can decide this on a company / organisation policy. For example:

C=GB,ST=London,L=London,O=Libvirt Project,CN=name_of_client

The process is the same as for setting up the server certificate so here we just briefly cover the steps.

1. Make a private key:

   certtool --generate-privkey > clientkey.pem
   

2. Act as CA and sign the certificate. Create client.info containing:

   country = GB
   state = London
   locality = London
   organization = Libvirt Project
   cn = client1
   tls_www_client
   encryption_key
   signing_key
   

   and sign by doing:

   certtool --generate-certificate --load-privkey clientkey.pem \
     --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \
     --template client.info --outfile clientcert.pem
   

3. Install the certificates on the client machine:

   cp clientkey.pem /etc/pki/libvirt/private/clientkey.pem
   cp clientcert.pem /etc/pki/libvirt/clientcert.pem
   

Troubleshooting TLS certificate problems ¶

failed to verify client's certificate

On the server side, run the libvirtd server with the '--listen' and '--verbose' options while the client is connecting. The verbose log messages should tell you enough to diagnose the problem.

You can use the virt-pki-validate shell script to analyze the setup on the client or server machines, preferably as root. It will try to point out the possible problems and provide solutions to fix the set up up to a point where you have secure remote access.

libvirtd configuration file ¶

Libvirtd (the remote daemon) is configured from a file called /etc/libvirt/libvirtd.conf, or specified on the command line using -f filename or --config filename.

This file should contain lines of the form below. Blank lines and comments beginning with # are ignored.

setting = value

The following settings, values and default are:

IPv6 support ¶

The libvirtd service and libvirt remote client driver both use the getaddrinfo() functions for name resolution and are thus fully IPv6 enabled. ie, if a server has IPv6 address configured the daemon will listen for incoming connections on both IPv4 and IPv6 protocols. If a client has an IPv6 address configured and the DNS address resolved for a service is reachable over IPv6, then an IPv6 connection will be made, otherwise IPv4 will be used. In summary it should just 'do the right thing(tm)'.

Limitations ¶

• Fine-grained authentication: libvirt in general, but in particular the remote case should support more fine-grained authentication for operations, rather than just read-write/read-only as at present.

Please come and discuss these issues and more on the mailing list.

• Download
• Contribute
• Docs

Contact

• email
• irc

Community

• twitter
• stackoverflow
• serverfault