Skip to main content
Version: Next

Ceph RBD

Since VirtFusion v3.1 Ceph block devices (RADOS) are supported.

Ceph RBD (RADOS Block Device) is a component of the Ceph distributed storage system. Ceph is an open-source software-defined storage platform that provides scalable and reliable storage.

Important

There are a few limitations you should be aware of.

  • Offline migrations are not supported (Live migration is fully supported. Even with a mix of local and remote disks).
  • Backups will be disabled for remote disks.
  • Disaster Recovery will also ignore remote disks. Only the local configurations and assets will be stored.
Very Important

VirtFusion will not set up and manage your storage cluster. It is expected that you already have your cluster in place and managed by experienced administrators.

We will not be able to offer support for any of your attached storage.

Configuration​

Creating a Ceph user​

You may already have your Ceph access setup, however these are simple instructions to create a Ceph user with the correct pool permissions.

On your Ceph cluster this command will create a user called virtfusion with access to the vps_data RBD pool.

ceph auth get-or-create client.virtfusion mon 'profile rbd' osd 'profile rbd pool=vps_data'

Expected output:

[client.virtfusion]
        key = AQD5kFNl2LgUCBAANc2gVXBw4WJDhmIlV8Zrbg==

The key should be saved for future use.

Install required tools​

The following packages will need to be installed on each hypervisor.

qemu-block-extra
ceph-common

Configuring access for hypervisors​

On your first hypervisor, you need to create a secret for the Libvirt tool stack.

cat > secret.xml <<EOF
<secret ephemeral='no' private='no'>
   <usage type='ceph'>
    <name>client.virtfusion secret</name>
   </usage>
</secret>
EOF
virsh secret-define --file secret.xml

Expected output:

Secret 4b5f6d26-81c3-42b1-a196-dd668854ce3b created

The UUID should be saved for future use.

virsh secret-set-value --secret <UUID> <USER_KEY>

UUID should be the UUID generated for the secret and the USER_KEY should be the key that was generated for the user.

Expected output:

Secret value set
Important

This secret must be the same across all of your hypervisors. You don't need to generate a new one for subsequent hypervisors. You can copy the one you just generated.

The secret files are stored in /etc/libvirt/secrets and will be in the form of <UUID>.base64 and <UUID>.xml. These can be directly copied to the rest of your hypervisors.

Edit /etc/ceph/ceph.conf

[global]
mon_host = <MON_HOST_1> <MON_HOST_2> <MON_HOST_3> <MON_HOST_4> ...

[client.virtfusion]
        key = AQD5kFNl2LgUCBAANc2gVXBw4WJDhmIlV8Zrbg==

Define your list of monitor host IPs and the user that was created earlier. Just note this is a bare minimum and you will most likely have your own advanced configuration.

Restart the Ceph monitor.

systemctl restart ceph-mon

If all goes well, you should be able to successfully connect.

ceph version --name client.virtfusion

Expected output:

ceph version 16.2.14 (238ba602515df21ea7ffc75c88db29f9e5ef12c9) pacific (stable)

VirtFusion Configuration​

Storage​

From the Admin menu, select Compute Resources -> Datastores

Click the Add Storage button at the top right of the screen.

ceph

Select the correct Type and give it a Name. Then click Add.

On the next screen you can configure certain cluster settings.

ceph

Fill in the pool Capacity, the Username generated in the previous step, the Pool name and the Libvirt Secret UUID that was also in the previous step.

Click the Update button to save the settings.

Hypervisor​

Edit the hypervisor you would like to add the storage to and click the Add Storage button.

ceph

Give it a Name, select Type Datastore, select the Datastore you created, leave Configuration blank and then click the Add button.

You should now be able to create servers using the new datastore.