OpenStack block storage integration for X-IO’s ISE G3 products was added as part of the Kilo release in April of 2015. The X-IO Cinder volume driver allows the administrator to take advantage of ISE features like Media Affinity placement, Thin Provisioning and Quality of Service.
These features can be applied to individual volumes using Cinder volume types. In this post, I will go over the different QoS settings that the ISE supports and show how to apply them using volume types in Cinder. I am using the video
“ISE Quality of Service” by Hollis Beall (X-IO’s Director of Technical Marketing) as a reference. It does a great job of showcasing the QoS functionality at work in an ISE G3 780.
The ISE supports three QoS settings on a per volume basis, Min Goal IOPS, Max Limit IOPS, and Burst Limit IOPS. Min Goal IOPS is the IOPS the volume is expected to achieve.
If Min Goal is not reached then another volume may be throttled. Max Limit IOPS is a soft ceiling that a volume can’t exceed if another volume is not utilizing its Min Goal. Finally, the Burst Limit is a hard ceiling and the volume will be hard throttled if the Burst Limit is reached.
Now let’s look at how to set those attributes on a Cinder volume. The initial design of Cinder had the QoS settings as regular extra specs and the administrator would add them to a volume type. This has since been redesigned and now QoS settings are entered in its own configuration set, called QoS Specs. QoS Specs can then be associated with one or more volume types.
In Hollis’ video, he starts out with 16 volumes with five of them being designated as “noisy neighbors”. All 16 volumes are configured without any QoS attributes set. In the video, we can see that once the five noisy neighbors are let loose, they start impacting the performance of all other volumes. To address this, all volumes are assigned the following QoS settings: Min Goal IOPS of 4000, Max Goal IOPS of 6000 and a Burst Limit of 7000 IOPS. I’ll use a set of five volumes in my example to show how we do this in OpenStack using the Horizon Dashboard UI.
First, let’s create five volumes without a specified volume type:
Next, we’ll create a set of QoS attributes in a QoS spec. The QoS Specs configuration can be found under the volume type tab under Systems>Volumes.
In the screen shot below, I’ve created a volume type called xiotype and a QoS spec called xio-regular.
The second column under QoS specs specifies the consumer as “back-end”. That indicates that the QoS spec is going to be applied on the back end storage as opposed to the hypervisor level. The next column shows the individual attributes that have been assigned to this QoS spec. The X-IO Cinder volume driver recognizes three keywords for QoS specs. Min Goal IOPS is specified using the keyword minIOPS, Max Limit IOPS is specified using the keyword maxIOPS, and finally the Burst Limit IOPS is specified using the keyword burstIOPS. We set minIOPS to 4000, maxIOPS to 6000, and burstIOPS to 7000.
Once the QoS spec is set up the way we want it, we go ahead and associate it with the xiotype volume type. That is done using the “Manage QoS spec association” from the volume type action drop down menu in the far right column.
The screen shot below shows the QoS Spec assigned to the volume type xiotype.
Now we have a QoS spec with QoS attributes and it has been assigned to a volume type. Next, we assign this volume type to all volumes. To do this, we switch over to the project volume list and choose Change Volume Type from the Actions menu for the volume of choice.
In the Change Volume Type pop-up window, we select xiotype in the Type drop down menu and press Change Volume Type. Behind the scenes a so-called retype operation is sent to the volume driver where the attributes on the existing volume are updated.
The X-IO Cinder volume driver supports updating the QoS specs on the fly and the ISE back end will apply the updated values immediately. We go ahead and change the type on all volumes to xiotype.
All five volumes are now guaranteed to get 4000 IOPS, but also bounded by a hard ceiling of 7000 IOPS. This way the performance will be predictable and any noisy neighbors wont’ affect the rest of the volumes.
Now, let’s say we get a request to allow a 15,000 IOPS burst limit on Volume1.
To do this we go back to the volume type tab, create another QoS Spec defining the same minIOPS, maxIOPS and the higher burstIOPS. We also create a new volume type and assign the new QoS Spec to the new volume type as shown in the screen shot below.
Once this is done, we go back to the new volume type can be assigned to Volume1 using the Change Volume Type action. The Cinder volume driver will go through the retype operation and tell the ISE back end to assign the updated QoS attributes to Volume1.
That’s it. A simple, but powerful way to get predictable performance at individual volume level.