The ISE software overview
The ISE software is based on a Linux microkernel, running on each MRC. While the ISE software performs a wide variety of functions within the ISE, as described above, five major services within the stack form the heart of the ISE software services:
- Matrixed RAID Data Allocation
- Continuous Adaptive Data Placement
- In-Situ Remanufacturing and Restoration
- CorteX Web Services
- ISE Mirroring
Matrixed RAID data allocation
Data is allocated, within all ISE, using a patented matrix-based RAID model that provides protection for data, superior to conventional RAID models, while optimizing the performance of each medium within the DataPac, and driving devices to their theoretical maximum performance, generally yielding 2 to 4 times the host IO performance-per-drive of any alternative storage array.
The ISE’s matrixed RAID implementation produces an array that is both high-performance and highly reliable: two features that are generally thought of, in the storage industry, as mutually exclusive. The ISE’s matrixed RAID implementation also produces an array that performs as well when it is full as it does when initially provisioned—a unique feature of the ISE. The matrixed RAID model additionally allows the ISE to reserve media capacity within each DataPac, on each disk and within each HDD platter surface, against the inevitable loss of capacity from failed or marginal operating media. That reserve capacity is provided at no cost by X-IO, above and beyond purchased capacity, and is used, by the in-situ remanufacturing and restoration processes, described below, when failed or suboptimal media require attention.
Continuous adaptive data placement
When an ISE contains hybrid DataPacs, pooling HDD and SSD media, all data is stored, initially, on the enterprise class, mission critical, SAS HDDs in the DataPacs. The ISE CADP software employs a patented cost analysis algorithm to ensure that the hottest data blocks are continuously and dynamically up-tiered to the DataPacs’ SSDs, where those data blocks remain for as long as they are “hot.” Every five seconds, the CADP engine determines which new blocks need to be promoted into SSD, and when the SSD space is at full capacity, which SSD-resident blocks have cooled, and can therefore be returned to HDD storage. CADP not only enables acceleration through data movement to SSD, but also by providing an even access pattern on the HDD which in turn provides faster seek operations as there is less time spent on hotspots.
CADP requires no configuration or human intervention of any kind, and is instantaneously responsive to any change in the server workload it is supporting, making the ISE the only self-tuning hybrid storage array in the market. When data is accessed from HDD rather than SSD, it is delivered at the X-IO enhanced enterprise SAS speed and reliability (rather than low duty cycle consumer-grade SATA or nearline SAS speed, as is the case with many other storage arrays), and its increased heat is noted by CADP for potential promotion.
In-Situ remanufacturing and restoration (ISR)
While the ISE is in operation, in-situ remanufacturing processes observe the detailed telemetry collected by the MRCs, on each HDD in each DataPac, via X-IO’s custom drive firmware. When disk telemetry indicates that it is in danger of becoming compromised, ISR processes immediately migrate the data on that medium, to the spare capacity reserved within the DataPac, and then perform a series of diagnostics to determine whether the drive can be restored to full operation. In the vast majority of cases, a “failed” drive can be restored to reliable operation with simple interventions like power cycling, drive resets, and firmware reinstallations.
But the ISE ISR services are fully capable of employing more sophisticated in-situ remanufacturing operations when necessary, including recalibration of drive internals, taking heads and platters within drives offline, and restoring drives to service with reduced capacity. In this latter case, there is no appreciable performance loss, and the capacity lost in this low-level remanufacturing operation is supplied by the reserve capacity on other media within the DataPac.
ISR services produce no impact, from either a performance or capacity perspective, on ISE operations. Several best-in-world published benchmarks conducted with ISEs were produced, in controlled environments, in situations that necessitated ISR. While the benchmark was in process, the ISR impact was undetectable by the benchmark team, and no evidence of the remanufacturing was visible in the benchmark results.
Cortex web services
Scale-out frame-free storage architectures require that every unit of storage—every “brick” in the storage architecture—be fully manageable, and able to be fully provisioned, remotely, by an arbitrary application, including the application using the storage.
To enable remote inspection, interrogation, provisioning, re-provisioning, and ongoing management of small and large pools of ISEs from arbitrary software environments, the ISE is wrapped in X-IO’s CorteX RESTful APIs. Every atomic operation that can be performed on an ISE is available, via the CorteX APIs, to an external program written in an arbitrary language. All ISE management software, including the ISE Manager product, made available by X-IO use the CorteX APIs.
The CorteX Orchestration Services include in-built discovery and enumeration, so that a RESTful inquiry made to any single ISE within a pool yields the information on all ISEs within that pool, making it simple for an arbitrary application to discover and manage any ISE or a combination of ISEs, within the ISE pool.
X-IO customers with large ISE fields report that they can provision, re-provision, and manage their ISEs with a few dozen lines of high-level code that is written in languages as widely variant as Visual Basic and Ruby.
ISE mirroring and active-active mirrors
ISE Mirroring provides enhanced availability and portability through synchronous mirroring of pairs of ISEs.
ISE Mirroring includes options for:
- One of the industry’s only active-active replication solutions when using Active-Active Mirroring
- Traditional disaster recovery measures, using active-passive replication with Basic Mirroring
- Volume Migration, enabling movement between different ISE performance tiers and RAID transformation
- Volume Copy, creating clones of production data in order to make test and development easier
Active-Active Mirroring enables ISEs to be paired in active-to-active, geographically separate couplings (up to 40 kilometers apart) that appear, for all practical purposes, to server workloads as a single ISE. All write operations, committed against either half of the active pairing, are guaranteed to be committed to both halves of the pairing before the operations are acknowledged. The loss of one half of the pairing—due, for example, to a power loss or a severed fabric connection—is invisible to the servers using the active-active pair of ISE.