Storage Concepts: Software Defined Storage (SDS)

What is SDS?

Software Defined Storage, or SDS, is now integral to most virtualisation technologies. Indeed, the term is often used interchangeably with “Storage Virtualisation” – rightly or wrongly is material for another article. SDS could be said to define the suite of tools available to virtualise storage than the virtualisation itself, but I will confine this article to speaking interchangeably about SDS and storage virtualisation. As with all “Software Defined…” concepts, the key premise is abstraction, in this case of physical storage, and its presentation as a pool of usable data storage that can be allocated to devices, often through automation at the virtualisation layer. A much over-used expression is the Single Pane of Glass (SPOG) but it is applicable here in that SDS software provides an overall view of the storage without the need to monitor and manage individual storage devices. The software, quite literally, defines how the storage is allocated and used.

How is SDS used?

Imagine an environment with a mature storage estate that comprises a number of disparate storagedevices, let’s say EMC VNX, IBM V7000 and HDS HUS300, managed across several management platforms (EMC Ionix, IBM Flex System Management, Hitachi Device Manager, etc.). There is a big push for virtualisation through VMware but the storage side needs to remain in place until the virtualisation project is completed. Freeing up space from the virtualisation project, through consolidation, archive etc; means that these existing storage systems are more than adequate to service requirements of the virtualised platform. SDS software can aggregate the storage from all three storage devices and allow control of provisioning through the virtualisation software. The storage is aggregated into pools and provisioning can be manually or automatically done, depending on the organisation’s needs, entirely through the virtualisation software which in this case would be VMware vSphere. I guess I should say that other platform virtualisation technologies are available. Whether I believe it or not.

Automation

There is a high level of automation available to those who wish to virtualise their storage. It is possible, though maybe not always advisable, to allocate multiple tiers of storage into a large pool and let the underlying storage take care of the tiering. Consider a datastore has been created from a virtualised storage pool containing tier 1, 2 and 3 storage – let’s say a couple of EMC VNXs and an HP P4335 (odd mix, but you know how these things often happen). The storage can be pooled and allocated to a subset of virtual machines which all have differing storage performance requirements (IO profiles if you will) and the underlying storage will take care of the location of the storage through automated tiering of areas of high activity and performance requirement (hot-spots). Those virtual machines, after a period of learning, will have their performance requirements automatically met, assuming there is enough higher tier storage available.

Virtual Storage Appliances

Virtual Storage Appliances (VSA) are used to perform the storage virtualisation function. These will take an underlying storage system and allow it to be used with the abstraction layer’s software capabilities. One example of which would be where cross-site clustering is required but existing storage is unable to perform the synchronicity required to keep everything up to date. Implementing a VSA to sit above the storage and manage the replication and synchronisation allows a lower cost, cross-site storage solution to be implemented. Whether a low cost implementation of business critical clustered applications is appropriate is a discussion for those who will be picking up the bill.

Is SDS New?

As with many technical innovations, they are variations on an old theme. Mainframe computing used the concept of storage virtualisation way back in the ‘70s, believe it or not. IBM’s MVS (which stands for Multiple Virtual Storage) made use of shared memory across many virtual systems and workloads. Storage meant “memory storage” in this case but the principle was much the same. Storage systems as we would recognise them were still some way off.

What SDS Solutions Are Out There?

Here are a few of the better known SDS vendors:

• Nexenta
• Symantec (Storage Foundation/InfoScale)
• DataCore
• VMware
• EMC (ViPR, vVNX)
• HP (StoreVirtual VSA)

All provide the ability to centralise storage management and provide overall control of the storage landscape to virtualisation platforms. Some of these may be contentious and open to argument around whether they truly offer software defined storage, but they all put in a pretty good claim.

This article details these vendor’s offerings, and more, rather well.

Who Uses SDS?

Pretty simple to answer – anyone who requires a high level of automation, simplification and abstraction from their storage environment. That would include:

• Cloud service providers
• Anyone wishing to virtualise their estate
• Anyone with heterogeneous, legacy storage environments
• Anyone with a wide range of performance requirements

Anyone who wants to simplify storage management, in a nutshell.

Storage Concepts: Erasure Coding (EC)

What is Erasure Coding?

Wikipedia: “In information theory, an erasure code is a forward error correction code for the binary erasure channel, which transforms a message of k symbols into a longer message with n symbols such that the ...”

The Wikipedia entry is probably not the best place to start, let’s look at it from above the algorithms and formulae. What does Erasure Coding mean broadly, in terms of data storage? Well, erasure coding is a method of securing data on disk and can be thought of as an evolution of RAID in some respects, so to touch on RAID briefly before we look at Erasure Coding, traditional RAID will secure data by either mirroring it across one or more disks (RAID 1) or will write parity to one or more disks in a RAID set (RAID 5, RAID 6 aka RAID DP etc.) in order to ensure data is not lost in the event of a disk failure. Hot spares are kept in order to write the data from the failed disk onto, until the original disk is replaced. There are obvious benefits to this way of protecting data but the downside is becoming more evident as data volumes, and perhaps more pertinently, disk sizes increase. Mirroring and parity will require more and more disk as data volumes grow and replacing failed disks means  times on these vast disks is greatly increased, exposing the data to further risk.

Example: File F is split into 9(k) parts, 

which are themselves encoded into

 12(n) further parts, each containing 

effectively 1/9th of the data (it’s not quite

 as simple as that, but helps

 illustrate the principle). As long

 as we still have 9 of the 12 x 1/9th parts 

the original data can be reconstituted.

On to Erasure Coding. Data is split up, expanded with redundant data and sent to a target device in another location, which can be on the same array, local arrays in the same DC and/or remote arrays nationally or geographically. The data at the other ends can be decoded from this source or any other source of the data from any other location meaning that if this particular packet of data is lost, it can be retrieved from a subset of the same data from the other locations. To use a rather tenuous analogy, if you pack elements of your holiday wardrobe across three suitcases, you will arrive in Barbados safe in the knowledge that you will have something to wear even if one of your suitcases ends up in Luton. If two cases are lost, you may end up wearing just speedos and a smile all week. In other words, your suitcase fault tolerance is 2 out of 3.


In a little more detail, Erasure Coding splits data and shares it across locations to be reconstructed in the event of a corruption or loss of data in one or a number of the others. The concept is actually relatively simple to get a handle on. A file or object, F, is split up into k parts, which are then themselves encoded into n parts. To recreate F we would need k out of n parts to recreate the data. An EC configuration might require 9(k) from 12(n)parts, meaning that any 9 subsets of the data out of 12 can be used to reconstruct your original data (see example, right). Any less than that and you can’t reconstruct.

This number can be lower, for example 6 from 12, but then you would need a lot more space and bandwidth to accommodate the fact that all of the data needs to be retrievable from less locations, so therefore contain more redundant* data for the other locations – you might as well mirror or replicate.

* Redundant in this case taking the high availability meaning, as opposed to 'unused'

Is Erasure Coding new?

The theory isn’t. As is often the case with new technology, the principle has been around for some time in the form of Forward Error Correction, first proposed by Reed and Solomon in 1960. In order to ensure that transmissions were received, they were split and encoded, along with redundant data, with an algorithm. As long as the algorithm is known at the other end, any errors can be corrected accordingly. Put horribly simply, if I sent a transmission containing C1H2R3I4S5, and my fiendishly complex algorithm of adding a number to each letter was known at the other end, the receiver would know there was a problem if they received D4A3V2E1 along with the C1H2R3I4S5 in the same transmission. Erasure coding is Forward Error Correction for data replication, storage and security, but instead of correcting errors we are protecting against data loss.

Security

There is an inherent security in Erasure Coding. The fact that the data is broken up and recreated using an algorithm means that, unless that algorithm is known, the data is meaningless. This lends the data transmission an in-flight security of its’ own.

Who has implemented it?

There are a number of storage that have taken up Erasure Coding, often in conjunction with Object Based Storage:

•         SwiftStack
•         EMC Atmos
•         DDN ObjectAssure
•         Scality
•         Hadoop
•         NetApp
•         Cleversafe

Is DR still the IT poor relation?

DR – Disaster Recovery, not Diminishing Returns

Disaster Recovery solutions are still all too often seen as an afterthought, a necessity that drains money and resource, that will not provide an ROI and will most likely never be used in a real-life situation. The 2001 attack on the World Trade Center could be said to have been the trigger for many organisations taking disaster recovery far more seriously, resulting in a greatly increased focus on DR capability which led to the rapid deployment of additional hardware and resource to provide cover against similar disasters, whether man-made or natural.

Time and distance have eroded those early concerns over availability, which means that there are disaster recovery solutions that were put in place shortly after the events of 2001 that are now many years old, sparsely reviewed and rarely, if ever, tested.

Making appropriate plans for Disaster Recovery is essential.  There are some key questions that will factor in every DR decision making process, and they have a common theme: what are the consequencesof the organisation being out of action?

The answers will shape your Disaster Recovery solution.

Disaster Recovery doesn’t just refer to the infrastructure in place to support a recovery from a catastrophic failure. The right technology is essential of course, but just as important are the processes and the people who make up the day to day operation of the organisation. Effective DR planning needs to incorporate an equally effective Business Continuity strategy. Once an effective strategy has been decided upon, it needs to be documented and accessible to everyone who needs to know what to do in the event of DR invocation.

Having implemented a sound Disaster recovery and Business Continuity strategy it is vital that it be tested regularly. Full testing is imperative: the systems and services that are used in daily operation need to be offline and operation in in the DR location needs to be able to continue for as long as the recovery will take. Adequate provision for DR resources is an essential part of the DR solution, especially where there might be a significant delay factored into the return to business as usual.

One more point that will always stand repetition is that cost should not be a determining factor when planning a DR solution. There is a very simple equation which is ignored surprisingly often.

To put it simply, the cost of the DR solution should be relative to the cost of an outage. If your organisation would lose $10m an hour, your disaster recovery solution should be capable of recovering in a lot less than an hour and if it costs $10m to have the system in place to do it, the organisation wins in the event of DR invocation. If you spend $10k but your operation is down for 3 days… well the point is clear.

Now, some of this may not come as much of a surprise and it could be said that much of it is obvious. Maybe so, but it might just be worth taking a look at those DR and BCP plans to see if these obvious requirements are actually met. It might not be as simple to accomplish as it first seems so it is always worth seeking help from experienced professionals who make it their business to ensure your business continues.

Storage Concepts: Object Based Storage (OBS)

What is object-based storage?

Essentially, object-based storage, or OBS, a different way of storing and accessing data. Traditionally storage has been stored using block and file structures. Just to summarise all of these:

Representation of traditional block 

based storage, with data stored 

in equal sized block on disk

Block storage can be thought of in terms of LUNs (Logical Unit Numbers) that are presented to servers and applications as discrete disks that act as local devices on the server - essentially buckets containing blocks of data. SAN infrastructure is the mechanism by which the data is presented to servers and applications using the standard scsi protocols over fibre channel, iscsi and FCoE etc. Access to block data is controlled through the storage array through host mapping and further through fibre channel switches through zoning.

File based storage, using heirarchical files and folders 

common on NAS devices.

File storage is presented as folders and/or directories over the network using the NFS and CIFS protocols from NAS devices (“filers”). The structure can be thought of as nested drawers of files that are shared out to devices on a network for access by anyone with the relevant credentials for authentication. NFS has been experiencing something of a revival in the virtualisation space, being highly proficient in presentation of datastores to virtual environments.

Object based storage can be represented 

as a lookup table containing information 

needed to retrieve the object from storage.

Object Based Storage introduces the concept of a flat structure with unique identities for data objects. Each object has its’ own hash signature which is created according to the contents of the object.  Additionally, the metadata (or descriptive header) of an object can contain a great deal more than a traditional file. The data is accessed through the http protocol using the established authentication methods associated with it (such as X-Auth). It is worth noting that block and object storage are not mutually exclusive. The two complement each other with block storage providing the back-end and the object storage software providing the features and functionality off-array.  

Is object-based storage new technology?

No. It existed for some time in some form as CAS (variably known as Content Addressed, Addressable or Aware Storage).  In fact, the concept has been around for nearly 20 years but, necessity driving innovation, companies have developed their object storage strategies at an accelerated pace over the last 6 years or so in response to the rising need to tame their vast data estates and make them as accessible, responsive, contained and cost-effective as possible. 

Why is object storage better than traditional storage?

It is not better in all cases, such as pure file-sharing environments and the frequent access and update requirements of a relational database, but there are some definite advantages in a number of areas. Object-based storage can certainly be a cheaper alternative to the expensive, high end storage products that use proprietary management and administration tools. Cheaper back-end block storage can be used with an object storage software system to provide all of the additional features and functionality. Another benefit comes with the object metadata. With traditional block and file storage, a content management system, often with a relational database, would have to be used to store additional information on a file that cannot be stored with it. The objects in object storage have this information contained within them and it is highly customisable.

How and where is object storage used?

Any organisation that has a requirement to create and/or store large amounts of relatively static data would find object-based storage extremely effective. Examples include

• Cloud providers make extensive use of object storage due to its’ scalability, cost efficiency and ease of interface with cloud portal development.
• Social networking sites with thousands and perhaps many millions of pictures, videos and texts (Facebook, Twitter, Linkedin).
• Publishing companies that produce and are required to keep periodical publications for use in stats gathering and Big Data analysis.
• Healthcare organisations that are required to maintain a repository of scan and x-ray images, which are often used once and then kept to be referred to infrequently, if at all.

The common factor is evident from this list. Vast quantities of “unstructured” data that needs to be stored at as low a cost and footprint as possible

Is object storage really “infinitely scalable”?

Well, “infinite” is an over-used expression in the IT world but one could, in theory, continue to add storage nodes until you run out of money, space or time to an object storage environment. The object storage management system can accommodate continued additions to the back-end storage as long as the metadata and ID references can be maintained on the object storage nodes.

How is availability and redundancy maintained in object storage?

A key attribute of object storage is that copies of the objects can be stored, accordingly to definable policies, across multiple locations locally, nationally and internationally making it truly resilient to local DR scenarios.

Which vendors have implemented it?

Several players have emerged with a head above the clouds (pun intended) in the object storage arena, notably:-

• EMC – ISOLON, ATMOS, Centera
• DirectData Networks – Web Object Scalar (WOS)
• NetApp - StorageGRID
• Scality – Software based “RING” scale-out storage solutions
• HDS – HCP (Hitachi Content Platform)

Others are coming along behind, and undoubtedly every storage vendor has their own agenda in this area.

Where Is My Data?

Availability and Control

Everyone with an interest in data and data management will know that there are scaling levels of availability and assurance. With increasing availability levels comes a decreasing control over exactly where your data resides and, to some, a security risk. Let’s consider a humble word document (other document types are available) and your control over it as it progresses up the availability tree.

Location	Availability	Control
Hard drive (Laptop or PC internal/external USB etc.)	Back it up. Lose it and it’s a restore from last known good backup, sometimes with less than optimal results.	It’s there, in front of you and you can literally see the external drive itself and know that your doc is in there, but depending on where you backed it up to you could potentially lose it in a disaster situation[1].
Simple RAID	RAID two drives together (1 or 10), lose one and you still have another one. Also back it up.	You’ll still be confident that your doc is on both those drives and under your control but, again depending on where you back it up, you could still lose it in a disaster situation.
Storage Array	More complex raid levels (5, 6, DP etc.) so you can potentially lose more than one drive and still have data availability confidence.	Starting to get a little hazy on exactly where your data is. It’s on those hard drives but accessing those drives individually you’d really struggle to get your data back.
Multiple site storage arrays	RAID levels locally which will keep your doc safe and will replicate it across to another site some distance away. Alternatively, the whole array is clustered so it is “highly” available on both sides.	Seems a lot of expense and effort for a little word document … So exactly where is that doc? Does it exist on both sides? How different are they, in terms of a point in time? Can you put your finger on exactly where it is at any given time? Doubtful.
Private Cloud	Multiple arrays across multiple locations that could be being served, storage wise, from one of any number of platforms. Loss of an entire site can be tolerated.	You can still cling to the fact that your doc is still in one of your datacentres somewhere but it is a very indistinct. The precise location is all but impossible to discern from the end user point of view
Public Cloud/Distributed	AWS, vCloud, Azure etc. potentially utilising object oriented storage[2] and erasure coding to keep data as available as it can possibly be, recoverable even if you lose multiple sites	Your doc now exists as a somewhat ethereal entity. If erasure coding is in play to ensure data is available in the event of multiple site failures, your word doc is pulled apart, combined with other hashing data and spread out over multiple locations. Any single piece of this will be unrecognisable. Try to reassemble it in flight and it will be unrecognisable.

Or …

With increasing availability your word doc moves further away from being under your direct control. Data sovereignty considerations aside, if you want to keep your data in a known state and location, you’ll need to compromise on availability. There’s an intersect somewhere between the two storage options that many organisations occupy, somewhere around the dual site, replication, clustering sort of area. These organisations also increase control with security measures like in-flight and at-rest encryption technologies that work happily alongside distributed data environments, but nonetheless you can’t honestly say “I know exactly where my data is”.

Clearly, knowing the exact location of your data may not be such a good thing in some situations. You can be compromised into divulging the details of your data if you know its’ exact whereabouts but the idea of giving up control of your data to a third party or to technology is still anathema to some.

Control is an illusion

Control is an illusion. No matter how much you think you can control your data there will always be scope for something to take a situation out of your control, so how would you lessen the impact of the relinquishing control? Let’s add another line to the little graph – responsibility.

It follows the availability line closely because, as you give over control, responsibility for loss of the data goes with it. Although you won’t ever lose all responsibility even when you’ve given over complete control, you no longer have the additional overhead of managing the data yourself and being responsible, financially or technologically, for its well-being. Perhaps more importantly, along with a diminished likelihood of data loss, there is greater scope for recourse if it does happen with contracts in place such as service level agreements, uptime guarantees and the like.

Availability is real

At the risk of everyone now having a certain song in their head for the rest of the day, I’ll say “let it go”. Not knowing the precise location of your data is a positive, both from an availability and security perspective. You may not know precisely where your word doc is, but neither does anyone else and, more to the point, in the age of error coded data and 2048bit encryption, it doesn’t actually matter. Being able to access that word document anywhere and in the face of some fairly high end disaster situations is more than a fair compromise for knowing exactly where it is at any given moment.

---

[1] A disaster can take many forms and mean many different things to the individual and the business but in general a flood fire or earthquake would have a similar impact on both in terms of data loss.

[2] See this related object oriented storage blog