Storage Concepts: Object Based Storage (OBS)

What is object-based storage?

Essentially, object-based storage, or OBS, a different way of storing and accessing data. Traditionally storage has been stored using block and file structures. Just to summarise all of these:

Representation of traditional block 

based storage, with data stored 

in equal sized block on disk

Block storage can be thought of in terms of LUNs (Logical Unit Numbers) that are presented to servers and applications as discrete disks that act as local devices on the server - essentially buckets containing blocks of data. SAN infrastructure is the mechanism by which the data is presented to servers and applications using the standard scsi protocols over fibre channel, iscsi and FCoE etc. Access to block data is controlled through the storage array through host mapping and further through fibre channel switches through zoning.

File based storage, using heirarchical files and folders 

common on NAS devices.

File storage is presented as folders and/or directories over the network using the NFS and CIFS protocols from NAS devices (“filers”). The structure can be thought of as nested drawers of files that are shared out to devices on a network for access by anyone with the relevant credentials for authentication. NFS has been experiencing something of a revival in the virtualisation space, being highly proficient in presentation of datastores to virtual environments.

Object based storage can be represented 

as a lookup table containing information 

needed to retrieve the object from storage.

Object Based Storage introduces the concept of a flat structure with unique identities for data objects. Each object has its’ own hash signature which is created according to the contents of the object.  Additionally, the metadata (or descriptive header) of an object can contain a great deal more than a traditional file. The data is accessed through the http protocol using the established authentication methods associated with it (such as X-Auth). It is worth noting that block and object storage are not mutually exclusive. The two complement each other with block storage providing the back-end and the object storage software providing the features and functionality off-array.  

Is object-based storage new technology?

No. It existed for some time in some form as CAS (variably known as Content Addressed, Addressable or Aware Storage).  In fact, the concept has been around for nearly 20 years but, necessity driving innovation, companies have developed their object storage strategies at an accelerated pace over the last 6 years or so in response to the rising need to tame their vast data estates and make them as accessible, responsive, contained and cost-effective as possible. 

Why is object storage better than traditional storage?

It is not better in all cases, such as pure file-sharing environments and the frequent access and update requirements of a relational database, but there are some definite advantages in a number of areas. Object-based storage can certainly be a cheaper alternative to the expensive, high end storage products that use proprietary management and administration tools. Cheaper back-end block storage can be used with an object storage software system to provide all of the additional features and functionality. Another benefit comes with the object metadata. With traditional block and file storage, a content management system, often with a relational database, would have to be used to store additional information on a file that cannot be stored with it. The objects in object storage have this information contained within them and it is highly customisable.

How and where is object storage used?

Any organisation that has a requirement to create and/or store large amounts of relatively static data would find object-based storage extremely effective. Examples include

• Cloud providers make extensive use of object storage due to its’ scalability, cost efficiency and ease of interface with cloud portal development.
• Social networking sites with thousands and perhaps many millions of pictures, videos and texts (Facebook, Twitter, Linkedin).
• Publishing companies that produce and are required to keep periodical publications for use in stats gathering and Big Data analysis.
• Healthcare organisations that are required to maintain a repository of scan and x-ray images, which are often used once and then kept to be referred to infrequently, if at all.

The common factor is evident from this list. Vast quantities of “unstructured” data that needs to be stored at as low a cost and footprint as possible

Is object storage really “infinitely scalable”?

Well, “infinite” is an over-used expression in the IT world but one could, in theory, continue to add storage nodes until you run out of money, space or time to an object storage environment. The object storage management system can accommodate continued additions to the back-end storage as long as the metadata and ID references can be maintained on the object storage nodes.

How is availability and redundancy maintained in object storage?

A key attribute of object storage is that copies of the objects can be stored, accordingly to definable policies, across multiple locations locally, nationally and internationally making it truly resilient to local DR scenarios.

Which vendors have implemented it?

Several players have emerged with a head above the clouds (pun intended) in the object storage arena, notably:-

• EMC – ISOLON, ATMOS, Centera
• DirectData Networks – Web Object Scalar (WOS)
• NetApp - StorageGRID
• Scality – Software based “RING” scale-out storage solutions
• HDS – HCP (Hitachi Content Platform)

Others are coming along behind, and undoubtedly every storage vendor has their own agenda in this area.