Enterprises switch from legacy to software-defined storage (SDS) for many of the same reasons they chose other software-defined networking (SDN) technologies. It’s more agile and flexible. Separating the storage from the hardware often reduces costs, and it can more efficiently be deployed for data-intensive workloads.

“IDC believes the future of storage is software-defined, server-based, and cloud-connected,” said IDC analyst Ritu Jyoti. “So it ties to all the mega trends.”

IDC forecasts spending on software-defined storage will grow from about $7 billion this year to $9.1 billion in 2019. Despite the projected market growth, however, it’s still not very well understood.

What exactly is SDS and what legacy systems is it replacing? Essentially, it’s a system of hardware and software in which the software storage is decoupled from the hardware.

IDC says this autonomous software stack must be able to run on commodity, off-the-shelf hardware.

The Storage Network Industry Association (SNIA) lists additional characteristics that it says SDS systems must include. These are automation, standard interfaces, scalability, and transparency. The latter gives customers the ability to monitor and manage their own storage consumption against available resources and costs.

Vendors sell different SDS products that support block, file, object, and/or hyperconverged storage.

And most industry analysts and storage veterans agree that these products are replacing legacy hardware-based storage systems.

Traditional storage systems

The two primary types of traditional external storage systems are network attached storage (NAS) and storage area networks (SANs). NAS is a single device that operates on data files. It connects to a local area network, usually through Ethernet.

A SAN is a network of storage devices that operate on disk blocks, hence the name block storage. SANs commonly use fibre channel to connect the devices.

“Traditional storage systems have been appliance-based where if I buy a certain amount of block storage on SAN, that’s all I can use that storage for,” said Dell EMC’s Manuvir Das, SVP and GM for unstructured data storage, in an earlier interview with SDxCentral. “It’s purpose built and only useable for one thing.”

Cloud computing and SDN, however, showed new models that could be applied to storage as well.

“I can have standardized hardware, and deploy software on that, and that gives me a level of elasticity,” Das said. “If I need to expand, I just deploy that software on more servers. If I need to shrink, I just remove the software from some servers.”

Two big factors – the rise of cloud computing and object storage – helped usher in software-defined storage. Amazon Web Services (AWS), with its public cloud and object storage called Amazon Simple Storage Service (Amazon S3), played a major role in both.

“Amazon came up with this [S3] solution that you can provision on demand,” Jyoti said. “It’s very cheap and deep storage, and you can get it without even knowing what the heck is object storage.”

Object storage

Object storage manages data as objects, as opposed to files or blocks, and is typically used in the cloud. It’s a hierarchy-free method, meaning it does not use a directory tree like file storage systems. This makes retrieving data more efficient because there are no bottlenecks created by complex directory systems.

Object storage is ideal for unstructured data such as media and web content.

In addition to AWS, Microsoft Azure, and Google offer their own public cloud object storage. Many traditional storage vendors have added object storage to their portfolios as well.

IBM acquired object-storage startup Cleversafe in 2015 and based its IBM Cloud Object Storage on this technology.

Hitachi Data Systems calls its object storage play the Hitachi Content Platform portfolio. In addition to storing data, it allows companies to share and manage this data from anywhere, on any device.

Dell EMC’s cloud-scale object storage platform, called Elastic Cloud Storage (ECS), recently added a service that enables hybrid deployment models for ECS.

And at Dell EMC World 2017 in May, the company previewed “Project Nautilus,” a new SDS project for storing and analyzing high volumes of streaming Internet of Things (IoT) data. Once it moves from a project to a product, it will allow businesses to make real-time decisions based on streaming device data.

The top three SDS vendors (in no particular order) are Dell EMC, IBM, and VMware, Jyoti said.

“Incumbent vendors like Dell EMC and IBM are facing up to the reality and starting to offer more software-based storage solutions as the market demands,” said Gartner analyst Arun Chandrasekaran. “On the other side, companies like VMware and Red Hat that, in the past did not have a strong storage portfolio, have either made acquisitions or organically launched new products.”

IDC predicts market growth in file-, object-, and block-based SDS. It put the compound annual growth rate (CAGR) of these three categories between 2014 and 2019 at 10.5 percent, 16.2 percent, and 7.5 percent, respectively.

But the SDS product category that will see the most growth is hyperconverged SDS, growing at a CAGR of 59.7 percent during this same period, according to the analyst group.

Hyperconverged SDS

Hyperconverged infrastructure (HCI) systems provide tightly integrated compute, networking, and storage, all of which can scale. Software is key to these systems, which run on commodity hardware and are tailored for virtual workloads.

HCI systems are attractive to customers because “they support improved performance using flash storage, provide an expanded set of storage services, and enable better integration of the management interface with cloud orchestration platforms,”Jyoti wrote in an IDC white paper.

A number of vendors offer HCI systems that include SDS. These include Nutanix, SimpliVity (acquired by Hewlett Packard Enterprise earlier this year), and Dell EMC’s ScaleIO, a hardware-agnostic, enterprise-grade software-defined block storage solution.

VMware’s vSAN is the virtualization company’s SDS play and one of the three components of VWmare’s HCI offering for software-defined data centers.

Additionally, NetApp and Cisco last month launched their latest FlexPod product that uses Cisco hardware for storage and includes NetApp SolidFire all flash-storage.

HCI systems, which Gartner includes in its “infrastructure SDS” category, will continue to see strong growth, Chandrasekaran said.

“A lot of customers are interested in SDS today because of cost-related reasons,” he explained. “They want to simplify their cost of acquisition, they want the flexibility to use commodity hardware, and they want lower upgrade and lower hardware costs.”

But this decoupling of hardware and software also presents a challenge to greater SDS deployment, he added.

When companies buy a traditional appliance, it’s already pre-tested to work with the software. Buying software and deploying it on any hardware requires additional engineering and testing.

“The reality is there aren’t many organizations with a strong engineering team that can do this testing,” he said.“ Having that integrated experience is very critical for the users. The [SDS] industry, as a whole, needs to very clearly move in that direction.”