In the face of today’s data deluge, organizations are realizing that traditional storage architectures aren’t enough. In the traditional paradigm, the answer to storage needs is to buy more servers. But the cost of this approach is unsustainable, let alone the time it would take to scale to the level required. Even if organizations had unlimited time and money to add many new servers, a problem remains. Vertical storage architecture contains bottlenecks that slow performance to a near-crawl.

One of the selling points of software-defined storage (SDS) is that it detaches the programming that controls storage-related tasks from the physical storage hardware. This dramatically reduces costs associated with hardware. Fewer, less-expensive servers can be used to improve both capacity and performance. Administration is simplified and made more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads.

That’s great news for organizations in need of more storage capacity. Gartner recently reported that by 2020, anywhere from 70 to 80 percent of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.

Not only is there far more data than ever before, but most of that data—80 percent—is unstructured. Consequently, storage solutions that offer file systems currently represent 80 percent of the market. While it is widely understood that unstructured data is best managed with a file system, for some reason, many SDS offerings focus solely on block or object store. Few offerings focus on file systems or do them well. Without a file system overlaying this data, it becomes very difficult to manage that data.

All three types of storage are needed, as each has an area of expertise:

• The basic foundation of storage is block. It is used for storing virtual machines or databases, but you need files as well to deal with all the unstructured data.
• The latest rage is object storage, which is used for machine-to-machine/Internet of Things (IoT) transactions and other applications that require extreme scalability. However, it isn’t that much better than block when it comes to managing data.
• The real breadwinner of storage is file systems. They aren’t as hyped as object, but they are the best at handling unstructured data.

Everyone knows that file systems are critical to software-defined storage, and some vendors claim to provide file system with their offerings. However, these file systems are usually based on Samba (see below) and exclude some features most Windows users are used to.

Samba is a freeware module that enables support for SMBs and allows end users to access and use files on the company’s intranet or network. Many in need of a file system have turned to this option to fill in the gaps. However, providing file services through Samba, which is open source, often means going without needed features.

That’s a huge problem. Organizations need file systems in order to deal with unstructured data, but they must have file-related features as well. These include:

• Quota: This feature helps monitor the amount of storage being used. You can set a soft limit quota that will warn you when part of a file system is close to reaching its storage limit but still allow data to be saved. If you set up a hard limit quota, and the quota is reached, no new data can be saved.
• Snapshot: A snapshot is a read-only copy of the contents of a file system or an independent file set taken at a single point in time. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included in the snapshot.
• Tiering: This is a policy that lets you determine where a specific file will go and if and when the file will be migrated between file system pools. You can define both file placement and migration policies. By using a policy you create a filter which designates a specific file type to a particular tier. Tiered storage is more efficient and boosts performance.
• Retention: This feature allows for the automatic creation of a single folder or a hierarchy of folders on file servers to be deleted according to assigned policies.

With the understanding that SDS needs both file systems and robust features, it’s perplexing to look across the vendor landscape and see how few offer effective versions of these essentials elements. Some organizations turn to open source options, but these offer inadequate feature sets. What’s needed is an SDS solution that can accommodate all storage types: object, block and file system. Research solutions carefully to ensure that your organizations gets all the elements it needs to implement a storage solution that truly solves the data deluge problem.