Finally! New Storage Paradigms for Enterprise Petabytes and Exabytes

June 1, 2018

Steve McDowell, Senior Analyst, Moor Insights & Strategy

By Steve McDowell, Senior Analyst, Moor Insights & Strategy
The storage used to be really boring, but it’s not any longer. There’s a number of trends that are influencing how we deploy storage in the enterprise, and what that means to the solutions that IT practitioners are – are, you know, deploying and – and learning to work with.

McDowell said that interconnect disruption is happening. But is direct-attached storage dead, he asked.

The overriding trend is – is data is exploding. The volume and velocity of data arriving into the data centre is out of control, and it’s coming from a variety of sources. It’s coming from the internet of things, it’s coming from external data sets, it’s coming from internal – you know IT and technology people, we like to measure and track everything, right. We have this explosion of data that needs to be processed, it needs to processed close to the source with low latency and – and fast processing.

We talk about IOT. Analytics and machine learning is another disrupter that’s impacting how we think about storage in the data centre, and how we think about information architecture.

Artificial intelligence is rapidly moving out of the realm of scientific computing, more into the enterprise. We’re looking at all of these data, all of these data points, and enterprise is struggling with how do I apply artificial intelligence and machine learning to derive new insights.

Harvard Business Review put a quote out a few months ago that said one-half of one per cent of the data collected by enterprises is actually used for anything. We’re seeing, you know, the AI world stood up and said well we can help solve those problems and drive new insights. Every business today is a technology business.

At the same time, you have these trends influencing what people want to do with their data, and how that influences storage architecture. From a technology provider’s side, there’s like a couple of trends that are also impacting and merging into how we solve these problems.

One is the rise of software defined. So we’re pushing from HCI – Hyper Converged – up from the bottom, composable kinetic infrastructure from the top, and at the end of the day what we’re seeing deployed in data centres for IT is really a kinetic infrastructure to use Dell EMC’s term but – but one where I can leverage resources, and use software to orchestrate those solutions.

Finally, new storage technologies from media and inter-connect are really disrupting the way that we can build these systems. We – we take these new technologies, put software define on top, and it really introduces a world of flexibility into the kind of storage eco system that we have not seen before.

If you look at storage, you’re familiar with kind of this data tier that we think about, right. At the top is very fast hot data that we need right now with no latency and infinite – infinite throughput. At the bottom is kind of the archival glazier, not on the chart, but at the bottom there is glazier storage that we never touch, you know, data that we keep for regulatory and other reasons.

On the media front, this has been disruptive. In 2019, the top tier we’re going to see persistent memory, byte addressable, if you think about byte addressable SSD. That we live in a DIMM slot that kind of changes the way we think about server design and persistence. That impacts file systems, databases, anything that lives on your server and that’s not traditional tiered storage, right, so we think about that differently.

That’s been enabled by technologies like 3D Crosspoint which entails branding Optane. We’re seeing that same technology today, you know, the SSD market is itself segmenting. We’re seeing Optane at 3D Crosspoint being put into SSDs where I have no latency very high throughout, but it’s expensive, and you don’t get the densities you get with traditional 3D NAND. At the same time the 3D NAND space is moving into the mainstream. It’s fast displacing hard drives and server and storage right across the board. It is commoditising, and we are continuing to see innovations there. I think Micron this week announced their QLC technology which gives us higher densities and in greater read performance.

At the bottom, hard drives don’t go away. They will always have a place in at least in the eight to 10 years because the economics are just right for that. So that is happening on the storage media front.

Interconnects are also – you know, this is probably the most exciting time since CN fibre channel arrived in the late 1990s in terms of interconnect disruption which is going to drive some of these kinetic architectures.

iSCSI fibre channel aren’t going away, but they are being replaced by NVMe over Fabric, which gives me – which takes the protocol stack essentially out of the path. So I get very high performance, low latency access to external storage. NVMe over Fabric, you know, it’s not just one thing. We’re seeing that over InfiniBand, we’re seeing that over fibre channel, we’re seeing that over Ethernet. The winners – the winners will shake out over time.

Then finally, all of these technology things are working to define new storage architecture. So I put a box at the top here that says software defined, but that’s inclusive of hyperconverged, composable, converged, all of these different things. What it really means is I can use software orchestration now to treat my resources as fungible to give me as an IT practitioner great leverage in how I deploy storage systems.

Those are kind of the trends that are driving disruption in the storage market. My final slide here is really a call to action for the people in this room. If we look at the Tier 1 OEMs – HPs, the Dells, IBMs, the [wallways] of the world – yes, they move markets and they drive markets, but the trends that are influencing them long term, some of those come out of those organisations, but most often they come out of them more disruptive and non-Tier 1 players. I think as press and analysist, I’m certainly guilty.

We generate a lot of ink for the big names, but I think that does a disservice a little bit to our ultimate consumers of the information that we put out, right, the IT practitioners, and that storage disruption, disruption across all the technologies, is very often not from the Tier 1s.

Leave a Reply