All Flash Arrays: Hybrid Means Compromise
January 7, 2016 7 Comments
Sometimes the transition between two technologies is long and complicated. It may be that the original technology is so well established that it’s entrenched in people’s minds as simply “the way things are” – inertia, you might say. It could be that there is more than one form of the new technology to choose from, with smart customers holding back to wait and see which emerges as a stronger contender for their investment. Or it could just be that the newer form of technology doesn’t yet deliver all of the benefits of the legacy version.
The automotive industry seems like a good example here. After over a century of using internal combustion engines, we are now at the point where electric vehicles are a serious investment for manufacturers. However, fully-electric vehicles still have issues to overcome, while there is continued debate over which approach is better: batteries or hydrogen fuel cells. Needless to say, the majority of vehicles on the road today still use what you could call the legacy method of propulsion.
However, one type of vehicle which has been successful in gaining market share is the hybrid electric vehicle. This solution attempts to offer customers the best of both worlds: the lower fuel consumption and claimed environmental benefits of an electric vehicle, but with the range, performance and cost of a fuel-powered vehicle. Not everybody believes it makes sense, but enough do to make it a worthwhile venture for the manufacturers.
Now here’s the interesting thing about hybrid vehicles… the thing that motivated me to write two paragraphs about cars instead of flash arrays… Nobody believes that hybrid electric vehicles are the permanent solution. Everybody knows that hybrids are a transient solution on the way to somewhere else. Nobody at all thinks that hybrid is the end-game. But the people who buy hybrid cars also believe that this state of affairs will not change during the period in which they own the car.
Hybrid Flash Arrays (HFAs)
There are two types of flash storage architecture which could be labelled a hybrid – those where a disk array has been repopulated with flash and those which are designed specifically for the purpose of mixing flash and disk. I’ve talked about naming conventions before, it’s a tricky subject. But for the purposes of this article I am only discussing the latter: systems where the architecture has been designed so that disk and flash co-exist as different tiers of storage. Think along the lines of Nimble Storage, Tegile and Tintri.
Why do this? Well, as with hybrid electric vehicles the idea is to bridge the gap between two technologies (disk and flash) by giving customers the best of both worlds. That means the performance of flash plus its low power, cooling and physical space requirements – combined with the density of disk and its corresponding impact on price. In other words, if disk is cheap but slow while flash is fast but expensive, HFAs are aimed at filling the gap.
Hybrid (adjective)of mixed character; composed of different elements.bred as a hybrid from different species or varieties.
As you can see there are a lot of synergies between this trend and that of the electric vehicle. Also, most storage systems are purchased with a five-year refresh cycle in mind, which is not dissimilar to the average length of ownership of a car. But there’s a massive difference: the rate of change in the development of flash memory technology.
In recent years the density of NAND flash has increased by orders of magnitude, especially with the introduction of 3D NAND technology and the subsequent use of Triple-Level Cell (TLC). And when the density goes up, the price comes down – closing the gap between disk and flash. In fact we’re at the point now where Wikibon predicts that “flash … will become a lower cost media than disk … for almost all storage in 2016″:
Image courtesy of Wikibon “Evolution of All-Flash Array Architectures” by David Floyer (2015)
That’s great news for customers – but definitely not for HFA vendors.
Conclusion
And so we reach the root of the problem with HFAs. It’s not just that they are slower than All Flash Arrays. It’s not even that they rely on the guesswork of automatic tiering algorithms to move data between their tiers of disk and flash. It’s simply that their entire existence is predicated on the idea of being a transitory solution designed to bridge a gap which is already closing faster than they can fill it.
If you want proof of this, just look at the three HFA vendors I name checked earlier – all of which are rushing to bring out All Flash versions of their arrays. Nimble Storage is the only one of the three to be publicly listed – and its recent results indicate a strategic rethink may be required.
When it comes to hybrid electric vehicles, it’s true that the concept of mass-owned fully-electric cars still belongs in the future. But when it comes to hybrid flash arrays, the adoption of All-Flash is already happening today. The advice to customers looking to invest in a five-to-seven year storage project is therefore pretty simple: Mind the gap.
flashdba 
Wikibon’s study is centred on 15K and 10K disks. Currently SSD is about 6x the cost of NL-SAS. A couple of years ago it was 10x. In a few year it will be 3x. At some point it will be cheaper than disk. For now the TCO arguments for all flash are mostly based on flash having better data reduction than disk and only looking at expensive disk. This is a false comparison as lots of disk now has thin provisioning, compression, duplication etc., and a lot of hybrids use NL-SAS. I think most people see this weakness which is why at the moment a lot less is spent on all flash than on systems that involve disk. It won’t always be this way, but it will be for a few years.
You make a good point on the challenge tiering architectures have, but I suggest you research Nimble’s architecture which is caching based, no tiering involved. The do per volume service levels of pin in flash, standard caching and no caching can – there is no reason to cache a backup target …
Nimble Storage challenge the traditional tiering approach just as much as the All Flash vendors, just coming at the problem from a CPU centred perspective rather than a media centred one. They have said they are about to launch an all flash array. It will be very interesting to find out how their All Flash fits next to their Hybrid.
To be honest, whether the architecture is built on caching or tiering it still has the same fundamental flaw in my opinion: unpredictability. For data centres to adopt flash storage in a strategic manner, modern solid-state storage systems need to be able to deliver levels of sustained and predictable performance which are workload agnostic. Any architecture designed to leverage tiering and/or caching as a bridge between disk and flash is essentially making a trade off between cost and predictability. The unspoken question is “Would you like to accept more performance risk in exchange for a lower acquisition cost?”
The debate continues over whether flash is cheaper than disk, or at what point it will become so. I do not think we will ever get a consensus until it’s indisputably so – and I’m happy to leave the argument to organisations like Wikibon, Gartner or IDC. What I do stand by, however, is the view that the collective business strategy of the Hybrid Flash Array vendors is in serious doubt. I think the market performance of Nimble Storage goes a long way to support that view.
You make a good point on predictability. Nimble see cache hits about 97% of the time and a few applications might be sensitive to that. This was the reason they introduced the pinned service level to ensure predictability. As they plan an AFA I am guessing they are seeing more demand for that answer – which could be as much market perception as actual need other than a handful of apps. It would be very dangerous to read anything into short term market movements (my degree is economics), every eventually great firm sees big dips and every dead firm sees big dips – other than the game is afoot little meaningful can be inferred. On the other hand I think you make a good point on there being a massive question (IMO on a 3-5 year timeframe) over hybrid only approaches. I think the spotlight will be on Nimble over the next couple of months as they have been the most successful of all the new storage vendors – >7,000 customer, adding >600 in their last quarter that disappointed the market – but they will need to say more on where they are headed. Interesting times 🙂
I agree with you on the dangers of reading too much into stock market activity. And I’ve got no particular problem with Nimble by the way, it’s just that they are the only public HFA vendor to examine.
I believe that caching (and tiering for that matter) only works when managed from the top of the stack and not the bottom. I.e. it must be application aware – in fact, business-rule aware. Anything sitting lower down (e.g. in the database or on the storage) is too far isolated and is therefore doomed to playing catch up by trying to predict the future based on the past. You can of course take the trouble to pin certain elements in cache but then you have a) created an additional management complexity that must be constantly monitored and maintained, and b) created yourself a poor-mans All Flash Array.
And I 100% agree with your statement that these are interesting times!
I think that caching is workflow dependent. Given how well Nimble have done I think their algorithms must be pretty good for general IT workloads and unless your an big enterprise you may not have anything that doesn’t fit what they currently do. That might be behind their AFA, large scale nasty enterprise apps. I think some approaches like SAP HANA present a different challenge as it ends up nearly 100% writes, which Nimble is pretty good at. (HANA shows that sometimes the best answers aren’t about storage at all …) 3D Xpoint is almost with us, so nothing is standing still 😉
DSSD is talked up but with Michael’s track record I expect it won’t end up living up to it’s promise.
Hybrid cars really are a bad idea. Youre paying for 2 engines which will lead to less range owning to the added weight. Not to mention extra maintenence and insurance costs. Science has yet to develop a method to produce Hydrogen in large scale at a reasonable cost. The way to go is with NG powered fuel cells in cars made of carbon fibre which owning to less weight have great range. Theres tons of NG in NA which could n permanently replace arab oil imports which would have major benefits for world piece and economic development.
As to hybrid flash thats a good idea from my experience with having a hybrid flash disk in my portable server.
My Prius cost me £10,000, is less than £200 pa to insure full comp and in three years I have not had a year when servicing has cost me more that £250 (that included 2 tyres). I agree with Terry that long term Hydrogen will be the answer. ItThe Prius range is about 500 miles compared to my Honda Civic diesel which is about 600 miles. The Civic is more ‘interesting’ (150hp, great turbo) but the Prius is an auto and bigger. There are often compromises, but the Prius does a pretty good job, particularly on the M25. I hate people waving at me thinking I’m a taxi. (Don’t get a Honda Insight, just a bad version of a Prius).