On the differences between Memory-Hierarchy and Intelligent-Tiering
What started as a quiet Saturday of catching up on “stuff” is turning into a wake-up call. Data indeed has value, and collecting information in my many RSS feeds folders, is not equivalent to reviewing them in a timely manner. Since there is so much relevant data that has accumulated, I need to devise a mechanism to tier this large amount of data by value, age, relevancy, urgency, importance, repetitive -which should be de-duplicated and so forth and so on.
I have missed a fun week. Tom Georgens’ statement during NetApp’s F3Q10 Earnings Call has generated a buzz. Having read several posts by my esteemed colleagues, I will add this “Weekly Edition” to the multiple threads.
Since I did not participate in NetApp’s Earnings Call, I can only base my understanding upon the call transcript.
…”frankly I think the concept of tiering is dying. And I probably don’t want to go into a long speech on that, but at the end of the day, the simple fact of the matter is tiering is a way to manage migration of data between fiber-channel based systems and serial ATA-based systems. With the advent of Flash, and we talked about our performance acceleration module, basically these systems are going to large amounts of Flash, which are going to be dynamic with serial ATA behind them, and the whole concept of HSM and tiered storage is going to go away…”
This statement by NetApp’s CEO combines Flash Technology, SSDs, SATA and FC drives with Tiering and HSM. Confusing media technology with use cases and fluidly of data does lead to wrong conclusions.
There is no argument concerning what tiers of memory will be used in future systems. In 2008 I gave a talk at a local Silicon Valley panel I organized about Flash Technology in the Enterprise. During the panel I hosted I did not use a crystal ball attempting to predict the future. The agenda centered on explaining current trends. Most of what I’ve said in 2008 is still valid.
Prices have stabilized, startups came out of stealth, MLC is gaining on SLC, controllers to increase robustness and mitigating (lack of) reliability issues emerged, no more FC HDDs will be sold after 2010, yet nothing really changed.
Ironically, since I limited the examples I’ve used during my talk to products that have already been announced, I used NetApp’s PAM (Mk I) which was already announced in 2008 to make two points: (a) explaining caching to masses and (b) explaining the lack of maturity of Flash, since PAM Mk I used DRAM technology, and explaining (Which is not the same as predictingJ), that PAM Mk II will use Flash Technology.
The example Tom Georgens used is about Memory-Hierarchy Design. Since slower memory is cheaper (I include Silicon Technology, Rotating Media and off-line archiving media all under “Memory”), and thus the corollary is that faster memory is smaller, all computer systems deploy hierarchical memory architectures. Since the cost of a storage system made of SRAMs will equal the national debt, and since the latency of accessing slow drives directly (in a many to many scenario), is larger than the time it will take the user to drink multiple cups of coffee waiting for the data to be retrieved, we all look for the optimal ratios and the relevant technology per each level in the hierarchy. Since the dawn of computer Systems all designers optimized for temporal and spatial localities of references, and designing such that hit rates will increase and miss penalties will decrease.
Indeed NetApp uses Flash Technology to create another level in the hierarchy. It is a good example of optimizing for size and cost. The disappearance of 15k RPM Fibre Channel drives is another valid example of one media Technology being replaced by another. Between Flash Technology and SATA drives, the economics for yet another level diminish. (Avoiding a crystal ball to predict future cost, currently, depending on the drive attributes, the cost per bit is between one and two order of magnitude more expensive between Flash and SATA.)
So, based on all of the above, why do all those frequently used and much appreciated storage tiers keep insisting that the rumors of their death have been greatly exaggerated?
Since the definition of storage tiers as defined by NetApp (“the simple fact of the matter is tiering is a way to manage migration of data between fiber-channel based systems and serial ATA-based systems”…), is wrong, or misleading, or both.
Most of the good arguments have already been made. Without any intention to ignore other posts, I did like the post and several of the comments posted on The Storage Architect.
I also like the post by my good friend Michael Hay.
I am also cognizant of being redundant with some of my points being the Johnny-come-lately to this debate.
Storage Tiering is not “simply a way to manage migration of data between fiber-channel based systems and serial ATA-based systems”. While the media tiers have, are and will change according to the economics of the media, the foundation of Intelligent Tiering will retain its validity.
Since no data ever gets erased, and since data has value associated with it, it is about transparent, automated, fluid movement of the data.
Tiering is definitely not dying. NetApp does not offer this functionality or capability so it makes sense that they would try to divert people’s attention away from tiering. I don’t think customers’ attention will be so easily diverted.
Let’s start with the data already in place. Customers have invested billions of dollars in Fibre Channel, SAS, and SATA disks that make up primary, secondary, and archive tiers. They need intelligent, efficient tools to manage the movement of their data between these tiers. The data is the lifeblood of any organization and its transparent, automated, fluid movement is important.
What is dead are the days of excess and seemingly unlimited budgets to just keep adding more high performance disk drives to satisfy the insatiable need for storage. For this reason customers have to be able to match the value of their data to the appropriate disk technology to meet access requirements and efficiently utilize current infrastructure to squeeze every drop of ROI from it. With fewer people to manage this infrastructure, automating the movement of data through an easy to use policy-based interface adds to ROI by making better use of storage administrators’ time. But here is the real key – making this movement transparent. Transparent, meaning the storage admin does not have to “reconnect” applications after data has been moved, nor do users have to be notified that their data now resides in a different location. In other words, the data has moved to a different tier, but the access path from the original point of storage is preserved, no matter how many times the data moves between tiers.
So the real discussion is not about tiering, but about transparency. The SAN vendors can do this at the block level and have been able to for years. But for the NAS vendors, doing this at the file level has only been achieved by BlueArc. We call it Intelligent Tiering and have been doing it since 2006.
Another key advantage of Tiering is the ability to extend our transparent, Intelligent Tiering to NAS systems other than our own. The ability to transparently migrate data to foreign file systems is what has enabled HDS and BlueArc to migrate from BlueArc to HCP or BlueArc to NetApp, with the major difference being that the data migration engine is intrinsic to the NAS device and not an external appliance. (This is better articulated by Michael Hay). We also provide software to address spikes in demand for data on secondary or archive tiers so that high performance access is assured no matter what type of disk drive the actual data is being read from. So is it any wonder that the biggest NAS vendor in the market would like everyone to think “tiering is dying”? They can’t come close to offering Intelligent Tiering, and if I were their customer I may start to think they are seriously out of touch with my needs.
The computer used to render the scene was CCI’s Power6/32 supermini. As a young engineer, I was part of the design team for the Power6/32 UNIX Supermini. The CCI Power6/32 Unix Supermini was selected by Lucasfilm as the CGI rendering platform for its speed.
What surprises one finds far away from home
