January 2014

Welcome to 2014 (or MMXIV)!

I'll spare you another round of forecasts about what might and might not happen in the storage industry in 2014, there have been plenty of those lists published in the past couple weeks!
Instead, let me summarize a couple noteworthy news for you - which of course is also a way to hint at what -I personally believe- will be important trends in 2014!
One battle certainly to continue is the battle over the control over storage or rather the "location of the intelligence" in storage: remember 10 years ago: common wisdom was that most of the intelligence would be located in the SAN and the back-end storage would be mere commodity (EMC Invista!). Well, it turned out that customers preferred to invest in sophisticated storage controllers while the network became a commodity...
Fast forward to 2013: There is a lot of hype around SDS (Software Defined Storage), which would relegate the "intelligence" to some "open source" storage controller running on a commodity server.
Hypervisor vendors like VMware and Microsoft on the other hand are integrating more of the storage controls into their hypervisors and hope to make storage intelligence obsolete. And obviously related to that is the discussion and battle about block or file access to storage.
The article here outlines reasons why VMware needs to step up efforts on NFS!
But watch out, on the block side of the protocol segment, the FCIA is finalizing 32Gbps Fibre Channel - or Gen6 Fibre Channel as they are going to call it.

According to "The Register", 2013 was the year "..when Google made Tape cool again"!
Now there is more exciting news to be expected in 2014: Read this summary from the Fujitsu "IT Global Summit" and the "Nanocubic coating" being introduced for magnetic tape!
So watch this space for announcements from IBM during 2014!

Talking about IBM, there's two recent news that clearly show the direction my company is taking: First, the beta availability of ICStore, a unique implementation of "cloud gateway" technology for our Storwize platform providing data protection " ..that will allow organizations to use multiple cloud storage services interchangeably, reducing dependence on any single cloud vendor and ensuring that data remains available even during service outages."

And second, the rollout of WaaS ("Watson as a Service") in late 2013:

“The next generation will look back and see 2013 as a year of monumental change,” said Stephen Gold, vice president of the Watson project at IBM.

“This is the start of a shift in the way people interact with computers.”

IBM is wielding Watson in a fight to control the world of cloud computing — huge collections of computer servers connected over the Internet — with other big technology companies like Amazon.com, Google and Microsoft. It is no coincidence that IBM discussed its Watson news the same week Amazon was hosting clients at a conference here to pitch its own computing cloud, called Amazon Web Services or A.W.S."

This is definitely a key element of IBM's plans to resume revenue growth in 2014!

And by the way: Probably the last acquisition IBM made in 2013 is immediately related to cloud services: Aspera, a company with ".. an Emmy-award-winning, high-speed bulk data transfer protocol."

So all the best to you for 2014 and let's hope that the European economy continues on the positive trend that we seem to have started in Q3 2013!

December

Hot off the press:
IDC's Q3  "Worldwide External Disk Storage Systems Factory Revenue" numbers.
"Despite the revenue contraction in the third quarter, we see strong demand for offerings targeted at heavily virtualized environments, such as integrated infrastructure," said Eric Sheppard, research director, IDC storage. "However, this demand has been offset by several factors: reduced spending from the U.S. government, increased use of storage efficiency technologies, increased investment in public cloud capacity, and general price pressures associated with increased competitive environments."
See the detailed report here!

November II

It has been quite a while since the last announcement of bigger HDDs: We have had 4 TB disk drives available for about 24 months now! In the past couple weeks however, there seems to be renewed activity in the segment of high-capacity HDDs with HGST (Hitachi Global Storage; subsidiary of Western Digital) starting sale of the much-anticipated 6 TB Helium-drive (add two more platters; same areal density!).

Western Digital (WD) adding to the buzz with their plans (no product yet!) to provide increased areal density using HAMR (heat assisted magnetic recording), a technology which is in development over at Seagate as well!

The third promising candidate in the race to squeeze more bits into a 3.5 inch real-estate is "shingled magnetic recording" dubbed as SMR, details are outlined here.
While this technology basically uses hardware that is available today, it kind of changes some of the very characteristics of HDDs: In-place updates become impractical to impossible with the shingled, intertwined tracks (by the way: we have used this approach in tape drives for many years!). Means that HDDs using that technology will be used preferably for data which is not updated, so archive-type of usages! That's the reason why vendors are actually looking into using LTFS (linear tape file system) to access and manage SMR type of disk drives! See details from SNIA and WD here!

I have not figured out yet which technologies can be combined and which ones don't go together - but to me it certainly looks like as if the next wave of capacity increases for HDD is upon us! The question remains if the costs will be at an acceptable level: with the consumer market (tablets, smartphones) using mostly flash storage, the volumes for these new technology HDDs will probably remain lower than what we have been used to in the age of desktops and laptops?

And finally, wrapping up my update in spinning rust, here's an interesting report from Backblaze (a backup service provider) on how long these small mechanical wonders are supposed to last:
"For disk drives, it may be that all of them will wear out before they are 10 years old. Or it may be that some of them last 20 or 30 years. If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.
The number that we will be able to compute soon, and the one that is more likely to be useful, is the median lifespan of a new drive. In other words, at what age have half of the drives failed? We are starting to get an idea what the answer will be."
For additional thoughts and details, please also refer to this report from Google on their observations around drive failure expectations.

November

Object Storage - not sure if it's only me but I seem to see a lot of buzz, hype and news around this topic in the past couple months!
So I've started to track some news sites and have compiled some material I want to share with you. To start off, here's a great and very brief summary on what Object Storage actually is and how it's different from file systems: "Instead of organizing files in a directory hierarchy, object storage systems store files in a flat organization of containers (called "buckets" in Amazon S3) and use unique IDs (called "keys" in S3) to retrieve them."
Read the details here. (thanks to my friend John Sing to provide that link!)
There is also some additional details and diagrams on the OpenStack site here.

Now, object storage will most likely be cloud-based storage and two companies have announced two very different products and approaches to actually store objects: In the case of Seagate (disk drives) they propose a new type of hard disk to directly connect "to the cloud":
"Seagate is building hard disk drives with a direct Ethernet interface and object-style API access for scalable object stores, a plan which - if it works - would destroy much of the existing, typical storage stack. Drives would become native key/value stores that manage their own space mapping with accessing applications simply dealing at the object level with gets and puts instead of using file abstractions."

In the case of SpectraLogic (tape libraries) they promote low-cost tape storage as the foundation for object clouds: "Spectra’s Deep Storage story opens with a new RESTful object storage API that Spectra’s dubbed DS3, for Deep Simple Storage Service. DS3 is an extension of the Amazon S3 API that’s becoming a standard for not just cloud storage services but also on-premise data center object storage systems. The object API allows organizations to take advantage of the low cost, and power consumption, of tape and still have their data available to applications written by mere mortals."

Eventually, they both make sense but I think adding a low-cost (and automatic) tape tier makes lots of sense for objects - which tend to be inactive of nature and clients will probably prefer lower costs over split-second retrieve times?!

September II

Lots of news these days, so time for an update and summary. As always in September, industry analysts did release their assessment of the 2013 Q2 storage market.
I have compiled the facts and comments for you - here as a general statement: "...Worldwide external disk storage systems factory revenues posted a year-over-year decline of -0.8%, totaling just over $5.9 billion, in the second quarter of 2013 (2Q13), according to the International Data Corporation (IDC) Worldwide Quarterly Disk Storage Systems Tracker. For the quarter, the total (internal plus external) disk storage systems market generated $7.7 billion in revenue, representing a -5.0% decline from the prior year's second quarter and a slight sequential decline compared to the first quarter of 2013. Total disk storage systems capacity shipped reached 8.2 exabytes, growing 21.5% year over year."

• IDC Press Release and summary of report with market numbers here.
• Computerworld comment here. (by Lucas Mearian)
• The Register comment here. (by Chris Mellor)
• And some more detailed numbers for EMEA specifically here. (by our friends at StorageNewsletter)

In the midterm, the Gartners, IDCs and the likes will have to figure out if they eventually need to include storage provisioned thru cloud services as well? As of today, I think the capacities sold and installed off-premises by companies like Amazon Webservices, Rackspace or Google are not accounted for...they don't procure systems from the legacy storage vendors in most cases, so don't show up in the statistics here!

And actually this links right into the next topic: an interesting podcast by Jon Toigo on cloud:
"...I did think that maybe one of the better models for cloud going forward -- a sustainable business model for cloud, would be cloud that is specialized in holding huge repositories of certain kinds of data. I asked experts about this. Jeff Jonas at IBM, I asked him, would it make sense for a cloud service provider to stand up big data so I don't have to buy the infrastructure myself?"

And in that same week a statement from NetApp on how they think their systems will interact and integrate with the cloud, I personally think they have a great story there: they have rolled out DOT 8.2 starting this year which enables clustered Data ONTAP and adding cloud gateway capability and a couple interfaces to public cloud services should not represent a major hurdle.
See the short 2' video clip here.
But ultimately, it will be our customers call to decide if and how they will entrust their data to any form of "cloud", see the five reasons for - and five reasons against doing it...great writeup by Trevor Pott: "The cloud will inevitably replace all other forms of IT? The cloud is a passing fad?"

And lastly in a surprise but important move, CISCO announced its intent to acquire Whiptail, a vendor of Flash Storage systems. While CISCO in their own press release emphasize the use of Whiptail technology inside their UCS servers, the rest of the industry sees this move mostly as a way of CISCO to expanding their reach into storage. Read this great analysis here.

September

Today I'll continue with a couple flash topics from last month and provide some links and comments regarding the news and developments announced at the recent VMworld conference.

When the SSD hype started first around 2007/2008, there was a widespread belief that SSD would be a temporary and intermediate technology: The "write penalty" and very limited durability seemed to be obstacles to a successful use and implementation in enterprise storage systems. Now, the industry has fixed many of these issues, partly by using smart controller designs (wear-leveling) partly by improving SSD core technology (eMLC, TLC, 3D-Flash, ...)
At the same time, industrialization of more advanced "storage class memory" like PCM (phase change memory) and MRAM (magnetoresistive random access memory) has taken a backseat to the current hype around NAND flash. Read the complete analysis in CW here:
"It's going to be a long time until NAND flash runs out of steam," said Jim Handy, an analyst at Objective Analysis, during a presentation."

There's an interesting possible combination of technologies which so far (to my knowledge) has not been productized: Flash storage and deduplication: Greenbites proposes the use of Flash storage to achieve " ...deduplication tech that has near-zero latency and possibly offers the world's fastest in line deduplication." Seems to make a lot of sense: deduplication has an inherent requirement for very low latency read operations (to compare data patterns while deduplicating at ingest) and Flash is still a very expensive way to store data -so you want to avoid duplicate data. Combining the two seems kind of obvious!
And Violin, one of the most visible startups in the realm of Flash Systems, is getting ready to an IPO!
Looking at how detached from the real-world economy Wall St. recently seems to be, I'm honestly not sure that's a good thing - but some people will earn some serious money...

Unrelated to Flash, the article here caught my interest: "On Tuesday, Nasuni introduced a cloud-to-cloud mirroring option to give customers extra assurance that their data will be available in case of a service outage. The same day, cloud-to-cloud backup vendor Backupify added more choices for where users can have their data sent."
We all have heard about the outages of the likes of Amazon and Google, so backing up your data in multiple clouds seems to be the smart thing to do! Here's a nice way to visualize this (and I wonder how long it will take until we see cloud companies with meteorology-names like Cumulus and Nimbostratus...

On to VMworld which took place in San Francisco (unfortunately, I did not attend but supposedly 22'000 folks did!): while they had a zillion of press releases and announcements, many of the new functions and strategies revolve around two areas: The network and storage.
"VMware NSX is a software-defined network (SDN) that uses controllers and overlay networking. I'll examine just a few of the key aspects of the announcement and how they apply to your data center strategy." See additional background here.

Not to be confused with CISCO VSANs, the storage integration/tiering functions in VMware are now captured under that same terminology: " VMware launched the public beta of VSAN (virtual SAN), its software for pooling server-based flash among multiple physical servers. VSAN creates what is effectively a hybrid storage array using flash and hard disk drives on each of a cluster of servers."

August

304 - This was the magic number for July. 304 hours of sunshine in Zurich in July 2013...most likely won't be repeated for a couple years. And I was lucky enough to get three weeks of vacation during that time!

Now, back to business this week!

There was quite a bit of news coming from the FlashMemory Summit taking place in Santa Clara this week during the otherwise quite days of summer. My today's update will thus be somewhat SSD- and Flash-biased!
Faster, cheaper flash storage is hitting the enterprise market and Computerworld had a great overview on new products and technologies being introduced.
As it has been the case with every piece of IT technology in the past, consumer products lead the way and the volumes achieved there eventually help drive down costs and make components (Flash Memory in this case) viable for enterprise products. In the case of SSD, the Ultrabooks and Tablets are driving up volume shipments, as outlined in CW.

Amazingly enough, there's companies like Facebook who require "the worst flash possible...just make it dense and cheap." Write-once, read-never would be the spec for this type of Flash. I'd think that probably tape storage would make a lot more sense in such an environment?!

As a matter of fact, Flash Memory and SSDs have reached enough momentum in the market to stir some countermeasures and activities on the side of the HDD ("spinning rust") vendors: "The Storage Products Association (SPA) was formed as a mutual benefit nonprofit corporation in California to promote the use and understanding of rotating magnetic media hard drive (RHD) technologies as a critical component of the future digital storage ecosystem."
Companies behind SPA are (Aug. 2013): HGST, Seagate, Toshiba and WD.

On their website, they have compiled a very nice infographic with a couple amazing facts around disk storage!