November

Object Storage - not sure if it's only me but I seem to see a lot of buzz, hype and news around this topic in the past couple months!
So I've started to track some news sites and have compiled some material I want to share with you. To start off, here's a great and very brief summary on what Object Storage actually is and how it's different from file systems: "Instead of organizing files in a directory hierarchy, object storage systems store files in a flat organization of containers (called "buckets" in Amazon S3) and use unique IDs (called "keys" in S3) to retrieve them."
Read the details here. (thanks to my friend John Sing to provide that link!)
There is also some additional details and diagrams on the OpenStack site here.

Now, object storage will most likely be cloud-based storage and two companies have announced two very different products and approaches to actually store objects: In the case of Seagate (disk drives) they propose a new type of hard disk to directly connect "to the cloud":
"Seagate is building hard disk drives with a direct Ethernet interface and object-style API access for scalable object stores, a plan which - if it works - would destroy much of the existing, typical storage stack. Drives would become native key/value stores that manage their own space mapping with accessing applications simply dealing at the object level with gets and puts instead of using file abstractions."

In the case of SpectraLogic (tape libraries) they promote low-cost tape storage as the foundation for object clouds: "Spectra’s Deep Storage story opens with a new RESTful object storage API that Spectra’s dubbed DS3, for Deep Simple Storage Service. DS3 is an extension of the Amazon S3 API that’s becoming a standard for not just cloud storage services but also on-premise data center object storage systems. The object API allows organizations to take advantage of the low cost, and power consumption, of tape and still have their data available to applications written by mere mortals."

Eventually, they both make sense but I think adding a low-cost (and automatic) tape tier makes lots of sense for objects - which tend to be inactive of nature and clients will probably prefer lower costs over split-second retrieve times?!

September II

Lots of news these days, so time for an update and summary. As always in September, industry analysts did release their assessment of the 2013 Q2 storage market.
I have compiled the facts and comments for you - here as a general statement: "...Worldwide external disk storage systems factory revenues posted a year-over-year decline of -0.8%, totaling just over $5.9 billion, in the second quarter of 2013 (2Q13), according to the International Data Corporation (IDC) Worldwide Quarterly Disk Storage Systems Tracker. For the quarter, the total (internal plus external) disk storage systems market generated $7.7 billion in revenue, representing a -5.0% decline from the prior year's second quarter and a slight sequential decline compared to the first quarter of 2013. Total disk storage systems capacity shipped reached 8.2 exabytes, growing 21.5% year over year."

• IDC Press Release and summary of report with market numbers here.
• Computerworld comment here. (by Lucas Mearian)
• The Register comment here. (by Chris Mellor)
• And some more detailed numbers for EMEA specifically here. (by our friends at StorageNewsletter)

In the midterm, the Gartners, IDCs and the likes will have to figure out if they eventually need to include storage provisioned thru cloud services as well? As of today, I think the capacities sold and installed off-premises by companies like Amazon Webservices, Rackspace or Google are not accounted for...they don't procure systems from the legacy storage vendors in most cases, so don't show up in the statistics here!

And actually this links right into the next topic: an interesting podcast by Jon Toigo on cloud:
"...I did think that maybe one of the better models for cloud going forward -- a sustainable business model for cloud, would be cloud that is specialized in holding huge repositories of certain kinds of data. I asked experts about this. Jeff Jonas at IBM, I asked him, would it make sense for a cloud service provider to stand up big data so I don't have to buy the infrastructure myself?"

And in that same week a statement from NetApp on how they think their systems will interact and integrate with the cloud, I personally think they have a great story there: they have rolled out DOT 8.2 starting this year which enables clustered Data ONTAP and adding cloud gateway capability and a couple interfaces to public cloud services should not represent a major hurdle.
See the short 2' video clip here.
But ultimately, it will be our customers call to decide if and how they will entrust their data to any form of "cloud", see the five reasons for - and five reasons against doing it...great writeup by Trevor Pott: "The cloud will inevitably replace all other forms of IT? The cloud is a passing fad?"

And lastly in a surprise but important move, CISCO announced its intent to acquire Whiptail, a vendor of Flash Storage systems. While CISCO in their own press release emphasize the use of Whiptail technology inside their UCS servers, the rest of the industry sees this move mostly as a way of CISCO to expanding their reach into storage. Read this great analysis here.

September

Today I'll continue with a couple flash topics from last month and provide some links and comments regarding the news and developments announced at the recent VMworld conference.

When the SSD hype started first around 2007/2008, there was a widespread belief that SSD would be a temporary and intermediate technology: The "write penalty" and very limited durability seemed to be obstacles to a successful use and implementation in enterprise storage systems. Now, the industry has fixed many of these issues, partly by using smart controller designs (wear-leveling) partly by improving SSD core technology (eMLC, TLC, 3D-Flash, ...)
At the same time, industrialization of more advanced "storage class memory" like PCM (phase change memory) and MRAM (magnetoresistive random access memory) has taken a backseat to the current hype around NAND flash. Read the complete analysis in CW here:
"It's going to be a long time until NAND flash runs out of steam," said Jim Handy, an analyst at Objective Analysis, during a presentation."

There's an interesting possible combination of technologies which so far (to my knowledge) has not been productized: Flash storage and deduplication: Greenbites proposes the use of Flash storage to achieve " ...deduplication tech that has near-zero latency and possibly offers the world's fastest in line deduplication." Seems to make a lot of sense: deduplication has an inherent requirement for very low latency read operations (to compare data patterns while deduplicating at ingest) and Flash is still a very expensive way to store data -so you want to avoid duplicate data. Combining the two seems kind of obvious!
And Violin, one of the most visible startups in the realm of Flash Systems, is getting ready to an IPO!
Looking at how detached from the real-world economy Wall St. recently seems to be, I'm honestly not sure that's a good thing - but some people will earn some serious money...

Unrelated to Flash, the article here caught my interest: "On Tuesday, Nasuni introduced a cloud-to-cloud mirroring option to give customers extra assurance that their data will be available in case of a service outage. The same day, cloud-to-cloud backup vendor Backupify added more choices for where users can have their data sent."
We all have heard about the outages of the likes of Amazon and Google, so backing up your data in multiple clouds seems to be the smart thing to do! Here's a nice way to visualize this (and I wonder how long it will take until we see cloud companies with meteorology-names like Cumulus and Nimbostratus...

On to VMworld which took place in San Francisco (unfortunately, I did not attend but supposedly 22'000 folks did!): while they had a zillion of press releases and announcements, many of the new functions and strategies revolve around two areas: The network and storage.
"VMware NSX is a software-defined network (SDN) that uses controllers and overlay networking. I'll examine just a few of the key aspects of the announcement and how they apply to your data center strategy." See additional background here.

Not to be confused with CISCO VSANs, the storage integration/tiering functions in VMware are now captured under that same terminology: " VMware launched the public beta of VSAN (virtual SAN), its software for pooling server-based flash among multiple physical servers. VSAN creates what is effectively a hybrid storage array using flash and hard disk drives on each of a cluster of servers."

August

304 - This was the magic number for July. 304 hours of sunshine in Zurich in July 2013...most likely won't be repeated for a couple years. And I was lucky enough to get three weeks of vacation during that time!

Now, back to business this week!

There was quite a bit of news coming from the FlashMemory Summit taking place in Santa Clara this week during the otherwise quite days of summer. My today's update will thus be somewhat SSD- and Flash-biased!
Faster, cheaper flash storage is hitting the enterprise market and Computerworld had a great overview on new products and technologies being introduced.
As it has been the case with every piece of IT technology in the past, consumer products lead the way and the volumes achieved there eventually help drive down costs and make components (Flash Memory in this case) viable for enterprise products. In the case of SSD, the Ultrabooks and Tablets are driving up volume shipments, as outlined in CW.

Amazingly enough, there's companies like Facebook who require "the worst flash possible...just make it dense and cheap." Write-once, read-never would be the spec for this type of Flash. I'd think that probably tape storage would make a lot more sense in such an environment?!

As a matter of fact, Flash Memory and SSDs have reached enough momentum in the market to stir some countermeasures and activities on the side of the HDD ("spinning rust") vendors: "The Storage Products Association (SPA) was formed as a mutual benefit nonprofit corporation in California to promote the use and understanding of rotating magnetic media hard drive (RHD) technologies as a critical component of the future digital storage ecosystem."
Companies behind SPA are (Aug. 2013): HGST, Seagate, Toshiba and WD.

On their website, they have compiled a very nice infographic with a couple amazing facts around disk storage!

July

Recently, I had one of these "aha" moments, situations where things come together and you suddenly realize that a thought or idea you had -only as a faint impression- suddenly becomes real! Most often this happens to me when I read an article which exactly sums up -in an eloquent way-what I had only been guessing or feeling.
The "Register" article which had that effect on me last week is this one: "Amazon, currently estimated to have $2bn in storage sales with an expected 65 per cent growth in storage revenue in each of the next 2-3 years which will make them a top 3 player in the storage industry".

Top 3 player in the storage industry? For decades, the likes of EMC, IBM, HP, NetApp, HDS have been switching positions in the quarterly IDC and Gartner charts... now Amazon?
How come?
Well, here's why: "The move to cloud computing means fewer companies will buy huge numbers of servers and storage arrays for their own use. Over the next five years, Morgan Stanley’s expects that 3 percent to 17 percent of current spending could be sucked up by cloud-based IT service providers."
And this is how "cloud storage" gets to "top three": 17% of the storage market today actually would be the #2 spot (behind market leader EMC).
So customers only hold very limited amounts of active data in their "on-premise" storage farms (most likely on SSDs) and push/pull everything else (including archives and backups) to the cloud.
As we all know, cloud providers like Amazon are not known as buyers of huge amounts of storage systems. Like Google and Facebook and other "hyperscale data" companies, they build their own infrastructure, based on chips from Intel or AMD and disk drives directly from the vendors like WD and Seagate.
Now, preparing for this era and in order to allow enterprises to mitigate the dependence on one specific cloud technology or vendor, IBM is working on a project (this was actually demo'ed at the Edge 2013 conference in June) to provide redundancy by using an on-premise IBM Storwize V7000 system connected to multiple clouds (RAID'ed clouds if you will): Please read the research paper here!

On to the network: (after all "cloud" is enabled only by powerful networks!)
You may not have realized it, but Ethernet did celebrate its 40th birthday this past spring, here's an interview with Bob Metcalfe, the inventor of Ether-Net!
At the same time, 16 Gbps Fibre Channel is gaining momentum as is reported here. The fact that CISCO recently announced their own 16 Gbps SAN director definitely is an endorsement for the Fibre Channel market and provides clients a choice of CISCO or Brocade gear for their storage networks!

And lastly, as many of us will be enjoying vacations in the coming weeks, exploring new countries and making new experiences, here's some great thoughts related to "live-blogging": "Even for those who are always pulling out their phones to snap pictures or shoot videos, to text or tweet or tumble or otherwise share the moments of their being, the pleasure lies mainly in the recording, not in the record. The act of recording is itself a disposable experience. The tools for recording and sharing are disposable as well. They get old."
Have fun!

June

Hyperscale, Hyper-V, and OpenStack are my topics today! And of course SDE!
You have not heard about SDE?
Well, as a start, read Steve Duplessie's comments here!

Now, with that put into perspective, let's look at some recent exciting news in the realm of cloud storage: The OpenStack Open Source Cloud Mission:
"... to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable."

IBM is very actively participating and contributing to that initiative and if you can spend half an hour, listen to this presentation delivered at the recent OpenStack Summit conference!

Now, hyperscale storage is a slightly different beast and could probably be used best to describe the architectures and challenges that some of those companies face who are actually providing these cloudy storage services: "The term hyperscale storage is coming into vogue to describe systems capable of rapid, efficient expansion to handle massive quantities of data from Web-serving, database, data analysis,high-performance computing and other especially busy applications"
Please read the complete article in searchstorage here!

And obviously, part of hyperscale storage requirements will increasingly be addressed by high-performance Flash based systems. After IBM has rolled out a family of NAND memory-based systems last month (see my previous post), there was an interesting launch of Pure Storage last week!

And two more things: in a reminiscence to IBM's DFSMS (data facility system managed storage) first introduced in 1989 (!!) Windows now implements a similar concept to enable intelligent and OS-based management of back end storage resources: Windows Storage Spaces! They even use some of the same terminology: Storage Pools as an example!

Lastly, around the topics of backup and archiving (which get confused all the time), I found this short ESG video blog to be very enlightening and easy to remember: backup is about RECOVERY, archive is about DISCOVERY...see for yourself here!
Many clients have moved to disk-based backup solutions using deduplication technologies to optimize/minimize the use of rotating disk for backup data. Deduplication uses hash-based signatures to mark files or segments of data as unique and there is often discussions about the likelihood of so-called hash-collisions - thus two different segments/files resulting in the same signature value. Read this article here about the underlying math and likelihood of such collisions!

Off-topic: we have not seen the sun here for weeks, almost months, struggling thru the worst spring on record for many parts of central Europe, including Switzerland. So it feels good to be reminded that my country was recently voted as the #1 place to be born.

May

Very busy times - as always in April/May there is lots of activities, product announcements, events and customer projects, so I did not pay too much attention on what's going on in the industry for the past couple weeks...
One of the major events was SNW Spring which took place in Orlando FL, you can find a nice write-up of the major topics and trends from that event here.

As you might guess, SSD was a hot topic at that conference! And actually, one of the recent key IBM announcements was the first IBM FlashSystem and related to that, a massive investment in flash technology for the next couple years. FlashSystem is a "blue-washed" version of the TMS RamSan systems which IBM acquired in 2012.
But other than using SSD drives in a "legacy" disk array, FlashSystems are designed around the very characteristics of Flash Storage and thus can take advantage of the very low latency (100 microseconds!) and high performance that these storage building blocks offer!

As we all know, flash technology and SSDs are increasingly used in huge volumes for tablets and smart phones. And that market transition away from desktop and laptop PCs is felt in the HDD industry: Seagate has to deal with a significant drop in the amount of HDDs sold: "Hard drive-maker Seagate's third quarter profits slipped by a whopping 64 per cent year-on-year as the company's bottom line was hit by users deserting desktops for tablets."
However, HDD prices still seem to be declining as reported here, albeit at a much slower rate and partly due to the increased pricing levels we had in early 2012 after the Thailand flooding.

Now, if you want to go really back in time and track HDD pricing in the past decades, here's a great overview: Did you know that only 15 years ago - in spring 1998 - a 6 GB HDD sold for about $330?

And -while remaining in the past- StorageNewsletter has compiled an amazing list of web resources around the "History of the Storage Industry", this requires some time to enjoy, so store that link for a rainy weekend (which we will have again very soon for sure!).
Finally, and concluding my trip to the past for today, here's another view at the past 20 years of our industry: Did you know that "IBM Adstar" was the worlds biggest storage company in terms of revenues in 1992?