March 2016

It's been a great and long journey - but it comes to an end!
I had started that blog in mid 2007 - which seems ages ago, but only yesterday at the same time.
One of my topics back then was "the comeback of solid-state-storage" - and I'm kind of proud to realize, that I was not too far off in summer 2007, considering all the buzz around flash today and the fact, that IDC just declared that "..the adoption of flash was remarkably quick during the year, with flash systems, both HFA and AFA, overtaking HDD-only systems and representing 51% of total value shipped in 2015 versus 40% in 2014"!

Now, LinkedIn was initially launched in 2003,  Twitter started off in 2006 and these two are the main reason why I feel that investing time to write up my blog does no longer make sense.
And make no mistake: the area of storage and networking is more exciting than ever and our industry is changing by the day, so there would be no lack of news to report!
But typically, by the time I get to summarize my thoughts here, these news nowadays are common knowledge and almost outdated.

So if you are interested to follow my view and ideas on storage and the technologies related to it, please look me up in LinkedIn or Twitter ( @MattWernerCH )

Best regards
Matthias Werner

September

A hot summer season comes to an end - with temperatures in Zurich below 10C for the first time since probably May.

Only a few quick updates today, mostly around hyper-converged infrastructures and Software-defined datacenter. Two trends that obviously go hand in hand. Plus some hot Flash-memory news at the end.

That very topic is discussed in this great overview by George Crump of Storage Switzerland: "A core component of a HCA is SDS. Most HCA solutions are scale-out SDS solutions that run on each node in a hypervisor cluster. They then aggregate the storage in each node to create a shared storage pool accessible by all the virtual machines in the cluster. The value of HCA is that it eliminates a specialized storage network and significantly reduces the cost of storage, since a dedicated shared storage system is not needed. Like any other SDS solution, these HCA solutions then provide most of the necessary storage services like snapshots and cloning but many are missing key services like data protection and replication. For small organizations, a HCA may be all the business needs."

Read the details here!
And for short video to put things into perspective and make sure hype does not go thru the roof, watch this Toigo-clip here!

SDDC also was one of the major topics during the recent VMworld conference which took place in San Francisco: actually VMware renamed their previously discussed EVO:Rack product into EVO SDDC - to my knowledge the first real product being released under that name.
The related marketing lingo sounds as follows: "With our vision for SDDC, VMware has led the way to the next-generation data center architecture. We believe the SDDC is the best architecture for the cloud. By adopting a software-driven, automated approach to data center design and management, organizations gain the IT efficiency, agility and control they need to compete and innovate while achieving tremendous savings — IT organizations can slash CAPEX by as much as 49 percent. (1)
The value of an SDDC-based private cloud has been well recognized by our customers who are asking us for way to accelerate their journey. With EVO SDDC, we are taking a major step forward in providing a turnkey solution that will help our customers to stand up a complete SDDC in a matter of hours and more efficiently manage it at data center scale."

Lastly: the race and journey to "storage after spinning rust" is in full swing. The Flash Memory Summit -which takes place in August- currently is the storage event to be and got a lot of attention from the industry and customers.

As you can read in this press release from HGST, much of the R&D investments go into research focusing on non-volatile memory ""post NAND-flash: "In-memory computing is one of today's hottest data center trends. Gartner Group projects that software revenue alone for this market will exceed US $9B by the end of 2018³. In-memory computing enables organizations to gain business value from real-time insights by offering faster performance and greater scalability than legacy architectures."

And to conclude today's news on a local note: IBM Zurich Research made an important step to eliminate "drift" in PCM (phase change memory) devices: "Phase change memory (PCM) devices have been investigated since the early 1970s and in the past 12 months IBM scientists have made tremendous progress with this technology publishing a number of milestone papers which demonstrate multiple bits per cell, making the technology extremely competitive with Flash.
But PCM doesn’t come without some drawbacks —  a primary culprit being resistance drift. Drift is the change in resistance of the stored levels over time. Essentially the data moves causing your text document or your photo to eventually become corrupted and unusable — a very bad characteristic for a storage technology."

VMware EVO SDDC (previously code named VMware EVO: RACK™), will be a fully automated software suite for delivering the software-defined data center as an integrated system. Enterprises and service providers will be able to use VMware EVO SDDC to deploy a software-defined data center at scale. With VMware EVO SDDC, IT organizations can meet key data center scale initiatives, ranging from application and infrastructure delivery automation to business mobility to high availability and resilient infrastructure, without compromising security, control or choice. - See more at: http://www.vmware.com/company/news/releases/vmw-newsfeed/VMware-Unveils-the-Easiest-Way-to-Deploy-and-Operate-the-Software-Defined-Data-Center-at-Scale/1984725#sf40638508

VMware EVO SDDC (previously code named VMware EVO: RACK™), will be a fully automated software suite for delivering the software-defined data center as an integrated system. Enterprises and service providers will be able to use VMware EVO SDDC to deploy a software-defined data center at scale. With VMware EVO SDDC, IT organizations can meet key data center scale initiatives, ranging from application and infrastructure delivery automation to business mobility to high availability and resilient infrastructure, without compromising security, control or choice. - See more at: http://www.vmware.com/company/news/releases/vmw-newsfeed/VMware-Unveils-the-Easiest-Way-to-Deploy-and-Operate-the-Software-Defined-Data-Center-at-Scale/1984725#sf40638508

VMware EVO SDDC (previously code named VMware EVO: RACK™), will be a fully automated software suite for delivering the software-defined data center as an integrated system. Enterprises and service providers will be able to use VMware EVO SDDC to deploy a software-defined data center at scale. With VMware EVO SDDC, IT organizations can meet key data center scale initiatives, ranging from application and infrastructure delivery automation to business mobility to high availability and resilient infrastructure, without compromising security, control or choice. - See more at: http://www.vmware.com/company/news/releases/vmw-newsfeed/VMware-Unveils-the-Easiest-Way-to-Deploy-and-Operate-the-Software-Defined-Data-Center-at-Scale/1984725#sf40638508

VMware EVO SDDC (previously code named VMware EVO: RACK™), will be a fully automated software suite for delivering the software-defined data center as an integrated system. Enterprises and service providers will be able to use VMware EVO SDDC to deploy a software-defined data center at scale. With VMware EVO SDDC, IT organizations can meet key data center scale initiatives, ranging from application and infrastructure delivery automation to business mobility to high availability and resilient infrastructure, without compromising security, control or choice. - See more at: http://www.vmware.com/company/news/releases/vmw-newsfeed/VMware-Unveils-the-Easiest-Way-to-Deploy-and-Operate-the-Software-Defined-Data-Center-at-Scale/1984725#sf40638508

VMware EVO SDDC (previously code named VMware EVO: RACK™), will be a fully automated software suite for delivering the software-defined data center as an integrated system. Enterprises and service providers will be able to use VMware EVO SDDC to deploy a software-defined data center at scale. With VMware EVO SDDC, IT organizations can meet key data center scale initiatives, ranging from application and infrastructure delivery automation to business mobility to high availability and resilient infrastructure, without compromising security, control or choice. - See more at: http://www.vmware.com/company/news/releases/vmw-newsfeed/VMware-Unveils-the-Easiest-Way-to-Deploy-and-Operate-the-Software-Defined-Data-Center-at-Scale/1984725#sf40638508

July

The first heat wave is on us this week with temperature forecasts of up to 37C in Zurich - and with it starts Summer vacation time! So if you are headed for the beach or the mountains: Enjoy and don't worry about business during these lazy days of Summer!

If you, like myself, enjoy reading, I can recommend "No Ordinary Disruption" by Dobbs, Manyika, and Woetzel - a team of McKinsey researchers. It summarizes and explains the four major disruptive trends for the early 21st century which are:

1. Urbanization
2. Accelerating Technology Change
3. An Aging World (changing demographics)
4. Global Connections
   

As you can guess, at least two of them have a very close relationship to our industry (Technology Change and Global Connections) and the authors provide a clear and sometimes scary view of how these disruptions will change our world in the years to come.

Couple news around storage base technologies - HDD and SDD:

IDC just released the Q1 2015 worldwide storage revenue numbers. As every quarter since Carter was president of the U.S., the capacity sold grew double digit: "Total capacity shipments were up 41.1% year over year to 28.3 exabytes during the quarter."
Somewhat hidden in the charts is the fact that "ODM" (which IDC defines as ".. storage systems sales by original design manufacturers (ODM's) selling directly to hyperscale datacenter customers) accounted for 12.6% of global spending during the quarter - an increase of 23% compared to Q1 2014. Which means in my own terms that "the cloud" is the 3rd biggest storage vendor in Q1 2015! Agree?

Capacity-wise, the first 10TB disk drives are on the horizon: HGST announced a helium-based SMR drive (shingled magnetic recording) to be available later this year. As you may know, SMR drives lend themselves to "write-once type" of usage patterns. Read the above article for the detailed technical reasons for that behavior!

"SSDs are expected to eventually dominate HDDs in laptops and desktops, but that isn't expected to happen for years. At the end of last year, SSDs were only in about 15% of new notebooks.
And, prices for SSDs are many times higher than that of HDDs.
For example, a data center-class HDD with 6TB of capacity sells for $185 today and will drop to about $165 by the end of the year -- about 3 cents per gigabyte, according to market research firm Gartner. A 4TB HDD for a laptop sells for $95 to computer manufacturers or about 2 cents per gigabyte"
Read this great summary (Computerworld) on the race for capacity and price between the various HDD and SSD technologies!

Lastly, switching to the topic of hyperconverged: Nutanix has made some waves about an upcoming new solution: "Nutanix is working on a scale-out file server and thinks it will set the cat among the pigeons in array-land, especially NetApp's filer business."
They are positioning themselves for an upcoming IPO as you can see from these comments in the finance industry, so obviously try to dress up for Wall St. these days!

My company Avnet, by the way, is greatly positioned to help our clients and partners to grow into the era of "Converged" or "Hyperconverged" and take advantage of the cost, ease of use and flexibility benefits of these solutions. Watch this short video here to see what Avnet can do for you!

June

Errandum - talking about the perceived demise of Fibre Channel in my last post: meanwhile CISCO just quietly announced a new, compact 96-port 16 Gbps SAN switch , the MDS 9396S (in CISCO speak the "S" stands for 16 Gbps). Base configuration starts at 48 ports with SW/license upgrades in 12-port increments up to 96 ports.

Read the details here and here and find the product home here!

At the other end of the San Jose networking-neighborhood, at "130 Holger Way" (where Brocade is located), an almost flat SAN revenue was reported for their Q2 of FY2015: "SAN product revenue was $314 million, down 2% year over year. The decline was primarily the result of softer storage demand and operational issues at certain OEM partners.".

May II

Today, I have a bunch of networking news for you and try to give you some insight on where the industry is moving in terms of SDN (software defined networking).

But make no mistake: while marketing slides and vendor presentations suggest that everything "SD" is magic and easy, this is certainly not the case! "SD" at its best may enhance flexibility (customer choice of common hardware building blocks) and provide lower TCO, but the underlying mechanics and infrastructure still needs to provide the functions and performance levels of today's specialized products! So -as always- there's no free lunch here!
Point in case: read this article about the claimed TCO's for the CISCO ACI solution!

Good old Ethernet -as a protocol- is currently undergoing a major boost in terms of standardized speeds and -as a result- increased number of use cases. One of them is the new segment of "Ethernet disk drives". A very promising way of providing access to HDDs directly attached to standard Ethernet (thus no SATA, SAS, ...) as "The Register" writes in this article about a recent Seagate announcement here: "Kinetic is the object storage platform Seagate has built to make it possible to so useful work with its Ethernet-equipped disk drives. Seagate's ambition is to cut arrays out of the loop, allowing software to talk directly to disks. By cutting arrays and file systems out of the loop, Seagate reckons it can save users some cash and also speed things up."
To accommodate the increasing performance requirements of today's networks, the Ethernet Alliance has recently issued an updated roadmap including native 25 Gb/s which is supposed to be productized as early as 2015!
"The Ethernet Alliance recently released its 2015 roadmap, which outlines Ethernet’s response to the ongoing desire for more bandwidth in data centers by adding new speeds for both in-rack and backbone connections. The roadmap calls for some eye-popping speeds in the future, but also charts out a plan for the low end of the market, representing an unprecedented level of activity for Ethernet." 
Read the complete article here!

With my longterm Fibre Channel background, I was wondering of course where this leaves the storage protocol which is still dominating most of the datacenter connectivity between compute and storage: For a number of agreeable reasons, as outlined in this great article here, you may argue that FC has a somehow limited future: "Pundits have long predicted the demise of Fibre Channel, but this time it's real. In fact, the decline has already begun and now that RoCE is routable, this decline will accelerate. The last years Dell’ Oro SAN report highlighted a 3% year over year decline in Fibre Channel revenues, citing the need for “distributed scale-out and object-based storage” and a preference for "Ethernet speeds and converged infrastructure.""
Would you agree?

May I

It's May - which means that global IT flocks to Vegas for their spring pow-wow's. Starting this week is EMC with IBM next week and others to follow.
So for today, let me focus on the most recent EMC announcements and give you a couple hints on where to find the details.

First, read an overview and summary of the Day 1 announcements here and here!
One of the more exciting - but less visible news seems to be the vVNX (read: virtual VNX or software-only entry storage) offer which shows up in the EMC store - check it out here!

A slightly older post by EMC president Jeremy Burton from two weeks ago also helps to position and understand the strategy behind these announcements as well as his take on how and when last year's DSSD acquisition ("..an insane server-attached flash thing..") will start to be visible in products.

Big news also on the VCE side of the EMC federation: they use the EMCworld platform to make noise about the VxRack (based on VMware EVO:Rack) solution to be shipped around Q3 this year.
Read the overview here and some insights by VCE CTO Trey Layton here.
Lastly, for an understanding what's under the hood in terms of VMware technology, you'll have to scroll back to this VMworld blog entry from last year by VMware Fellow Raj Yavatkar here.

Now wait, before we all get too excited about this, let's put this into perspective: "...Thus hyper-convergence is a special class of server SANs where VM workloads run alongside the storage workloads. It was conceived of to be cheaper, denser and more appealing than legacy convergence. Data centre convergence is a special class of hyper-convergence."
And lastly, there is (always) the issue of pricing, as is outlined here.

April

Let's start with base technology today: we are slowly approaching the 10TB capacity point for 3.5" HDDs.
After Western Digital / HGST (which has been selling a helium-based 8Tb drive for a couple months already), Seagate has announced a 8TB drive as well: Not using helium to allow more platters inside the 3.5" form factor - but obviously basing their product on SMR (shingled magnetic recording) to achieve the required density.
This also explains the specific use case outlined in the announcement, described as "backup and objects": SMR disks use track overlay to reduce track width and thus have a limited capability to perform write updates to existing data / tracks. This is explained very well in this Wikipedia article here. Also refer to my previous blog-entry from November 2013!

There was other breaking news regarding storage capacity last week: The teams of IBM Research Zurich and Fujifilm have demonstrated an areal recording density of 123 billion bits of uncompressed data per square inch on low cost, particulate magnetic tape, a breakthrough which represents the equivalent of a 220 terabyte tape cartridge that could fit in the palm of your hand ! Read the details here.

Moving up the solution stack, let's talk about " SAN" and it's younger sibling "VSAN": while SAN's have been around for some twenty years now, the current trend has been reversed and storage seems to move back into the server as a "virtualized" resource to be shared by multiple instances of the virtualized servers (VMware, Hyper-V, ....).
Here's a look back at SAN's and a thorough analysis of the capabilities (and limitations!) of VSAN!
VSAN is also one of the building blocks of VMware EVO:Rail and with the recent wave of EVO:Rail appliances hitting the market (among them the EMC VSPEX Blue) I found the article here to be a great summary and overview of this architecture/technology.

"Converged System circa 1977"

Which brings us to the "converged topic" and related material which is plentiful these days! Let's first look at the Q4 2014 market numbers as presented by IDC. The surprising #2 vendor for EMEA for the integrated infrastructure (with a triple-digit growth compared to Q4 2013!) is HP:

• The EMEA "Integrated Infrastructure and Platform Revenue" report for all of 2014 can be found here.
• The worldwide "Integrated Infrastructure and Platform Revenue" report for Q4 2014 can be found here.

As many of you certainly already figured out, "converged" is nothing new - it's rather the inversion of what started in the late 80ies and early 90ies as "client-server" architecture. Read this amusing story here!

March

Spring greets us with the latest IDC numbers about Q4 storage market details. I have compiled a couple of the reports below:

The one really stunning fact in the IDC worldwide external storage report (besides the not-so unexpected drop of 24% for IBM) is the 40% increase of revenue for ODM direct! You won't find ODM as a company - IDC adds up sales to all types of "managed service providers" under that category: overall, external storage shipments to ODM's account for 12.8% of the worldwide revenue in Q4! See details here.
And for EMEA: IDC EMEA external storage market report.


IDC worldwide PBBA (if you don't remember PBBA: purpose built backup appliance) report. Not much of a surprise there, EMC DataDomain has a very comfortable lead with over 60% share in that market - which is not really growing in terms of revenue as you can read in the report.
Where is this massive amount of storage going? Read here about the recent price offensive of Google for low-cost storage!

And don't forget to forward the clock on Sunday morning-if you live in Europe that is!

February

VSPEX Blue is the one hot topic today and is EMC's entry into the HCIA (hyper-converged infrastructure appliance) market! Based on VMware EVO:Rail and VSAN and preconfigured with RecoverPoint and EMC CloudArray software (among others).

Please read the official press release here and read Avnet's release here!

You may be slightly confused (as I still am) about the positioning of EVO:Rail VSAN and EMC ScaleIO from EMC which seem to address a very similar market and problem. I found  that article from "The Register" to be very helpful in understanding where which makes sense and what the similarities are.

Add Flash storage to the equation and read how PureStorage sees EVO.Rail and Flash and how they may or may not be combined in the future in AFHCIA's (all-flash hyper-converged infrastructure appliances).

January

Happy New Year!

I have summarized a couple of 2015 predictions, wish-lists and trends at the beginning of the year for you below. Of course they represent my personal beliefs about where the IT and Storage industry will be heading in the next 12 months - so feel free to disagree!

First let me revisit the state of the art in HDD capacity as of January 2015, here's a great writeup of what is available today: As you can see, we are rapidly approaching the 10 TB capacity limit - but data access performance remains an ever bigger problem with those huge disk drives. Related to that, RAID protection becomes less practical with every TB we add for single drives, so the industry is looking into new ways to protect data as you can read here!
And for the SSD segment of enterprise storage, read the 2015 wish list for AFA's (all-flash arrays).


Second, here's a number of analyst predictions for top 2015 storage trends and market developments:
Hot storage technologies for 2015 (most notably Cloud-based disaster recovery or DRaaS among them) and top storage start-ups (many of them supporting flavors of SSDs to maximize performance)  to watch this year!
And lastly from the NetApp CTO and industry veteran Jay Kidd, here's his personal view on 2015 trends.

Third: meet "Docker", the next big thing after hypervisors! Or as is explained in this paragraph on docker.com: "Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications". It may be a smart thing to start to learn about Docker early in 2015 - so I have listed the link to a four part summary/basic overview from "The Register" here. Enjoy!

December

It has been a busy time for me and while I have continuously followed what's going on in the IT market, I did not have time to sit down and digest. So before 2014 is over, let's have a look at some recent news and how they fit into current trends:
If I had to summarize what I see going on right now is three major areas of excitement and start-up-activity:

• Converged and Hyper-Converged Systems, goes hand in hand with the "software defined datacenter" topics and the Server-side SAN solutions.
• Flash Storage and the question how a future "memory" or "storage" hierarchy will look like?
• Cloud and all kinds of "managed services". Combined with the question which parts of IT will be on-premises - and what will be off-premises in a future scenario?

Along these lines, here's a writeup of recent news: The article here is a great analysis by industry veteran Jon Toigo about the new storage hype around server-SANs and the proclaimed death of the storage array as we know it. He states that "...there's no such thing as a true storage network, at least not one that fits the ISO model of networks. We have storage fabrics with switched DAS (SANs), and we have DAS with thin file server appliances (NAS). So, the idea that virtual servers won't work with anything but server-side direct attachment is silly."

It's impossible to understand server-SANs without knowing some details about VMware VVOLs (and here), VSANs (see also here), and most recently EVO:Rail.  Check the respective links for an explanation of each!
Please see my blogpost from Sept. 2013 as well, plus a great comparison of Hyper-Converged solutions here!

So while VSANs let's you control direct attached server storage from VMware, VVOLs provides a new construct (beyond NAS and LUNs) to manage external/SAN storage from VMware. And EVO.Rail (based on VSAN technology) is a complete software bundle from VMware which enables vendors (Dell, HP, and others have announced intents) to build their own "hyper-converged" appliances.

Reminds me of good old days of IBM mainframes where the "IO subsystem" was defined and under control of the base operating system. VMware obviously wants to gain control over the storage side of the datacenter and -in my opinion- they have a fair chance of winning that bet.

On to Flash: Combining the converged and flash hype will result in solutions like the one here: "Nutanix introduced a new all-flash storage appliance and a new feature that enables continuous availability across data centers."
And last week from Flash-leader PureStorage:  "Pure and Cisco have dreamed up FlashStack converged infrastructure (CI). This is a combination of combine Pure’s 400 Series arrays, Cisco UCS Blade Servers, Nexus switches, VMware vSphere 5 and Horizon 6".
However, this is mostly an unilateral attempt from Pure to hook up with CISCO - you will not find that specific solution on the CISCO CVD (CISCO validated design) site. No surprise; as they already have Invicta (to be integrated into UCS) and have created solutions with Nimble as well.

Closing todays comments with very recent news on AWS (Amazon web services):
"Amazon added storage features at its AWS re:Invent conference in November. They include support for larger and faster volumes on its Elastic Block Store (EBS). The enhancements center on solid-state drive (SSD)-backed EBS, which was launched back in June in all AWS regions. Amazon calls SSD EBS the fastest-adopted service in its Amazon Web Services portfolio."

What if - someday we may start over with a very disruptive and new way of computing? While dating back to Summer, the vision of "The Machine" sounds somewhat appealing to me!

September

Networking news this month: Will 25 Gbps Ethernet be the next speed of choice? Read here why there's a couple good reasons to think so: "The next generation of Intel "Grantley" CPUs are due to ship in the third quarter, and servers with these processors will be able to forward data faster than 10 Gbit/s. In the high-density server farms of cloud providers like Google and Microsoft, using multiple 10 GbE would require twice as many Ethernet switches with their associated space, power, and cooling costs."

And on the Fibre Channel side of the market, CISCO quietly (and maybe reluctantly?) roles out a bunch of 16 Gbps products:
A new low-cost 16 Gbps switch (MDS9148S) and a smaller model of the MDS 9710 director (MDS 9106), a 192-port director in a 9U form factor!

August

Lazy summer days - mostly wet and rainy this year. So lots of time to read blogs and follow the IT News-tickers!

Here's what I found to be interesting news in the last couple weeks - sorted by "level of excitement".
First sighting of phase-change memory (PCM) for commercial use in SSD's: at the 2014 "Flash Summit Conference" HGST (subsidiary of Western Digital) did showcase a blazingly fast SSD device not based on Flash memory but PCM, as Computerworld reports here:
" The device, which can be plugged into a server's PCIe slot like any SSD, isn't a new storage product but a platform for demonstrating a low-latency interface that the company developed with future solid-state media in mind. It implemented the experimental communications protocol in a Linux driver on the server and in the SSD's embedded software."
"Where NAND Flash takes about 70 microseconds to respond to a request for data, PCM can do so in about 1 microsecond. However, it will take two to three years of hardware development before PCM gets dense and cheap enough to compete head-on with flash"

Also read the stories on that same subject in "The Register" and the related HGST Press Release.

As noted above, new low-latency technology like PCM will require new architectures and will have to reside "right next to the CPU" to actually take advantage of the available speed!
To that end, HGST has developed a new low-latency interface dubbed "DC express". 
Watch this excellent 15' video clip here!

So you should really think of PCM as a lower cost DRAM rather than a high performance Flash Array. Will be interesting how this will affect "storage tiers"!?

Now, don't count Flash SSD's out yet: As reported here, there is a whole family of bigger (up to 1.6TB) and faster (12 Gbit/s SAS) SSD's entering the market these days!

July

It's about time to resume blogging and edit an update to my Blog. I have been "out of IT" for a few month - between Jobs - and now back to the grind.

One of my off-IT activities: Travel Maui

Good thing is that I realized during my time off: there's plenty of great things to do and think outside of IT; sad thing is, I also realized that only in IT, I have enough marketable skills to be able to pay my bills.
So back to IT business...
Topic today is pretty straightforward. Cloud and Flash!
I think those are pretty much the buzzwords that, as someone looking into our industry in summer 2014 would realize, are discussed most frequently.

Now looking at these trends and the technologies cooking in the labs, I've come to the conclusion that the two can be combined in what I believe to be a true statement:
"By 2020, all data will be either stored on Flash or in the Cloud"
There's a couple reasons why I think this statement may become true, listed below are a couple market and technology trends from the last couple weeks which seem to point into that direction.
But the very basic physical reason for it is "latency": Response time for any IO from the cloud is always measured in milliseconds (at least - try a PING if you don't agree) while response time for any IO from a Flash System is always measured in microseconds. Means that Flash will always sit either in the server or right next to it (next like: a couple meters away)!

Let's start with Flash: See these two great articles about function and design of all-flash arrays  (by George Crump of Storage Switzerland) vs. advantages of hybrid storage arrays (by HDS CTO Hu Yoshida). And then add this news from the Samsung lab (Samsung being the #1 provider of NAND Flash) about first shipments of 3D Flash.

On to the Cloud: As the Register article here states: "CIOs will likely increasingly say: dump the data center hardware ship and ship the data up into the cloud."
A couple vendors actually announced functions and plans to that end in the past couple weeks: Netapp ties in with the Microsoft Azure cloud (more details here);
while mighty EMC announced (among a flood of other solutions, plans, products, and the intend to acquire Twinstrata) the availability of a cloud gateway option for the new VMAX3.

As discussed above, cloud data storage will most likely not be based on relatively expensive and fast Flash technology for the years to come. The most obvious candidate for efficient, cheap and reliable cloud storage is rotating disks - HDD. And a second, surprising -but also likely candidate- is good old Tape!
In case you missed the May announcement about IBM's 154TB-Tape demo: Listen to the details here!

April

Well, some of you know that I'm on a two month sabbatical now - away from my previous job at IBM.
So I have not been following the storage market very closely these days - but have focused on other (more?) important things in life - like mountain biking and enjoying hikes with my family.

The great thing is that we had a beautiful spring this year in Zurich with lots of sunshine and warm weather, as you can see in the weather report for March here!

Still, there was a couple things that caught my attention and they are around SDN (software defined networking): While I was doing online research and was trying to figure out, where the industry is going in relation to this topic, I found a great series of blog posts and articles over at the InformationWeek site (thanks folks!).

I have summarized these for you:
First, start here with an industry overview and download the SDN market report (login required).
Then, you can read a detailed analysis by vendor - what their strategies and solutions are:
Part 2: CISCO, Arista and HP
 Details for CISCO ACI (application centric infrastructure), part 1 and part 2.
Part 3: Juniper, Brocade, Dell
Part 4: VMware, Avaya

March

Two major topics today: Object storage and cloud. And YES, I believe the two are in fact very closely related: Objects (defined as unstructured and permanent, unchanged data) seem to be by far the best fit for data to be stored in the cloud!
To get us started - and if your are like myself still trying to come to grips with the object Hype - here's a great Object Storage 101 for your reference (thanks George Crump!)

And Chris Mellor -over at the Register- points out that object storage is ".. headed towards mainstream status, with ViPR, Black Pearl and EVault leading the way. Others, though, held back. As of the year end, we saw IBM doing nothing much with objects, NetApp apparently standing still with its StorageGRID, and Dell exiting the field." Read the details here!

Now, expanding the focus and looking beyond objects and storage only, using cloud services may not be the nirvana many CxO's are dreaming about: Forbes recently had a great article about this and made the point very well by listing the following checkmark items for cloud management:

This means that at any moment a CIO must be able to answer the following questions:

1. Are all the applications and services in our control running properly?
2. Are all the applications and services running in the cloud or as SaaS running properly?
3. Is there a network problem in the network under our control?
4. Is there a network problem in the public Internet?
5. Is there a network problem in the network of the cloud or SaaS provider?

I like numbers 2. and 4.!

Still, cloud becomes more popular and less expensive by the week - as the most recent price war between the cloud "behemoths" Amazon and Microsoft shows:

 



February

Two articles on the long-running competition between FC and FCoE caught my attention this past weeks:
The major argument for high-speed 16 Gbps native FC connections these days are high-performance (and more importantly: low-latency) SSD and Flash storage systems: Evaluator Group recently did a benchmark to compare the two technologies with a series of state-of-the-art SSD Systems and here's what they found:
"The resulting validation found the FC environment provided 2 to 10 times faster responses as workloads surpassed 80% SAN utilization and 20-30% less CPU utilization than FCoE."
for the complete Report, please refer to the Website here (registration required).

At the same time, the adoption of 10 GbE is obviously picking up speed (as reported here), so -as so many times before in our industry- the lower cost datacenter connectivity (10 GbE) may be just "good enough" in the long run!

January 2014

Welcome to 2014 (or MMXIV)!

I'll spare you another round of forecasts about what might and might not happen in the storage industry in 2014, there have been plenty of those lists published in the past couple weeks!
Instead, let me summarize a couple noteworthy news for you - which of course is also a way to hint at what -I personally believe- will be important trends in 2014!
One battle certainly to continue is the battle over the control over storage or rather the "location of the intelligence" in storage: remember 10 years ago: common wisdom was that most of the intelligence would be located in the SAN and the back-end storage would be mere commodity (EMC Invista!). Well, it turned out that customers preferred to invest in sophisticated storage controllers while the network became a commodity...
Fast forward to 2013: There is a lot of hype around SDS (Software Defined Storage), which would relegate the "intelligence" to some "open source" storage controller running on a commodity server.
Hypervisor vendors like VMware and Microsoft on the other hand are integrating more of the storage controls into their hypervisors and hope to make storage intelligence obsolete. And obviously related to that is the discussion and battle about block or file access to storage.
The article here outlines reasons why VMware needs to step up efforts on NFS!
But watch out, on the block side of the protocol segment, the FCIA is finalizing 32Gbps Fibre Channel - or Gen6 Fibre Channel as they are going to call it.

According to "The Register", 2013 was the year "..when Google made Tape cool again"!
Now there is more exciting news to be expected in 2014: Read this summary from the Fujitsu "IT Global Summit" and the "Nanocubic coating" being introduced for magnetic tape!
So watch this space for announcements from IBM during 2014!

Talking about IBM, there's two recent news that clearly show the direction my company is taking: First, the beta availability of ICStore, a unique implementation of "cloud gateway" technology for our Storwize platform providing data protection " ..that will allow organizations to use multiple cloud storage services interchangeably, reducing dependence on any single cloud vendor and ensuring that data remains available even during service outages."

And second, the rollout of WaaS ("Watson as a Service") in late 2013:

“The next generation will look back and see 2013 as a year of monumental change,” said Stephen Gold, vice president of the Watson project at IBM.

“This is the start of a shift in the way people interact with computers.”

IBM is wielding Watson in a fight to control the world of cloud computing — huge collections of computer servers connected over the Internet — with other big technology companies like Amazon.com, Google and Microsoft. It is no coincidence that IBM discussed its Watson news the same week Amazon was hosting clients at a conference here to pitch its own computing cloud, called Amazon Web Services or A.W.S."

This is definitely a key element of IBM's plans to resume revenue growth in 2014!

And by the way: Probably the last acquisition IBM made in 2013 is immediately related to cloud services: Aspera, a company with ".. an Emmy-award-winning, high-speed bulk data transfer protocol."

So all the best to you for 2014 and let's hope that the European economy continues on the positive trend that we seem to have started in Q3 2013!

December

Hot off the press:
IDC's Q3  "Worldwide External Disk Storage Systems Factory Revenue" numbers.
"Despite the revenue contraction in the third quarter, we see strong demand for offerings targeted at heavily virtualized environments, such as integrated infrastructure," said Eric Sheppard, research director, IDC storage. "However, this demand has been offset by several factors: reduced spending from the U.S. government, increased use of storage efficiency technologies, increased investment in public cloud capacity, and general price pressures associated with increased competitive environments."
See the detailed report here!

November II

It has been quite a while since the last announcement of bigger HDDs: We have had 4 TB disk drives available for about 24 months now! In the past couple weeks however, there seems to be renewed activity in the segment of high-capacity HDDs with HGST (Hitachi Global Storage; subsidiary of Western Digital) starting sale of the much-anticipated 6 TB Helium-drive (add two more platters; same areal density!).

Western Digital (WD) adding to the buzz with their plans (no product yet!) to provide increased areal density using HAMR (heat assisted magnetic recording), a technology which is in development over at Seagate as well!

The third promising candidate in the race to squeeze more bits into a 3.5 inch real-estate is "shingled magnetic recording" dubbed as SMR, details are outlined here.
While this technology basically uses hardware that is available today, it kind of changes some of the very characteristics of HDDs: In-place updates become impractical to impossible with the shingled, intertwined tracks (by the way: we have used this approach in tape drives for many years!). Means that HDDs using that technology will be used preferably for data which is not updated, so archive-type of usages! That's the reason why vendors are actually looking into using LTFS (linear tape file system) to access and manage SMR type of disk drives! See details from SNIA and WD here!

I have not figured out yet which technologies can be combined and which ones don't go together - but to me it certainly looks like as if the next wave of capacity increases for HDD is upon us! The question remains if the costs will be at an acceptable level: with the consumer market (tablets, smartphones) using mostly flash storage, the volumes for these new technology HDDs will probably remain lower than what we have been used to in the age of desktops and laptops?

And finally, wrapping up my update in spinning rust, here's an interesting report from Backblaze (a backup service provider) on how long these small mechanical wonders are supposed to last:
"For disk drives, it may be that all of them will wear out before they are 10 years old. Or it may be that some of them last 20 or 30 years. If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.
The number that we will be able to compute soon, and the one that is more likely to be useful, is the median lifespan of a new drive. In other words, at what age have half of the drives failed? We are starting to get an idea what the answer will be."
For additional thoughts and details, please also refer to this report from Google on their observations around drive failure expectations.