Hot off the press:
IDC's Q3 "Worldwide External Disk Storage Systems Factory Revenue" numbers.
"Despite the revenue contraction in the third quarter, we see strong
demand for offerings targeted at heavily virtualized environments, such as
integrated infrastructure," said Eric Sheppard, research director, IDC
storage. "However, this demand has been offset by several factors: reduced
spending from the U.S. government, increased use of storage efficiency
technologies, increased investment in public cloud capacity, and general price
pressures associated with increased competitive environments."
See the detailed report here!
Friday, December 6, 2013
Tuesday, November 19, 2013
November II
It has been quite a while since the last announcement of bigger HDDs: We have had 4 TB disk drives available for about 24 months now! In the past couple weeks however, there seems to be renewed activity in the segment of high-capacity HDDs with HGST (Hitachi Global Storage; subsidiary of Western Digital) starting sale of the much-anticipated 6 TB Helium-drive (add two more platters; same areal density!).
Western Digital (WD) adding to the buzz with their plans (no product yet!) to provide increased areal density using HAMR (heat assisted magnetic recording), a technology which is in development over at Seagate as well!
The third promising candidate in the race to squeeze more bits into a 3.5 inch real-estate is "shingled magnetic recording" dubbed as SMR, details are outlined here.
While this technology basically uses hardware that is available today, it kind of changes some of the very characteristics of HDDs: In-place updates become impractical to impossible with the shingled, intertwined tracks (by the way: we have used this approach in tape drives for many years!). Means that HDDs using that technology will be used preferably for data which is not updated, so archive-type of usages! That's the reason why vendors are actually looking into using LTFS (linear tape file system) to access and manage SMR type of disk drives! See details from SNIA and WD here!
I have not figured out yet which technologies can be combined and which ones don't go together - but to me it certainly looks like as if the next wave of capacity increases for HDD is upon us! The question remains if the costs will be at an acceptable level: with the consumer market (tablets, smartphones) using mostly flash storage, the volumes for these new technology HDDs will probably remain lower than what we have been used to in the age of desktops and laptops?
And finally, wrapping up my update in spinning rust, here's an interesting report from Backblaze (a backup service provider) on how long these small mechanical wonders are supposed to last:
"For disk drives, it may be that all of them will wear out before they are 10 years old. Or it may be that some of them last 20 or 30 years. If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.
The number that we will be able to compute soon, and the one that is more likely to be useful, is the median lifespan of a new drive. In other words, at what age have half of the drives failed? We are starting to get an idea what the answer will be."
For additional thoughts and details, please also refer to this report from Google on their observations around drive failure expectations.
Western Digital (WD) adding to the buzz with their plans (no product yet!) to provide increased areal density using HAMR (heat assisted magnetic recording), a technology which is in development over at Seagate as well!
The third promising candidate in the race to squeeze more bits into a 3.5 inch real-estate is "shingled magnetic recording" dubbed as SMR, details are outlined here.
While this technology basically uses hardware that is available today, it kind of changes some of the very characteristics of HDDs: In-place updates become impractical to impossible with the shingled, intertwined tracks (by the way: we have used this approach in tape drives for many years!). Means that HDDs using that technology will be used preferably for data which is not updated, so archive-type of usages! That's the reason why vendors are actually looking into using LTFS (linear tape file system) to access and manage SMR type of disk drives! See details from SNIA and WD here!
I have not figured out yet which technologies can be combined and which ones don't go together - but to me it certainly looks like as if the next wave of capacity increases for HDD is upon us! The question remains if the costs will be at an acceptable level: with the consumer market (tablets, smartphones) using mostly flash storage, the volumes for these new technology HDDs will probably remain lower than what we have been used to in the age of desktops and laptops?
And finally, wrapping up my update in spinning rust, here's an interesting report from Backblaze (a backup service provider) on how long these small mechanical wonders are supposed to last:
"For disk drives, it may be that all of them will wear out before they are 10 years old. Or it may be that some of them last 20 or 30 years. If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.
The number that we will be able to compute soon, and the one that is more likely to be useful, is the median lifespan of a new drive. In other words, at what age have half of the drives failed? We are starting to get an idea what the answer will be."
For additional thoughts and details, please also refer to this report from Google on their observations around drive failure expectations.
Friday, October 25, 2013
November
Object Storage - not sure if it's only me but I seem to see a lot of buzz, hype and news around this topic in the past couple months!
So I've started to track some news sites and have compiled some material I want to share with you. To start off, here's a great and very brief summary on what Object Storage actually is and how it's different from file systems: "Instead of organizing files in a directory hierarchy, object storage systems store files in a flat organization of containers (called "buckets" in Amazon S3) and use unique IDs (called "keys" in S3) to retrieve them."
Read the details here. (thanks to my friend John Sing to provide that link!)
There is also some additional details and diagrams on the OpenStack site here.
Now, object storage will most likely be cloud-based storage and two companies have announced two very different products and approaches to actually store objects: In the case of Seagate (disk drives) they propose a new type of hard disk to directly connect "to the cloud":
"Seagate is building hard disk drives with a direct Ethernet interface and object-style API access for scalable object stores, a plan which - if it works - would destroy much of the existing, typical storage stack. Drives would become native key/value stores that manage their own space mapping with accessing applications simply dealing at the object level with gets and puts instead of using file abstractions."
In the case of SpectraLogic (tape libraries) they promote low-cost tape storage as the foundation for object clouds: "Spectra’s Deep Storage story opens with a new RESTful object storage API that Spectra’s dubbed DS3, for Deep Simple Storage Service. DS3 is an extension of the Amazon S3 API that’s becoming a standard for not just cloud storage services but also on-premise data center object storage systems. The object API allows organizations to take advantage of the low cost, and power consumption, of tape and still have their data available to applications written by mere mortals."
Eventually, they both make sense but I think adding a low-cost (and automatic) tape tier makes lots of sense for objects - which tend to be inactive of nature and clients will probably prefer lower costs over split-second retrieve times?!
So I've started to track some news sites and have compiled some material I want to share with you. To start off, here's a great and very brief summary on what Object Storage actually is and how it's different from file systems: "Instead of organizing files in a directory hierarchy, object storage systems store files in a flat organization of containers (called "buckets" in Amazon S3) and use unique IDs (called "keys" in S3) to retrieve them."
Read the details here. (thanks to my friend John Sing to provide that link!)
There is also some additional details and diagrams on the OpenStack site here.
"Seagate is building hard disk drives with a direct Ethernet interface and object-style API access for scalable object stores, a plan which - if it works - would destroy much of the existing, typical storage stack. Drives would become native key/value stores that manage their own space mapping with accessing applications simply dealing at the object level with gets and puts instead of using file abstractions."
In the case of SpectraLogic (tape libraries) they promote low-cost tape storage as the foundation for object clouds: "Spectra’s Deep Storage story opens with a new RESTful object storage API that Spectra’s dubbed DS3, for Deep Simple Storage Service. DS3 is an extension of the Amazon S3 API that’s becoming a standard for not just cloud storage services but also on-premise data center object storage systems. The object API allows organizations to take advantage of the low cost, and power consumption, of tape and still have their data available to applications written by mere mortals."
Eventually, they both make sense but I think adding a low-cost (and automatic) tape tier makes lots of sense for objects - which tend to be inactive of nature and clients will probably prefer lower costs over split-second retrieve times?!
Friday, September 20, 2013
September II
Lots of news these days, so time for an update and summary. As always in September, industry analysts did release their assessment of the 2013 Q2 storage market.
I have compiled the facts and comments for you - here as a general statement: "...Worldwide external disk storage systems factory revenues posted a year-over-year decline of -0.8%, totaling just over $5.9 billion, in the second quarter of 2013 (2Q13), according to the International Data Corporation (IDC) Worldwide Quarterly Disk Storage Systems Tracker. For the quarter, the total (internal plus external) disk storage systems market generated $7.7 billion in revenue, representing a -5.0% decline from the prior year's second quarter and a slight sequential decline compared to the first quarter of 2013. Total disk storage systems capacity shipped reached 8.2 exabytes, growing 21.5% year over year."
In the midterm, the Gartners, IDCs and the likes will have to figure out if they eventually need to include storage provisioned thru cloud services as well? As of today, I think the capacities sold and installed off-premises by companies like Amazon Webservices, Rackspace or Google are not accounted for...they don't procure systems from the legacy storage vendors in most cases, so don't show up in the statistics here!
And actually this links right into the next topic: an interesting podcast by Jon Toigo on cloud:
"...I did think that maybe one of the better models for cloud going forward -- a sustainable business model for cloud, would be cloud that is specialized in holding huge repositories of certain kinds of data. I asked experts about this. Jeff Jonas at IBM, I asked him, would it make sense for a cloud service provider to stand up big data so I don't have to buy the infrastructure myself?"
And in that same week a statement from NetApp on how they think their systems will interact and integrate with the cloud, I personally think they have a great story there: they have rolled out DOT 8.2 starting this year which enables clustered Data ONTAP and adding cloud gateway capability and a couple interfaces to public cloud services should not represent a major hurdle.
See the short 2' video clip here.
But ultimately, it will be our customers call to decide if and how they will entrust their data to any form of "cloud", see the five reasons for - and five reasons against doing it...great writeup by Trevor Pott: "The cloud will inevitably replace all other forms of IT? The cloud is a passing fad?"
And lastly in a surprise but important move, CISCO announced its intent to acquire Whiptail, a vendor of Flash Storage systems. While CISCO in their own press release emphasize the use of Whiptail technology inside their UCS servers, the rest of the industry sees this move mostly as a way of CISCO to expanding their reach into storage. Read this great analysis here.
I have compiled the facts and comments for you - here as a general statement: "...Worldwide external disk storage systems factory revenues posted a year-over-year decline of -0.8%, totaling just over $5.9 billion, in the second quarter of 2013 (2Q13), according to the International Data Corporation (IDC) Worldwide Quarterly Disk Storage Systems Tracker. For the quarter, the total (internal plus external) disk storage systems market generated $7.7 billion in revenue, representing a -5.0% decline from the prior year's second quarter and a slight sequential decline compared to the first quarter of 2013. Total disk storage systems capacity shipped reached 8.2 exabytes, growing 21.5% year over year."
- IDC Press Release and summary of report with market numbers here.
- Computerworld comment here. (by Lucas Mearian)
- The Register comment here. (by Chris Mellor)
- And some more detailed numbers for EMEA specifically here. (by our friends at StorageNewsletter)
In the midterm, the Gartners, IDCs and the likes will have to figure out if they eventually need to include storage provisioned thru cloud services as well? As of today, I think the capacities sold and installed off-premises by companies like Amazon Webservices, Rackspace or Google are not accounted for...they don't procure systems from the legacy storage vendors in most cases, so don't show up in the statistics here!
And actually this links right into the next topic: an interesting podcast by Jon Toigo on cloud:
"...I did think that maybe one of the better models for cloud going forward -- a sustainable business model for cloud, would be cloud that is specialized in holding huge repositories of certain kinds of data. I asked experts about this. Jeff Jonas at IBM, I asked him, would it make sense for a cloud service provider to stand up big data so I don't have to buy the infrastructure myself?"
And in that same week a statement from NetApp on how they think their systems will interact and integrate with the cloud, I personally think they have a great story there: they have rolled out DOT 8.2 starting this year which enables clustered Data ONTAP and adding cloud gateway capability and a couple interfaces to public cloud services should not represent a major hurdle.
See the short 2' video clip here.
But ultimately, it will be our customers call to decide if and how they will entrust their data to any form of "cloud", see the five reasons for - and five reasons against doing it...great writeup by Trevor Pott: "The cloud will inevitably replace all other forms of IT? The cloud is a passing fad?"
And lastly in a surprise but important move, CISCO announced its intent to acquire Whiptail, a vendor of Flash Storage systems. While CISCO in their own press release emphasize the use of Whiptail technology inside their UCS servers, the rest of the industry sees this move mostly as a way of CISCO to expanding their reach into storage. Read this great analysis here.
Tuesday, September 3, 2013
September
Today I'll continue with a couple flash topics from last month and provide some links and comments regarding the news and developments announced at the recent VMworld conference.
When the SSD hype started first around 2007/2008, there was a widespread belief that SSD would be a temporary and intermediate technology: The "write penalty" and very limited durability seemed to be obstacles to a successful use and implementation in enterprise storage systems. Now, the industry has fixed many of these issues, partly by using smart controller designs (wear-leveling) partly by improving SSD core technology (eMLC, TLC, 3D-Flash, ...)
At the same time, industrialization of more advanced "storage class memory" like PCM (phase change memory) and MRAM (magnetoresistive random access memory) has taken a backseat to the current hype around NAND flash. Read the complete analysis in CW here:
"It's going to be a long time until NAND flash runs out of steam," said Jim Handy, an analyst at Objective Analysis, during a presentation."
There's an interesting possible combination of technologies which so far (to my knowledge) has not been productized: Flash storage and deduplication: Greenbites proposes the use of Flash storage to achieve " ...deduplication tech that has near-zero latency and possibly offers the world's fastest in line deduplication." Seems to make a lot of sense: deduplication has an inherent requirement for very low latency read operations (to compare data patterns while deduplicating at ingest) and Flash is still a very expensive way to store data -so you want to avoid duplicate data. Combining the two seems kind of obvious!
And Violin, one of the most visible startups in the realm of Flash Systems, is getting ready to an IPO!
Looking at how detached from the real-world economy Wall St. recently seems to be, I'm honestly not sure that's a good thing - but some people will earn some serious money...
Unrelated to Flash, the article here caught my interest: "On Tuesday, Nasuni introduced a cloud-to-cloud mirroring option to give customers extra assurance that their data will be available in case of a service outage. The same day, cloud-to-cloud backup vendor Backupify added more choices for where users can have their data sent."
We all have heard about the outages of the likes of Amazon and Google, so backing up your data in multiple clouds seems to be the smart thing to do! Here's a nice way to visualize this (and I wonder how long it will take until we see cloud companies with meteorology-names like Cumulus and Nimbostratus...
On to VMworld which took place in San Francisco (unfortunately, I did not attend but supposedly 22'000 folks did!): while they had a zillion of press releases and announcements, many of the new functions and strategies revolve around two areas: The network and storage.
"VMware NSX is a software-defined network (SDN) that uses controllers and overlay networking. I'll examine just a few of the key aspects of the announcement and how they apply to your data center strategy." See additional background here.
Not to be confused with CISCO VSANs, the storage integration/tiering functions in VMware are now captured under that same terminology: " VMware launched the public beta of VSAN (virtual SAN), its software for pooling server-based flash among multiple physical servers. VSAN creates what is effectively a hybrid storage array using flash and hard disk drives on each of a cluster of servers."
At the same time, industrialization of more advanced "storage class memory" like PCM (phase change memory) and MRAM (magnetoresistive random access memory) has taken a backseat to the current hype around NAND flash. Read the complete analysis in CW here:
"It's going to be a long time until NAND flash runs out of steam," said Jim Handy, an analyst at Objective Analysis, during a presentation."
There's an interesting possible combination of technologies which so far (to my knowledge) has not been productized: Flash storage and deduplication: Greenbites proposes the use of Flash storage to achieve " ...deduplication tech that has near-zero latency and possibly offers the world's fastest in line deduplication." Seems to make a lot of sense: deduplication has an inherent requirement for very low latency read operations (to compare data patterns while deduplicating at ingest) and Flash is still a very expensive way to store data -so you want to avoid duplicate data. Combining the two seems kind of obvious!
And Violin, one of the most visible startups in the realm of Flash Systems, is getting ready to an IPO!
Looking at how detached from the real-world economy Wall St. recently seems to be, I'm honestly not sure that's a good thing - but some people will earn some serious money...
Unrelated to Flash, the article here caught my interest: "On Tuesday, Nasuni introduced a cloud-to-cloud mirroring option to give customers extra assurance that their data will be available in case of a service outage. The same day, cloud-to-cloud backup vendor Backupify added more choices for where users can have their data sent."
We all have heard about the outages of the likes of Amazon and Google, so backing up your data in multiple clouds seems to be the smart thing to do! Here's a nice way to visualize this (and I wonder how long it will take until we see cloud companies with meteorology-names like Cumulus and Nimbostratus...
On to VMworld which took place in San Francisco (unfortunately, I did not attend but supposedly 22'000 folks did!): while they had a zillion of press releases and announcements, many of the new functions and strategies revolve around two areas: The network and storage.
"VMware NSX is a software-defined network (SDN) that uses controllers and overlay networking. I'll examine just a few of the key aspects of the announcement and how they apply to your data center strategy." See additional background here.
Not to be confused with CISCO VSANs, the storage integration/tiering functions in VMware are now captured under that same terminology: " VMware launched the public beta of VSAN (virtual SAN), its software for pooling server-based flash among multiple physical servers. VSAN creates what is effectively a hybrid storage array using flash and hard disk drives on each of a cluster of servers."
Friday, August 16, 2013
August
304 - This was the magic number for July. 304 hours of sunshine in Zurich in July 2013...most likely won't be repeated for a couple years. And I was lucky enough to get three weeks of vacation during that time!
Now, back to business this week!
There was quite a bit of news coming from the FlashMemory Summit taking place in Santa Clara this week during the otherwise quite days of summer. My today's update will thus be somewhat SSD- and Flash-biased!
Faster, cheaper flash storage is hitting the enterprise market and Computerworld had a great overview on new products and technologies being introduced.
As it has been the case with every piece of IT technology in the past, consumer products lead the way and the volumes achieved there eventually help drive down costs and make components (Flash Memory in this case) viable for enterprise products. In the case of SSD, the Ultrabooks and Tablets are driving up volume shipments, as outlined in CW.
Amazingly enough, there's companies like Facebook who require "the worst flash possible...just make it dense and cheap." Write-once, read-never would be the spec for this type of Flash. I'd think that probably tape storage would make a lot more sense in such an environment?!
As a matter of fact, Flash Memory and SSDs have reached enough momentum in the market to stir some countermeasures and activities on the side of the HDD ("spinning rust") vendors: "The Storage Products Association (SPA) was formed as a mutual benefit nonprofit corporation in California to promote the use and understanding of rotating magnetic media hard drive (RHD) technologies as a critical component of the future digital storage ecosystem."
Companies behind SPA are (Aug. 2013): HGST, Seagate, Toshiba and WD.
On their website, they have compiled a very nice infographic with a couple amazing facts around disk storage!
There was quite a bit of news coming from the FlashMemory Summit taking place in Santa Clara this week during the otherwise quite days of summer. My today's update will thus be somewhat SSD- and Flash-biased!
Faster, cheaper flash storage is hitting the enterprise market and Computerworld had a great overview on new products and technologies being introduced.
As it has been the case with every piece of IT technology in the past, consumer products lead the way and the volumes achieved there eventually help drive down costs and make components (Flash Memory in this case) viable for enterprise products. In the case of SSD, the Ultrabooks and Tablets are driving up volume shipments, as outlined in CW.
Amazingly enough, there's companies like Facebook who require "the worst flash possible...just make it dense and cheap." Write-once, read-never would be the spec for this type of Flash. I'd think that probably tape storage would make a lot more sense in such an environment?!
As a matter of fact, Flash Memory and SSDs have reached enough momentum in the market to stir some countermeasures and activities on the side of the HDD ("spinning rust") vendors: "The Storage Products Association (SPA) was formed as a mutual benefit nonprofit corporation in California to promote the use and understanding of rotating magnetic media hard drive (RHD) technologies as a critical component of the future digital storage ecosystem."
Companies behind SPA are (Aug. 2013): HGST, Seagate, Toshiba and WD.
On their website, they have compiled a very nice infographic with a couple amazing facts around disk storage!
Friday, July 5, 2013
July
Recently, I had one of these "aha" moments, situations where things come together and you suddenly realize that a thought or idea you had -only as a faint impression- suddenly becomes real! Most often this happens to me when I read an article which exactly sums up -in an eloquent way-what I had only been guessing or feeling.
The "Register" article which had that effect on me last week is this one: "Amazon, currently estimated to have $2bn in storage sales with an expected 65 per cent growth in storage revenue in each of the next 2-3 years which will make them a top 3 player in the storage industry".
Top 3 player in the storage industry? For decades, the likes of EMC, IBM, HP, NetApp, HDS have been switching positions in the quarterly IDC and Gartner charts... now Amazon?
How come?
Well, here's why: "The move to cloud computing means fewer companies will buy huge numbers of servers and storage arrays for their own use. Over the next five years, Morgan Stanley’s expects that 3 percent to 17 percent of current spending could be sucked up by cloud-based IT service providers."
And this is how "cloud storage" gets to "top three": 17% of the storage market today actually would be the #2 spot (behind market leader EMC).
So customers only hold very limited amounts of active data in their "on-premise" storage farms (most likely on SSDs) and push/pull everything else (including archives and backups) to the cloud.
As we all know, cloud providers like Amazon are not known as buyers of huge amounts of storage systems. Like Google and Facebook and other "hyperscale data" companies, they build their own infrastructure, based on chips from Intel or AMD and disk drives directly from the vendors like WD and Seagate.
Now, preparing for this era and in order to allow enterprises to mitigate the dependence on one specific cloud technology or vendor, IBM is working on a project (this was actually demo'ed at the Edge 2013 conference in June) to provide redundancy by using an on-premise IBM Storwize V7000 system connected to multiple clouds (RAID'ed clouds if you will): Please read the research paper here!
On to the network: (after all "cloud" is enabled only by powerful networks!)
You may not have realized it, but Ethernet did celebrate its 40th birthday this past spring, here's an interview with Bob Metcalfe, the inventor of Ether-Net!
At the same time, 16 Gbps Fibre Channel is gaining momentum as is reported here. The fact that CISCO recently announced their own 16 Gbps SAN director definitely is an endorsement for the Fibre Channel market and provides clients a choice of CISCO or Brocade gear for their storage networks!
And lastly, as many of us will be enjoying vacations in the coming weeks, exploring new countries and making new experiences, here's some great thoughts related to "live-blogging": "Even for those who are always pulling out their phones to snap pictures or shoot videos, to text or tweet or tumble or otherwise share the moments of their being, the pleasure lies mainly in the recording, not in the record. The act of recording is itself a disposable experience. The tools for recording and sharing are disposable as well. They get old."
Have fun!
The "Register" article which had that effect on me last week is this one: "Amazon, currently estimated to have $2bn in storage sales with an expected 65 per cent growth in storage revenue in each of the next 2-3 years which will make them a top 3 player in the storage industry".
Top 3 player in the storage industry? For decades, the likes of EMC, IBM, HP, NetApp, HDS have been switching positions in the quarterly IDC and Gartner charts... now Amazon?
How come?
Well, here's why: "The move to cloud computing means fewer companies will buy huge numbers of servers and storage arrays for their own use. Over the next five years, Morgan Stanley’s expects that 3 percent to 17 percent of current spending could be sucked up by cloud-based IT service providers."
And this is how "cloud storage" gets to "top three": 17% of the storage market today actually would be the #2 spot (behind market leader EMC).
So customers only hold very limited amounts of active data in their "on-premise" storage farms (most likely on SSDs) and push/pull everything else (including archives and backups) to the cloud.
As we all know, cloud providers like Amazon are not known as buyers of huge amounts of storage systems. Like Google and Facebook and other "hyperscale data" companies, they build their own infrastructure, based on chips from Intel or AMD and disk drives directly from the vendors like WD and Seagate.
Now, preparing for this era and in order to allow enterprises to mitigate the dependence on one specific cloud technology or vendor, IBM is working on a project (this was actually demo'ed at the Edge 2013 conference in June) to provide redundancy by using an on-premise IBM Storwize V7000 system connected to multiple clouds (RAID'ed clouds if you will): Please read the research paper here!
On to the network: (after all "cloud" is enabled only by powerful networks!)
You may not have realized it, but Ethernet did celebrate its 40th birthday this past spring, here's an interview with Bob Metcalfe, the inventor of Ether-Net!
At the same time, 16 Gbps Fibre Channel is gaining momentum as is reported here. The fact that CISCO recently announced their own 16 Gbps SAN director definitely is an endorsement for the Fibre Channel market and provides clients a choice of CISCO or Brocade gear for their storage networks!
And lastly, as many of us will be enjoying vacations in the coming weeks, exploring new countries and making new experiences, here's some great thoughts related to "live-blogging": "Even for those who are always pulling out their phones to snap pictures or shoot videos, to text or tweet or tumble or otherwise share the moments of their being, the pleasure lies mainly in the recording, not in the record. The act of recording is itself a disposable experience. The tools for recording and sharing are disposable as well. They get old."
Have fun!
Monday, June 3, 2013
June
Hyperscale, Hyper-V, and OpenStack are my topics today! And of course SDE!
You have not heard about SDE?
Well, as a start, read Steve Duplessie's comments here!
Now, with that put into perspective, let's look at some recent exciting news in the realm of cloud storage: The OpenStack Open Source Cloud Mission:
"... to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable."
IBM is very actively participating and contributing to that initiative and if you can spend half an hour, listen to this presentation delivered at the recent OpenStack Summit conference!
Now, hyperscale storage is a slightly different beast and could probably be used best to describe the architectures and challenges that some of those companies face who are actually providing these cloudy storage services: "The term hyperscale storage is coming into vogue to describe systems capable of rapid, efficient expansion to handle massive quantities of data from Web-serving, database, data analysis,high-performance computing and other especially busy applications"
Please read the complete article in searchstorage here!
And obviously, part of hyperscale storage requirements will increasingly be addressed by high-performance Flash based systems. After IBM has rolled out a family of NAND memory-based systems last month (see my previous post), there was an interesting launch of Pure Storage last week!
And two more things: in a reminiscence to IBM's DFSMS (data facility system managed storage) first introduced in 1989 (!!) Windows now implements a similar concept to enable intelligent and OS-based management of back end storage resources: Windows Storage Spaces! They even use some of the same terminology: Storage Pools as an example!
Lastly, around the topics of backup and archiving (which get confused all the time), I found this short ESG video blog to be very enlightening and easy to remember: backup is about RECOVERY, archive is about DISCOVERY...see for yourself here!
Many clients have moved to disk-based backup solutions using deduplication technologies to optimize/minimize the use of rotating disk for backup data. Deduplication uses hash-based signatures to mark files or segments of data as unique and there is often discussions about the likelihood of so-called hash-collisions - thus two different segments/files resulting in the same signature value. Read this article here about the underlying math and likelihood of such collisions!
Off-topic: we have not seen the sun here for weeks, almost months, struggling thru the worst spring on record for many parts of central Europe, including Switzerland. So it feels good to be reminded that my country was recently voted as the #1 place to be born.
You have not heard about SDE?
Well, as a start, read Steve Duplessie's comments here!
Now, with that put into perspective, let's look at some recent exciting news in the realm of cloud storage: The OpenStack Open Source Cloud Mission:
"... to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable."
IBM is very actively participating and contributing to that initiative and if you can spend half an hour, listen to this presentation delivered at the recent OpenStack Summit conference!
Now, hyperscale storage is a slightly different beast and could probably be used best to describe the architectures and challenges that some of those companies face who are actually providing these cloudy storage services: "The term hyperscale storage is coming into vogue to describe systems capable of rapid, efficient expansion to handle massive quantities of data from Web-serving, database, data analysis,high-performance computing and other especially busy applications"
Please read the complete article in searchstorage here!
And obviously, part of hyperscale storage requirements will increasingly be addressed by high-performance Flash based systems. After IBM has rolled out a family of NAND memory-based systems last month (see my previous post), there was an interesting launch of Pure Storage last week!
And two more things: in a reminiscence to IBM's DFSMS (data facility system managed storage) first introduced in 1989 (!!) Windows now implements a similar concept to enable intelligent and OS-based management of back end storage resources: Windows Storage Spaces! They even use some of the same terminology: Storage Pools as an example!
Lastly, around the topics of backup and archiving (which get confused all the time), I found this short ESG video blog to be very enlightening and easy to remember: backup is about RECOVERY, archive is about DISCOVERY...see for yourself here!
Many clients have moved to disk-based backup solutions using deduplication technologies to optimize/minimize the use of rotating disk for backup data. Deduplication uses hash-based signatures to mark files or segments of data as unique and there is often discussions about the likelihood of so-called hash-collisions - thus two different segments/files resulting in the same signature value. Read this article here about the underlying math and likelihood of such collisions!
Off-topic: we have not seen the sun here for weeks, almost months, struggling thru the worst spring on record for many parts of central Europe, including Switzerland. So it feels good to be reminded that my country was recently voted as the #1 place to be born.
Wednesday, May 29, 2013
May
Very busy times - as always in April/May there is lots of activities, product announcements, events and customer projects, so I did not pay too much attention on what's going on in the industry for the past couple weeks...
One of the major events was SNW Spring which took place in Orlando FL, you can find a nice write-up of the major topics and trends from that event here.
As you might guess, SSD was a hot topic at that conference! And actually, one of the recent key IBM announcements was the first IBM FlashSystem and related to that, a massive investment in flash technology for the next couple years. FlashSystem is a "blue-washed" version of the TMS RamSan systems which IBM acquired in 2012.
But other than using SSD drives in a "legacy" disk array, FlashSystems are designed around the very characteristics of Flash Storage and thus can take advantage of the very low latency (100 microseconds!) and high performance that these storage building blocks offer!
As we all know, flash technology and SSDs are increasingly used in huge volumes for tablets and smart phones. And that market transition away from desktop and laptop PCs is felt in the HDD industry: Seagate has to deal with a significant drop in the amount of HDDs sold: "Hard drive-maker Seagate's third quarter profits slipped by a whopping 64 per cent year-on-year as the company's bottom line was hit by users deserting desktops for tablets."
However, HDD prices still seem to be declining as reported here, albeit at a much slower rate and partly due to the increased pricing levels we had in early 2012 after the Thailand flooding.
Now, if you want to go really back in time and track HDD pricing in the past decades, here's a great overview: Did you know that only 15 years ago - in spring 1998 - a 6 GB HDD sold for about $330?
And -while remaining in the past- StorageNewsletter has compiled an amazing list of web resources around the "History of the Storage Industry", this requires some time to enjoy, so store that link for a rainy weekend (which we will have again very soon for sure!).
Finally, and concluding my trip to the past for today, here's another view at the past 20 years of our industry: Did you know that "IBM Adstar" was the worlds biggest storage company in terms of revenues in 1992?
One of the major events was SNW Spring which took place in Orlando FL, you can find a nice write-up of the major topics and trends from that event here.
As you might guess, SSD was a hot topic at that conference! And actually, one of the recent key IBM announcements was the first IBM FlashSystem and related to that, a massive investment in flash technology for the next couple years. FlashSystem is a "blue-washed" version of the TMS RamSan systems which IBM acquired in 2012.
But other than using SSD drives in a "legacy" disk array, FlashSystems are designed around the very characteristics of Flash Storage and thus can take advantage of the very low latency (100 microseconds!) and high performance that these storage building blocks offer!
As we all know, flash technology and SSDs are increasingly used in huge volumes for tablets and smart phones. And that market transition away from desktop and laptop PCs is felt in the HDD industry: Seagate has to deal with a significant drop in the amount of HDDs sold: "Hard drive-maker Seagate's third quarter profits slipped by a whopping 64 per cent year-on-year as the company's bottom line was hit by users deserting desktops for tablets."
However, HDD prices still seem to be declining as reported here, albeit at a much slower rate and partly due to the increased pricing levels we had in early 2012 after the Thailand flooding.
Now, if you want to go really back in time and track HDD pricing in the past decades, here's a great overview: Did you know that only 15 years ago - in spring 1998 - a 6 GB HDD sold for about $330?
And -while remaining in the past- StorageNewsletter has compiled an amazing list of web resources around the "History of the Storage Industry", this requires some time to enjoy, so store that link for a rainy weekend (which we will have again very soon for sure!).
Finally, and concluding my trip to the past for today, here's another view at the past 20 years of our industry: Did you know that "IBM Adstar" was the worlds biggest storage company in terms of revenues in 1992?
Thursday, March 28, 2013
April I
Completing the March topic on Analyst Reports for 2012, here's a comment on what Gartner has to contribute. As Storage Newsletter lines out, IDC and Gartner numbers do not exactly match - which should not come as a big surprise given the complexity of the markets and the somewhat blurred lines between hardware and software revenues and how to account for them. Quote from the Gartner report: "Gartner ECB disk storage reports reflect vendor-branded hardware-only revenue, as well as hardware revenue associated with financial leases and managed services. Optional and separately priced storage software revenue and SAN infrastructure components are excluded."
Gartner also came out with a new iteration of their famous Magic Quadrant: the MQ for what they now call "General Purpose Disk Arrays". As per their definition, those are "... designed to address shared storage requirements within a single system that can support multiple virtual or physical servers, hosted virtual desktop infrastructures, multiple databases, and applications such as, but not limited to, SAP, Microsoft Exchange and Microsoft SharePoint."
Hmm...sounds like pretty much everything on the market today. To no surprise, the top right quadrant is very crowded, I pretty much doubt that anybody would draw any buying decisions from the cluster of dots arranged in there:
Appliances for a different purpose are the so called PBBAs (Purpose Built Backup Appliances). IDC reports market status and trends for that segment of the market, a great summary can be found here.
More locally to where I sit, there was a press release that last week about an enhanced way of building Flash Memory demonstrated by the EPFL in Lausanne:
"A Swiss government research lab has reinvented flash memory using graphene and molybdenite in a way that should be faster, scale smaller, use less energy and yet more flexible than boring old NAND.
Molybdenite is MoS2, molybdenum disulfide, which is similar to graphite and also has a lubricating effect. Atomically it is a layer of molybdenum atoms between top and bottom layers of sulfide atoms. It is a semiconductor and can be used to create transistors."
In a kind of bold move back to basics, Brocade has announced their latest and final addition to the 16 Gbps family of Fibre Channel switches and directors, now referred to as G5: The Brocade 6520, a 96-port 2U switch.
They not only emphasize the importance of FC SANs (sic!) but also present a strategy to enable easier management integration for Cloud deployments and also comment on their plans for a future 32 Gbps product: " Brocade also is working with the OpenStack Foundation on open source software to manage FC fabrics in clouds. Brocade and some of its OEM partners plan to propose a Fibre Channel SAN Zone Management blueprint at the OpenStack Summit in April."
My take: they should have sticked to their core business which is FC instead of spending millions to try to become another CISCO. Now, at last, I think they realize where their margins and revenue are coming from and are focusing on that important segment of the business again. And FCoE? Yes, what about it? Exactly!
The official FC roadmap can be found at fibrechannel.org here!
Gartner also came out with a new iteration of their famous Magic Quadrant: the MQ for what they now call "General Purpose Disk Arrays". As per their definition, those are "... designed to address shared storage requirements within a single system that can support multiple virtual or physical servers, hosted virtual desktop infrastructures, multiple databases, and applications such as, but not limited to, SAP, Microsoft Exchange and Microsoft SharePoint."
Hmm...sounds like pretty much everything on the market today. To no surprise, the top right quadrant is very crowded, I pretty much doubt that anybody would draw any buying decisions from the cluster of dots arranged in there:
More locally to where I sit, there was a press release that last week about an enhanced way of building Flash Memory demonstrated by the EPFL in Lausanne:
"A Swiss government research lab has reinvented flash memory using graphene and molybdenite in a way that should be faster, scale smaller, use less energy and yet more flexible than boring old NAND.
Molybdenite is MoS2, molybdenum disulfide, which is similar to graphite and also has a lubricating effect. Atomically it is a layer of molybdenum atoms between top and bottom layers of sulfide atoms. It is a semiconductor and can be used to create transistors."
In a kind of bold move back to basics, Brocade has announced their latest and final addition to the 16 Gbps family of Fibre Channel switches and directors, now referred to as G5: The Brocade 6520, a 96-port 2U switch.
They not only emphasize the importance of FC SANs (sic!) but also present a strategy to enable easier management integration for Cloud deployments and also comment on their plans for a future 32 Gbps product: " Brocade also is working with the OpenStack Foundation on open source software to manage FC fabrics in clouds. Brocade and some of its OEM partners plan to propose a Fibre Channel SAN Zone Management blueprint at the OpenStack Summit in April."
My take: they should have sticked to their core business which is FC instead of spending millions to try to become another CISCO. Now, at last, I think they realize where their margins and revenue are coming from and are focusing on that important segment of the business again. And FCoE? Yes, what about it? Exactly!
The official FC roadmap can be found at fibrechannel.org here!
Friday, March 15, 2013
March II
March is the time when the industry tallies up the previous year's storage numbers, 2013 being no exception!
So let me start with a couple statements and links from IDC regarding the 2012 status of the worldwide storage industry:
First, the official IDC press release, it states that "...the external disk storage systems market generated $24.7 billion in sales during the full year, which represented a 4.7% increase over 2011. Total external disk storage systems capacity shipped during the year surpassed 20 exabytes, which represents an increase of 27% over 2011."
Storage Newsletter and The Register both had great comments and a couple more insights beyond the naked numbers, like the one here from Storage Newsletter: "The storage industry continues to be one of the most successful sector in the IT industry, with total disk systems revenues growing yearly since 2002 except in 2009. But the growth is diminishing. It was 17.4% from 2009 to 2010 then 8.2% from 2010 to 2011 and 4.8% from 2011 to 2012 even if the total capacity shipped continues to explode. It means that the price per gigabyte continues to decline rapidly. Among the reasons: lower price of HDDs, several technologies to reduce total storage capacity of the systems (thin provisioning, compression, de-dupe)."
And finally, a breakdown for the EMEA numbers here.
Agree, these numbers are mostly about HDDs. Technology based on SSDs still play a very minor role in the overall external storage market and occupy a small (but growing) niche. According to Ambuj Goyal, newly appointed head of IBM's Storage Group, this could change soon: "Our play is the all-flash array for transaction processing, not hybrids (arrays combining disk and solid state storage)."
Changing topics: According to the latest Quorum Disaster Recovery Report, natural disasters account for only 5% of the IT outages for SMB customers, see here what the other 95% are!
And talking about DR: Here's the latest speed record for data transfer across optical fibre: "According to a statement from the OFC/NFOEC organizers, AT&T was able to send 400Gbps data transmissions more than 7,456 miles with minimal loss, thanks to new materials and a new modulation technique. Team leader Xiang Zhou said that his setup addresses several key concerns for next-generation networks."
So let me start with a couple statements and links from IDC regarding the 2012 status of the worldwide storage industry:
First, the official IDC press release, it states that "...the external disk storage systems market generated $24.7 billion in sales during the full year, which represented a 4.7% increase over 2011. Total external disk storage systems capacity shipped during the year surpassed 20 exabytes, which represents an increase of 27% over 2011."
Storage Newsletter and The Register both had great comments and a couple more insights beyond the naked numbers, like the one here from Storage Newsletter: "The storage industry continues to be one of the most successful sector in the IT industry, with total disk systems revenues growing yearly since 2002 except in 2009. But the growth is diminishing. It was 17.4% from 2009 to 2010 then 8.2% from 2010 to 2011 and 4.8% from 2011 to 2012 even if the total capacity shipped continues to explode. It means that the price per gigabyte continues to decline rapidly. Among the reasons: lower price of HDDs, several technologies to reduce total storage capacity of the systems (thin provisioning, compression, de-dupe)."
And finally, a breakdown for the EMEA numbers here.
Agree, these numbers are mostly about HDDs. Technology based on SSDs still play a very minor role in the overall external storage market and occupy a small (but growing) niche. According to Ambuj Goyal, newly appointed head of IBM's Storage Group, this could change soon: "Our play is the all-flash array for transaction processing, not hybrids (arrays combining disk and solid state storage)."
Changing topics: According to the latest Quorum Disaster Recovery Report, natural disasters account for only 5% of the IT outages for SMB customers, see here what the other 95% are!
And talking about DR: Here's the latest speed record for data transfer across optical fibre: "According to a statement from the OFC/NFOEC organizers, AT&T was able to send 400Gbps data transmissions more than 7,456 miles with minimal loss, thanks to new materials and a new modulation technique. Team leader Xiang Zhou said that his setup addresses several key concerns for next-generation networks."
Wednesday, February 27, 2013
March
Here's a summary of worthwhile news for the last couple news in the realm of storage, let's start with hard disk drives (HDDs) and potential ways to push the envelope for more capacity beyond perpendicular recording and GMR heads:
Shingled writes (picture below), HAMR (heat-assisted recording) and Helium-filled drives seem to be three proposed solutions to the challenge! While HAMR and Shingled writes would help to increase aerial density of HDD platters, the Helium used inside HDDs aims rather at allowing more platters per real-estate by lowering air-friction. Combining the two would yield the best results in terms of capacity increase but also obviously raise manufacturing costs.
Aerial density growth for HDDs has been down at the 20% level (per year) in the last couple years as you can read in this article by Hu Yoshida here, so to maintain capacity expansion, new technologies will be needed!
That same article also has some interesting trends regarding flash disk cost levels: expectations are that the per capacity costs of SSDs will reach SAS HDD levels in a 2019 time frame.
The reasons why we desperately need more capacity are discussed in the most recent "Digital Universe" study: IDC predicts that "...the digital universe will reach 40 zettabytes (ZB) by 2020, an amount that exceeds previous forecasts by 5 ZBs, resulting in a 50-fold growth from the beginning of 2010". I personally would argue that the kind of snapshots taken all over the world by cell phone users are not an "act of creating information"... but obviously these pictures still use up storage space in various places. Find the complete report here!
The related comment in this Storage Networking article here: "Digital hoarding: do we have a problem"?
And an outlook for the HDD market thru 2017 can be found here.
On the Wall Street side of our industry, the Q3 results of VMware generated some mixed echos: "VMware sees trouble ahead" was the title of an article in the financial press. But nevertheless, VMware continues to build out it's in-house storage ecosystem and announced the intent to acquire Virsto, the maker of "..the first VM-centric Storage Hypervisor".
Which closes the loop: Jon Toigo has his very own and distinct opinion about storage hypervisors (see graph above): "Then, when VMware engineers climbed on stage a year or so ago and said that they were going to add another microkernel they called a storage hypervisor to the clusterf#@k of microkernels that they fancy to be a OS, I couldn’t believe my ears. Imagine if each of the leading hypervisor vendors decide to do a storage virtualization microkernel too: that would suck."
Friday, January 18, 2013
January 2013
It's January, it's cold and it's dark...time to book summer vacation - and review pictures from last year's summer vacation!
Although we got some snow in Zurich and we are reasonably well equipped to deal with snow and ice, the Canadians do it even better...very impressive parade of Snowplows!
Continuing with my December post on 2013 strategy statements and economic outlooks, here's the insights from HP's David Scott, Netapp's Jay Kid as well as a listing of the 15 most promising storage startups for the new year.
Even more interesting though is the list from a year ago (on that same link above) and the comparison between the two lists:
Some companies went from "most promising" to "unknown" (like Coraid) while others made it back onto that same list again (like Violin, ranked 2nd) or have been acquired (like XtremIO by EMC) or are well established brand names by now (like Dropbox).
You will notice that many of the listed companies are involved with some sort of SSD technology and solutions, the most exciting real product hitting the market these days is probably the Crucial 500, a 1 TB SSD drive for consumers which sells for under 600$!
Also in 2013, the industry expects the first SATA Express products, based on a 16 Gbps SATA standard which is under development. See the SATA-IO website here for details and this site here for additional information and resources.
Lastly, on the topic of the recent revival of tape, here's a great article by Jon Toigo: "The most important metric you're likely to confront in the next 10 to 20 years is the cost and availability of electricity."
Read here how he thinks SSD and Tape could be combined to address these challenges.
The Leibniz Supercomputer Center (LRZ) in Germany already uses this approach very successfully! Here's the related IBM Press Release.
Although we got some snow in Zurich and we are reasonably well equipped to deal with snow and ice, the Canadians do it even better...very impressive parade of Snowplows!
Continuing with my December post on 2013 strategy statements and economic outlooks, here's the insights from HP's David Scott, Netapp's Jay Kid as well as a listing of the 15 most promising storage startups for the new year.
Even more interesting though is the list from a year ago (on that same link above) and the comparison between the two lists:
Some companies went from "most promising" to "unknown" (like Coraid) while others made it back onto that same list again (like Violin, ranked 2nd) or have been acquired (like XtremIO by EMC) or are well established brand names by now (like Dropbox).
You will notice that many of the listed companies are involved with some sort of SSD technology and solutions, the most exciting real product hitting the market these days is probably the Crucial 500, a 1 TB SSD drive for consumers which sells for under 600$!
Also in 2013, the industry expects the first SATA Express products, based on a 16 Gbps SATA standard which is under development. See the SATA-IO website here for details and this site here for additional information and resources.
Lastly, on the topic of the recent revival of tape, here's a great article by Jon Toigo: "The most important metric you're likely to confront in the next 10 to 20 years is the cost and availability of electricity."
Read here how he thinks SSD and Tape could be combined to address these challenges.
Subscribe to:
Posts (Atom)
Posts
