Cloud is changing the storage business in more ways than just price per unit. It is fundamentally changing how we design our storage systems and which way we deploy, protect and recover them. For those most fortunate companies who are just starting out the cloud is an easy task as there is no legacy systems or tried and true methods, it has always been on the ‘cloud’.
For most companies that are trying to find ways to cut their storage cost while keeping some control of their storage, cloud seems to be the answer. But getting there is not an easy tasks as most have seen. The transfer of data, code that has to be rewritten, systems and processes that all have to be changed just to report back to their CIO that they are using the cloud.
Now there are many ways to get to the cloud but one that I am excited about is using technology originally deployed back in the late 90s.
GPFS (errr, $1 in the naughty jar) Spectrum Scale is a parralel file system that can spread the data across many different tiers of storage. From flash to spinning drives to tape, Scale has the ability to alleviate storage administration by policy based movement of data. This movement is based on the metadata and is written, moved and deleted based on policies set by the storage admin.
So how does this help you get to the cloud? Glad you asked. IBM released a new plug in for Scale that treats the cloud as another tier of storage. This could be from multiple cloud vendors like IBM Cleversafe, IBM Softlayer, Amazon S3 or a private cloud (Think Openstack). The cloud provider is attached to the cloud node over ethernet and allows your Scale system to either write directly to the cloud tier or move data as it ages/cools.
This will do a couple of things for you.
- Because we are looking at the last read date, data that is still needed but the chance you will read it is highly unlikely can be moved automatically to the cloud. If a system needs the file/object there is no re-coding that needs to be done as the namespace doesn’t change.
- If you run out of storage and need to ‘burst’ out because of some monthly/yearly job you can move data around to help free up space on-perm or write directly out to the cloud.
- Data protection such as snapshots and backups can still take place. This is valuable to many customers as they know the data doesn’t change often but like the idea they don not have to change their recovery process every time they want to add new technology.
- Cheap Disaster Recovery. Scale does have the ability to replicate to another system but as these systems grow larger and beyond multiple petabytes, replication becomes more difficult. For the most part you are going to need to recover the most recent (~90 Days) of data that runs your business. Inside of Scale is the ability to create mirrors of data pools. One of those mirrors could be the cloud tier where your most recent data is kept in case there is a problem in the data center.
- It allows you to start small and work your way into a cloud offering. Part of the problem some clients have is they want to take on too much too quickly. Because Scale allows customers to have data in multiple clouds, you can start with a larger vendor like IBM and then when your private cloud on Openstack is up and running you can use them both or just one. The migration would be simple as both share the same namespace under the same file system. This frees the client up from having to make changes on the front side of the application.
Today this feature is offered as an open beta only. The release is coming soon as they are tweaking and doing some bug fixes before it is generally available. Here is the link to the DevWorks page that goes into more about the beta and how to download a VM that will let you test these features out.
I really believe this is going to help many of my customers move into that hybrid cloud platform. Take a look at the video below and how it can help you as well.
Currently, I am working with a customer on their archive data and we are discussing which is the better medium for their data that never gets read back into their environment. They have about 200TB of data that is sitting on their Tier 1 that is not being accessed, ever. The crazy part is this data is growing faster than the database that is being accessed by their main program.
This is starting to pop up more and more as the unstructured data is eating up storage systems and not being used very frequently. I have heard this called dark data or cold data. In this case its frozen data.
We started looking at what it would cost them over a 5 year period to store their data on both tape and cloud. Yes, that four letter word is still a very good option for most customers. We wanted to keep the exercise simple so we agreed that 200TB would be the size of the data and there would be no recalls on the data. We know most cloud providers charge extra for the recalls so we wanted and of course the tape system doesn’t have that extra cost so we wanted an apples to apples comparison. As close as we could.
For the cloud we used Amazon Glacier pricing which is about $0.007 per GB per month. Our formula for cloud:
200TB X 1000GB X $0.007 x 60 months = $84,000
The tape side of the equation was a little more tricky but we decided that we would just look at the tape media and tape library in comparison. I picked an middle of the road tape library and the new LTO7 media.
Tape Library TS3200 street price $10,000 + 48 LTO7 tapes (@ $150 each) = $17,200
We then looked at the ability to scale and what would happen if they factored in their growth rate. They are growing at 20% annually which translates to 40TB a year. Keeping the same platforms what would be their 5 year cost? Cloud was..
200TB + (Growth of 3.33TB per month) x 1000GB x 60 months = $125,258
Tape was calculated at:
$10,000 for the library + (396TB/6TB LTO7s capacity)x$150 per tape = $19,900
We all here how cloud is so much cheap and easier to scale but after doing this quick back of the napkin math I am not so sure. I know what some of you are saying that we didn’t calculate the server costs and the 4 FTEs it takes to manage a tape system. I agree this is basic but in this example this is a small to medium size company that is trying to invest money into getting their product off the ground. The tape library is fairly small and should be a set it and forget it type of solution. I doubt there will much more overhead for the tape solution than a cloud. Maybe not as cool or flashy but for $100,000 over 5 years they can go out and buy their 5 person IT staff a $100 lunch everyday, all five years.
So to those who think tape is a four letter word and is that thing in the corner that no one wants to deal with, I say embrace it and squeeze the value out of them. Most IT shops have tape still and can show to their finical teams how they can lower their cost with out putting their data at risk in the cloud with this:
IBM changed the way they are going to market with the Spectrum Storage family of software defined storage platform. Since the initial re-branding of their software formerly known as Tivioli, XIV, GPFS, SVC, TPC and LTFS, the plan was to create a portfolio of packages that would aid in protecting and storing data on existing hardware or in the cloud. This lines up with how Big Blue is looking for better margins and cloud ready everything.
These platforms, based on a heritage of IBM products, now are available as a suite where a customer can order the license (per TB) with unlimited usage for all six offerings. The now allows customers to move more rapidly into the SDS environment not have a complex license agreement to manage. All of the Spectrum family is based on a similar look and feel and support is all done through IBM.
Clients will have to license the software only for production capacity. Since all of the software is part of the suite, clients can also test and deploy different items and mix and match as they see fit. If you need 100TB of data protection, this allows you to have 50TB or Spectrum Protect and maybe 50 TB of Spectrum Archive. If you then need to add storage monitoring IE Spectrum Control, then your license count doesn’t start from 0 but at 100TB. If anything has taught me working with IBM, the more you buy of the same thing the cheaper per unit it will be in the end.
For more information on the Spectrum Storage Suite go to the IBM home here:
So many things to talk about but a couple of notes of interest from today:
- DS8870 is a new system not just an upgrade. IBM went from the P6 server to the P7 which should give them a huge performance bump. I heard there are some impressive SPC numbers coming soon.
- XIV gets a GUI improvement with Multi-system manager. This will help drive some efficiency in management of those environments with larger deployments.
- V7000 Unified gets compression for file. Same story as on block but now for file objects.
Here are links to the hardware and software announcements from today.
|IBM System Storage DS8870 (Machine type 2423) Models 961 and 96E with three-year warranty|
|IBM System Storage TS1060 Tape Drive offers an Ultrium 6 Tape Drive for the TS3500 Tape Library|
|IBM Virtualization Engine TS7700 supports disk-based encryption|
|IBM System Storage DS8870 (Machine type 2421) Models 961 and 96E with one-year warranty|
|IBM System Storage DS8870 (Machine type 2424) Models 961 and 96E with four-year warranty|
|IBM System Storage DS8000 series high-performance flagship – Function Authorizations for machine type 239x|
|IBM System Storage DS8870 (Machine type 2422) Models 961 and 96E with two-year warranty|
|IBM Systems Director Standard Edition for Linux on System z, V6.3 now manages zBX blades|
|IBM Systems Director product enhancements provide tools to better manage virtual and physical networks|
|XIV management is designed to enable more effective XIV deployments into private cloud computing environments and improve multi-system management|
|IBM Storwize V7000 Unified V1.4 includes real-time compression, local authentication server support, four-way clustering, and FCOE support|
|IBM Programmable Network Controller V3.0, when used with OpenFlow-enabled switches, provides architecture for centralized and simplified networking|
|IBM SmartCloud Virtual Storage Center V5.1 offers efficient virtualization and infrastructure management to enable smarter storage|
|IBM Tivoli Storage Manager V6.4 products deliver significant enhancements to manage data protection in virtual environments|
|IBM Infoprint XT for z/OS, V3.1 provides support to transform Xerox data streams and highlight color resources for printing on AFP printers and enhances DBCS support|
|IBM Security zSecure V1.13.1 products and solutions enhance mainframe security intelligence, compliance, administration and integration|
|IBM Tivoli Storage FlashCopy Manager V3.2 extends application-aware snapshot management to IBM N series and NetApp devices and enables seamless disaster recovery|
IBM published a paper this week describing how the scale out NAS product, SONAS, works with a software package in the seismic processing space called Paradigm Epos4. The report goes into detail of both the hardware and software issues surrounding the massive amounts of data associated with finding deposits of fossil fuels in the strata.
The software supports NFS mounts which seems to be the sweet spot of the linux based SONAS system. One of the biggest problems with the oil and gas industry is the tremendous and rich amount of data.
The cost of drilling varies depending on the depth of the well, remoteness of the location and extra services required to get the oil or gas up to the surface. With some of the deepwater rigs the rates for 2010 was around $420,000 per day and could be more on higher performance rigs.
With so much on the line, it is very important to get information accurate and quickly so that companies can avoid costly mistakes. IBM has been working in the oil and gas industry for over 50 years. We have experts not only in the hardware, software and services but we understand the industry and how “big-data” is changing that industry faster than others.
SONAS allows for companies to have a large scale NAS solution that can have a single files system for multiple peta-bytes of data. SONAS also allows data to move from faster pools to other virtualized systems down to a tape archive. This increases the ROI by having the most recent accessed data on the faster drives and customers can expand their buying cycles further because they are not spinning old data.
The other variable in this industry is companies need to scale projects up quickly and not always with a 1:1 ratio of performance to storage space. SONAS is able to scale both of these variables independently of one another. As new systems are brought online, disks can be added and rebalanced non disruptively. The same can be done with the interface nodes.
More information about the testing can be found in the report here.
Top 10 Reasons clients choose to go with IBM N series
Some years ago I put together a list of reasons why people choose to buy from IBM rather than purchase directly from Netapp. IBM has an OEM agreement with Netapp and rebrands the FAS and V-series as their N series product line. They are both made at the same plant and the only difference between them is the front bezel. You can even take a Netapp bezel off and stick it on an N series box and it fits exactly.
The Software is the same exactly. All we change is the logos and readme files. The entire functionality of the product is exactly the same. IBM does not add or take away any of the features built into the systems. The only difference is it takes IBM about 90 days once Netapp releases a product to get it put online and change the necessary documents.
Support for N series is done both at IBM and Netapp. Much like our other OEM partners, they stand behind IBM as the developers and IBM handles the issues. Customers still call the same 1.800.IBM.SERV for support and speak to trained engineers who have been working on N series equipment for 6+ years now. IBM actually has lower turn over than Netapp in their support division and has won awards for providing top-notch support. The call home features that most people are used to still go to Netapp via IBM servers.
10. The IBM customer engineer (CE) that is working with you today will be the same person who helps you with the IBM N series system.
9. IBM GBS team can provide consultation, installation and even administration of your environment.
8. IBM is able to provide financing for clients.
7. When you purchase your N series system from IBM, you can bundle it with servers, switches, other storage and software. This gives you one bill, one place to go to if you need anything and one support number to call.
6. IBM has two other support offerings to help our clients, Our Supportline offering allows customers to call in and ask installation or configuration questions. We also have an Enhanced Technical Support (ETS) team that will assign a personal engineer that will know everything about your environment and will provide you with everything you need. They will help you with health checks to be sure the system is running optimally, updates on the latest technology and single point of contact in case you need to speak to someone immediately.
5. IBM N series warranty support is done by IBM technicians and engineers at Level 1 and Level 2. If your issue can not be resolved by our Level 2 team they have a hotline into the Netapp Top Enterprise Account team. This is a team only a few very large Netapp accounts can afford and we provide this support to ALL of the IBM N series accounts no matter how large or small.
4. Our support teams from different platforms (X series, Power, TSM, DS, XiV, etc) all interact with another and when tough issues come up we are able to scale to the size of the issue. We can bring in experts that know the SAN, Storage, Servers, and Software all under one umbrella. With those tough cases we assign a coordinator to make sure the client does not have to call all of these resources themselves. This person can reach out to all the teams, assigns duties and will coordinate calls with you the customer.
3. All IBM N series Hardware and Software undergoes an Open Source Committee who validates there are no violations, copy right infringements or patent infringements.
2. All IBM N series Hardware and Software is tested in our Tucson testing facility for interoperability. We have a team of distinguished engineers who not only support N series but other hardware and software platforms within in the IBM portfolio.
1. All IBM N series equipment comes with a standard 3 year warranty for both Hardware and Software. This warranty can be extended beyond the three years as IBM supports equipment well beyond the normal 3-5 years of a system.
When it gets down to it, customers buy because they happy. Since the systems are exactly the same it comes down to what makes them happy. For some, the Netapp offering makes them happy because they like their sales engineer, for others they like IBM because they have been doing business with us for over 30 years.
For more information about IBM N series, check out our landing page on http://www-03.ibm.com/systems/storage/network/
Now available is the IBM System Storage N series with VMware vSphere
Redbooks are a great way of learning a new technology or a reference for configuration. I have used them for years not just in storage but for X series servers and for software like TSM. The people that write the books spend a great deal of time putting them together and I believe most of them are written by volunteers.
This is the third edition of this Redbook and if you have read this before here are some of the changes:
-Latest N series model and feature information.
-Updated the IBM Redbook to reflect VMware vSphere 4.1 environments
-Information for Virtual Storage Console 2.x has been added
This book on N series and VMware goes through the introduction of both the N series systems and VMware vSphere. There are sections on installing the systems, deploying the LUNs and recovery. After going through this Redbook, you will have a better understanding of a complete and protected VMware system. If you need help with how to size your hardware there is a section for you. If you are looking to test how to run VMs over NFS, its in there too!
One of the biggest issues with virtual systems is making sure you have proper alignment between the system block and the storage array. This will negatively impact the system by a factor of 2 in most random reads/writes as two blocks will be required for one request. To avoid this costly mistake or to correct VMs you have already setup a section in the book called Partition alignment walks you through the entire process of correctly setting the alignment or fixing the older systems correctly.
Another area that I will point out is the use of deduplication, compression and cloning to drive the efficiency of the storage higher. These software features allow customers to store more systems on the storage array than if they used traditional hard drives. Also there is how to use snapshots for cloning, mirrors for Site Recovery Manager and long term storage aka Snapvaults. At the end of the book are some examples of scripts one might use for snapshots in hot backup modes.
Whether you are a seasoned veteran or newbie to the VMware scene, there is a great guide that will help you from start to finish setting up your vSphere environment. The information is there, use the search feature or sit down on a Friday with a high-lighter, which ever fits your style and learn a little about using a N series system with VMware.
Here is the link to this Rebook: