Prior to my happiness in using all things ZFS, I was eagerly testing Lustre. Primary concerns there was both management of the backend storage and ability to translate this network file system to standard NFS and CIFS for non-Linux clients. Cluster FileSystems was recently announced to be acquired by Sun, but prior to that, they made announcements about how Lustre and ZFS would be married. I always thought that perhaps it was that ZFS would be the exposed layer on a client, and that it was the same old OSTs and OSDs on the backend. Not so.
CFS Moves Lustre to ZFS
Instead, ZFS is the backend to the storage nodes. Albeit that the driver pool is somewhat less for OpenSolaris than Linux, it does make one wonder exactly what new flexibility is afforded by this arrangement. Also, how much of this will end up as open sourced code to be incorporated into highly manageable product, such as my new favorite appliance, NexentaStor? I'll be diving further into this as more details emerge. Perhaps others can chime in with more info or clarification.
Friday, November 02, 2007
Thursday, November 01, 2007
The Coming Out Party for Commodity Storage
If you have been following along, I remarked in http://jmlittle.blogspot.com/2007/09/multi-tier-storage-revisited.html that "the increasing capabilities of Nexenta's storage solution and its underlying OpenSolaris base have proceeded a pace, and I believe the future is very bright for this solution". Its one of the few bright spots that I've had the privilege of using to enable commodity-based storage solutions. I've been an early adopter of the NexentaStor multi-tier storage appliance, and I am happy to hear that not only is it approaching its first general release to customers, but a release candidate is being made available to the public. Although I run it directly on hardware, the VMware evaluation version of the product has been deemed fit enough for people to kick the tires and see exactly how this fits in the organization. Check out http://www.nexenta.com
Many will ask how this is different from either the hardware based NAS and Disk-to-Disk solutions, and others will wonder how does this compare to FreeBSD and Linux based solutions and projects already on the market. It comes down to what its does best now, and the potential of where it will go in the future. If you haven't been catching the storage news lately, NexentaStor is the first major product being built on the ZFS filesystem which brings to commodity storage much of what has till now only be accessible by the hardware vendors. Its that secret sauce that has justified those large margins and high priced "vendor-provided and tested" disk drives. What if you could just build it out on your own? Many open source solutions supposedly allow for just that, but its somewhat beyond a do-it-yourself level: the pieces aren't necessarily all integrated, nor is the complete solution truly comparable to commercial solutions or are they production ready. The test is would you feel safe having 50TB of your backups on that solution?
ZFS is all fine and good, but its the integration I speak of that have made me settle on this particular product. It also brings a fully developed commercial grade NFSv4 server solution, fully managed snapshots with the necessary scheduling, multiple replication and tiering services to integrate it anywhere in my digital archive flow, virtualized and thin provisioning, iSCSI target and client support of said storage, and when installed on brawnier hardware architected to grow, it will quickly eclipse many heavily marketed primary storage solutions, at a true fraction of the cost.
Nexenta is building this on OpenSolaris and their own hybrid opensolaris/debian-style distribution. Its has just started to stretch its legs when it comes to potential. However, our use is in second-tier storage, and that truly is where is shines right now. We've already thrown 50TB of disk at this via SCSI, iSCSI, SATA, and the like. It enables reuse of the storage you have now for a credible tiering architecture, and its both the web based interface and extensive command line interface that allow both legacy and new storage components to be managed. I could go to a zetabyte of storage with unlimited snapshots with the current installation, but one would undoubtedly want a more thought out long term hardware architecture. At least the current design allows for phasing in new technology while phasing out the old in the same pools I use today. Long term, I have high hopes that the product further simplifies data growth and management of a multitude of devices.
Now that this is finally available for public consumption, I'll be able to speak more and provide good best practice advice. Here is some ready advice to keep in mind:
1) As per disk capacity grows while prices drop, the exposure window of rebuilding any lost disk makes it more clear that RAID10 provides the best of all worlds for volume growth, redundancy, and recoverability.
2) Don't throw away your primary storage. Its still a mature product, and NexentaStor is best suited to secondary storage at this time. Long term, you can migrate that primary storage into the second tier, managed by NexentaStor. Once you are familiar and comfortable with commodity based storage solutions, you'll find it moving to primary storage environments when its good and ready.
3) That all said, commodity based storage solutions are now here. The wait is over, jump on in today.
Many will ask how this is different from either the hardware based NAS and Disk-to-Disk solutions, and others will wonder how does this compare to FreeBSD and Linux based solutions and projects already on the market. It comes down to what its does best now, and the potential of where it will go in the future. If you haven't been catching the storage news lately, NexentaStor is the first major product being built on the ZFS filesystem which brings to commodity storage much of what has till now only be accessible by the hardware vendors. Its that secret sauce that has justified those large margins and high priced "vendor-provided and tested" disk drives. What if you could just build it out on your own? Many open source solutions supposedly allow for just that, but its somewhat beyond a do-it-yourself level: the pieces aren't necessarily all integrated, nor is the complete solution truly comparable to commercial solutions or are they production ready. The test is would you feel safe having 50TB of your backups on that solution?
ZFS is all fine and good, but its the integration I speak of that have made me settle on this particular product. It also brings a fully developed commercial grade NFSv4 server solution, fully managed snapshots with the necessary scheduling, multiple replication and tiering services to integrate it anywhere in my digital archive flow, virtualized and thin provisioning, iSCSI target and client support of said storage, and when installed on brawnier hardware architected to grow, it will quickly eclipse many heavily marketed primary storage solutions, at a true fraction of the cost.
Nexenta is building this on OpenSolaris and their own hybrid opensolaris/debian-style distribution. Its has just started to stretch its legs when it comes to potential. However, our use is in second-tier storage, and that truly is where is shines right now. We've already thrown 50TB of disk at this via SCSI, iSCSI, SATA, and the like. It enables reuse of the storage you have now for a credible tiering architecture, and its both the web based interface and extensive command line interface that allow both legacy and new storage components to be managed. I could go to a zetabyte of storage with unlimited snapshots with the current installation, but one would undoubtedly want a more thought out long term hardware architecture. At least the current design allows for phasing in new technology while phasing out the old in the same pools I use today. Long term, I have high hopes that the product further simplifies data growth and management of a multitude of devices.
Now that this is finally available for public consumption, I'll be able to speak more and provide good best practice advice. Here is some ready advice to keep in mind:
1) As per disk capacity grows while prices drop, the exposure window of rebuilding any lost disk makes it more clear that RAID10 provides the best of all worlds for volume growth, redundancy, and recoverability.
2) Don't throw away your primary storage. Its still a mature product, and NexentaStor is best suited to secondary storage at this time. Long term, you can migrate that primary storage into the second tier, managed by NexentaStor. Once you are familiar and comfortable with commodity based storage solutions, you'll find it moving to primary storage environments when its good and ready.
3) That all said, commodity based storage solutions are now here. The wait is over, jump on in today.
Friday, October 12, 2007
Stanford's multi-tier solution: an Example
In my last post covering our year long foray into multi-tier storage, I promised I would detail the specific configuration of systems used at Stanford in production. Prominently featured are a NetApp FAS3050 (28TB raw, standard dual parity RAID4) used as the primary storage, as well as a Sun X4500 utilizing NexentaStor (24TB raw, RAIDZ). We have a secondary NexentaStor head for location independence as well as further expansion with both SCSI and iSCSI attached storage, adding about another 26TB. What's missing from the picture are two important facts. First, other general purpose file servers are also being tiered to the second head. The second point is that one of them is a 4TB NAS unit based on the same NexentaStor product. The various capabilites of Nexenta's product allow it to perform well as primary storage, but in the commodity hardware realm, you are currently lacking in some features the the most discerning of storage customers will still find in most integrated hardware solutions. Time alone will see where the proper mix of hardware and Nexenta's solution end up.
Friday, September 28, 2007
Multi-Tier Storage revisited
Its been a year since I posted here regarding Stanford's current and planned use of various storage solutions to virtually eliminate tape-based nightly archiving. Since then, the industry has gone through various changes, and in some cases, not much change at all.
Specifically, part of our solution used NeoPath's FileDirector for file based virtualization, and SBEI's iSCSI target solution for our backend storage. In the middle, pulling data from our primary NetApp fileservers, was a burgeoning solution being BETA tested at Stanford from Nexenta. So what has changed? NeoPath was acquired by Cisco, with the current product in production ceasing to be supported. SBEI has been acquired by NeoNode, and their iSCSI target, best in class for enabling commodity storage, isn't getting much love. How has Nexenta fared? While we will likely need to migrate away from the other solutions, the increasing capabilities of Nexenta's storage solution and its underlying OpenSolaris base have proceeded a pace, and I believe the future is very bright for this solution.
The NexentaStor product, in early BETA, delivers today on providing a snapshot based large scale file system, utilizing underlying storage pools (iSCSI, SCSI, SATA, FC, etc) and a well developed services architecture including data synchronization and replication, multi-host data tiering, and other facilities with data retention and disaster management to boot. Its base system disks even have bullet proof checkpointing, reversion, and safe updating, redundancy, all in a software solution. The future? Well, its easy to perceive with upcoming NFS v4.1 support that the product can tackle name space virtualization one has found in products such as the NeoPath. Already it can repurpose snapshot-based raw volumes as iSCSI targets, so if the underlying hardware is supported by OpenSolaris, you have an easily managed enterprise-feature level iSCSI target product.
Stanford has over a years worth of second tier data, in both 60 daily and 12 monthly snapshots, tiered from our NetApp. These are within many separate folders, representing over a thousand snapshots per volume. We've recently adopted the Sun X4500 24TB product and have migrated to this ideal solution for quicker disaster recovery. The read speeds on this 48 drive unit are great, and the price point rivals what we've built with iSCSI. Commodity storage is commodity storage, but we continue to utilize iSCSI, DAS (SATA-to-SCSI units), and other additional units to eclipse 48TBs of secondary storage. We have also utilized this solution for one organization as both first and second tier storage, an additional 16TB when we consider their solution, and it has proven its worth both in day to day NAS use as well as some data recovery and full disaster recovery modes.
Now that Nexenta supports some backup software as well as a client, we've only backed up directly to tape from the second tier once per year. We've let our LTO-2 tape library run continuously for around a week just to give us a full archived edition of our data. Are we missing tape reuse, tape-based recovery, or multiple library scheduling (and rescheduling) just to meet an ever growing nightly backup window? Nope. Nexenta looks to be here to stay.
I'll follow up later with specific details on configuration, where I hope things will go, and other random thoughts. On this anniversary, it would appear commodity-based multi-tier storage is practical and readily available.
Specifically, part of our solution used NeoPath's FileDirector for file based virtualization, and SBEI's iSCSI target solution for our backend storage. In the middle, pulling data from our primary NetApp fileservers, was a burgeoning solution being BETA tested at Stanford from Nexenta. So what has changed? NeoPath was acquired by Cisco, with the current product in production ceasing to be supported. SBEI has been acquired by NeoNode, and their iSCSI target, best in class for enabling commodity storage, isn't getting much love. How has Nexenta fared? While we will likely need to migrate away from the other solutions, the increasing capabilities of Nexenta's storage solution and its underlying OpenSolaris base have proceeded a pace, and I believe the future is very bright for this solution.
The NexentaStor product, in early BETA, delivers today on providing a snapshot based large scale file system, utilizing underlying storage pools (iSCSI, SCSI, SATA, FC, etc) and a well developed services architecture including data synchronization and replication, multi-host data tiering, and other facilities with data retention and disaster management to boot. Its base system disks even have bullet proof checkpointing, reversion, and safe updating, redundancy, all in a software solution. The future? Well, its easy to perceive with upcoming NFS v4.1 support that the product can tackle name space virtualization one has found in products such as the NeoPath. Already it can repurpose snapshot-based raw volumes as iSCSI targets, so if the underlying hardware is supported by OpenSolaris, you have an easily managed enterprise-feature level iSCSI target product.
Stanford has over a years worth of second tier data, in both 60 daily and 12 monthly snapshots, tiered from our NetApp. These are within many separate folders, representing over a thousand snapshots per volume. We've recently adopted the Sun X4500 24TB product and have migrated to this ideal solution for quicker disaster recovery. The read speeds on this 48 drive unit are great, and the price point rivals what we've built with iSCSI. Commodity storage is commodity storage, but we continue to utilize iSCSI, DAS (SATA-to-SCSI units), and other additional units to eclipse 48TBs of secondary storage. We have also utilized this solution for one organization as both first and second tier storage, an additional 16TB when we consider their solution, and it has proven its worth both in day to day NAS use as well as some data recovery and full disaster recovery modes.
Now that Nexenta supports some backup software as well as a client, we've only backed up directly to tape from the second tier once per year. We've let our LTO-2 tape library run continuously for around a week just to give us a full archived edition of our data. Are we missing tape reuse, tape-based recovery, or multiple library scheduling (and rescheduling) just to meet an ever growing nightly backup window? Nope. Nexenta looks to be here to stay.
I'll follow up later with specific details on configuration, where I hope things will go, and other random thoughts. On this anniversary, it would appear commodity-based multi-tier storage is practical and readily available.
Saturday, May 12, 2007
CommunityOne
I had the pleasure of presenting the case for Nexenta, an OpenSolaris distribution that combines the best of Solaris with what is best in an Ubuntu/Debian distribution, at CommunityOne last week. My slides are now available online. The demos actually reference Martin Man's great flash demos that he posted at his site.
Subscribe to:
Posts (Atom)