[nflug] Opinions on Linux and Massive Storage

[Erek Dyskant]

I'm in the process of designing a 10TB-50TB iscsi storage system using
ZFS and comstar.  Really, for what you're talking about, Linux storage
isn't the best option, as it doesn't have the filesystem and volume
management features to handle large data sets gracefully.  Sure, you can
set up a gigantically huge logical volume in LVM, but what about
performance friendly snapshots, replication, live expansion, etc.

In my opinion, ZFS is stability mature but potentially not featureset
mature.  It's stable, and many organizations are relying on it to manage
hundreds of TBs of data under heavy stress.  However, from a product
management perspective Sun isn't positive where they're going with it so
the featureset roadmap is a little fuzzy.

If you prefer the Linux userspace tools, have a look at Nexenta.  It's a
debian-based distribution with gnu tools, but the solaris kernel,
dtrace, zones, etc.  They do a good job of picking the solaris features
so you get stable stuff but a little more cutting edge than solaris
(closer to opensolaris express)

Also, be sure to get the pitches from EMC, Falconstor, NetApp, etc.  You
may find that when all's said and done you're still better with a
traditional hardware storage array.

Do update us on your progress along the way.  It sounds like a really
interesting project.

Cheers,
Erek

On Mon, 2008-06-30 at 13:41 -0400, Brad Bartram wrote:
> Thanks for the thoughts.
> 
> How has ZFS been for you?  I've looked into it but never quite got the
> feeling it was production ready - is that a fair assessment or am I
> mistaken?
> 
> Primarilly, this is going to be server to server communications.  A
> few clients that need immediate, high-speed, reliable access to big
> (~100GB) data sets for processing.
> 
> The money for the sparc, at this point isn't a huge deal.  We're
> already getting to the happy side of $100K to enter this project, so a
> few more dollars isn't going to bust the bank.  My big thing is
> scalability and reliability.  Within 12 - 18 months, I'm figuring on
> seeing Petabyte size storage, so I want to make sure the ground work
> is down correctly now.  It's not going to be too long before I'm over
> my head on this, so when I finally do hire a storage guy (or girl) to
> take the reigns, I don't want to have to backtrack - at least too
> much.
> 
> I'm definitely interested in finding out more from the Hitachi rep.
> You can send me his info off-list if you want.