[nflug] Opinions on Linux and Massive Storage

[Brad Bartram]

Believe it or not, I'm actually not doing anything with a database for
this.  This is going to be massive file storage - or storage for
massive files, if you prefer.  This will be mainly a read intensive
system, though there will be writes from time to time.

I'm looking at fibre channel for connections with fast disks.
Initially I'm looking to have a system between 40 - 100TB to start
with expansion in 4 - 6 months of at least another 100TB.

I've looked at some of the offerings from Sun, but both IBM and Dell
have SAN hardware that fits the bill.  I'd like to stay with Linux,
since I'm most familiar with it, but if there are limitations in
dealing with big storage arrays, I have no problem moving to a
different platform.

On Mon, Jun 30, 2008 at 12:56 PM, Robert Meyer <meyer_rm at yahoo.com> wrote:
> Well, for that kind of storage, I'd recommend getting an EMC SAN setup.  Are
> you planning on building something for static data that you're just going to
> be mostly reading or doing lots of read and write activity.  If it's the
> latter, you're going to want SCSI interfaces on the drives.  EMC makes both
> SCSI and ATA SANs.  You want a configuration that uses fiber connects to the
> SAN, if you can afford it.
>
> If you're doing heavy database work, when you build the RAIDs, I'd recommend
> going with a RAID 10 with as many disks as you can if you're doing lots of
> random access.  I found that with databasing, RAID5 is really bad at large
> scale writes.  RAID 10 with lotsa disks will give you more speed (the more
> disks that you can spread out over the better).  Not having to compute
> parity is a major win.  Go for more disks with less capacity, rather than
> fewer, large capacity disks, if speed is the major issue.
>
> Also, try to not build multiple systems on the same spindle sets if you're
> doing databasing.  I've watched a single spindle set show massive I/O wait
> when multiple systems were hitting it.
>
> This can get really complex, really fast.  Basically, I think I'd need lots
> more information on the intended use of the system in order to be able to
> help with it.  If you have a set of design requirements, that would help a
> lot.
>
> Cheers!
>
> Bob
>
> --
> "When once you have tasted flight, you will forever walk the earth with your
> eyes turned skyward, for there you have been, and there you will always long
> to return."
> --Leonardo da Vinci
>
> ----- Original Message ----
> From: Brad Bartram <brad.bartram at gmail.com>
> To: nflug at nflug.org
> Sent: Monday, June 30, 2008 12:33:50 PM
> Subject: [nflug] Opinions on Linux and Massive Storage
>
> I know there are some people in this list that have experience with
> massive storage using linux.  By massive I mean >20TB range.
>
> I'd love to hear your thoughts on building out and optimizing a system
> that is fast, scalable, and reliable.  If you have opinions on direct
> attached storage as well as those of you running storage area
> networks.
>
> It's kind of a broad topic, but I'm about to embark on a major build
> out and want to avoid as many pitfalls as possible.
>
> Thanks
>
> Brad
> _______________________________________________
> nflug mailing list
> nflug at nflug.org
> http://www.nflug.org/mailman/listinfo/nflug
>
>
> _______________________________________________
> nflug mailing list
> nflug at nflug.org
> http://www.nflug.org/mailman/listinfo/nflug
>
>