Growing Storage Capacity

[SmilingInSeattle]

A couple of years ago I built a RaidZ1 file system with five 2TB disks. At the time I knew I would run out of storage and planned to replace the 2TB drives with 4TB drives to double the capacity. This file server supports NFS and CIFS shares and has a rather simple file structure: 5 directories below the root of the file system, then each of those has from 2 to 200 directories below that, then each of those directories house files. In all, right now, about 14000 files. But, after reading forums discussing how long it takes to resilver just a 500MB drive (and running on a degraded file system while each drive is resilvered,) I began to think about potential consequences of having a degraded system for as long as a month. I add about 100GB each month too.

So I'm looking at an alternative of adding an eSATA port multiplied "toaster," populating it with 5 more 2TB drives (1TB is the "sweet spot" in the marketplace right now. I expect 2TB drives to get to that point within 6 months unless Thailand experiences a severe monsoon season again this year) and creating a new pool. Going this route, I'd like to name the first level directories in the new pool with the same names as are found in the original pool.

If I go that route, to make share access transparent to users on machines mounting via NFS or sharing" via CIFS can I simply do that by creating soft symlinks at the highest level?

I manage the (headless) server, and do my writes from a desktop/workstation running linux. but wonder about the pros and cons of mounting. Thinking about it, I could create symlinks on the linux workstation and mount the two pools there or I could create symlinks on the FreeNAS server and mount one "common" name depending on which way I write the syslink command. Thinking this through further, I'm concerned my plan might not work. When I write then with symlinks I then wouldn't have control over the physical pool new files would be written to. Now, as I understand, this approach works under growing an LV, but I don't have an LV file system here.

I'm open to suggestions of other ways to accomplish this task of growing my storage as well as practical comments about how to implement what I've suggested.

Thank you for your comments in advance.

Server:
HP Proliant N40L
FreeNAS 9.1(RC1), RAIDZ1 (8TB)
Proposed espansion, Sans Digital TR5M, RAIDZ1 (8TB)

Workstation:
Homebuilt, Ubuntu 12.04 (LTS)
5 Various Hard drives: Ext2, Ext3, NTFS

Supported devices:
SamsungTV (QNX?)
Samsung BlueRay player (QNX?)
Windows XP
Windows7
iPhone
iTouch
HP Touchpad (WebOS & Android CryogenMod 10.2 (ICS4.1))

 

[cyberjock]

What you really should do is looking to go with RAIDZ2. RAIDZ1/RAID5 died long ago and shouldn't be considered "safe".

Also, port multipliers are bad joo-joo for file servers. They just aren't reliable enough to use for such an important task. Plenty have lost data with port multipliers.

It really looks like you may have outgrown the N40L. :(

If you still insist on another RAIDZ1 via port multipliers then you should just add the new drives as another vdev. Your pool will expand to use the new drives and you won't have to play with symlinks or anything.

 

[SmilingInSeattle]

Thanks for your quick reply! You gave me just enough time to ear my supper!

I guess I'm sort of stuck with the hardware as the SanDisk Tower Raid was delivered as you were probably reading my message. I'd bought it while still searching for an answer as NewEgg had an "open box" at a price I couldn't pass up. What I received was NOT an open box, just a dusty one. The host computer Raid card that comes with this hardware does not have good reviews; however; I intended to use the N40L's controller in AHCI mode via the remaining eSATA port from the onset. I've been using the N40L modded BIOS for over a year with no events. Whereas my Linux box has been rebooted at least 10 times during that time, the only time this FreeNAS server has been rebooted was to upgrade from 8.2 to 9.1, a testament to the stability of FreeNAS (stability and transparency were prime considerations when I choose ZFS and FreeNAS over a Linux (RAID) solution. Of course, since the time I made the decision, Linux now supports what is reported to be a pretty good implementation of ZFS, but the decision has been made, and I'm still happy with it.

The trade off has been learning new CLI commands for things I know how to do in another OS. Who'da thought that camcontrol does a lot of what parted does. No similarity in the names at all! Before I retired my office used Solaris and HPUX, but, since we had a Systems Department commands that controlled hardware were not privy to users (except back in the days when we all had to load out 7 track backup tapes on the HPUX mainframe or ask the SYSAdmin for a larger allocation for some specified time.

So now, I'll take the time understand ZFS terminology better. I appreciate your candid comments about RaidZ1, I'll look at the work to change to RAIDZ2. I went RAIDZ1 in the first place because I wanted some redundancy in case of a drive failure (I couldn't afford enterprise grade HDs and ended up with Seagate ST2000DL001 (5900 rpm) models. The N40L only supports 3GB/sec transfer rates - and my users have never experienced read problems (which is all they do) with these drives. I'll probably populate this new box with more of the same. I've still got 6 months or more for the price to fall to less than $75 for new drives. I've looked into a lot of file systems. At this time I like ZFS. I tried Reiser. I think Journaling is an important function. BTRFS looks promising, but it's still really short on recovery and maintenance tools. There still is no perfect file system. Chosing one is all about priorities and weighing tradeoffs.

To ally you fears of failure, I back up everything twice onto offline storage, and there's also shared cloud storage. So a potential catastrophic failure primarily is dealing with the personal frustration of babysitting a restoration (which would include the old fashioned "respond to a mount request from a user" mounting of files that are immediately needed. Perhaps a bigger frustration will be the initial diagnosis of what went wrong and what is really damaged - fix the problem, not just the symptoms!

 

[gpsguy]

You may find yourself short on RAM, after you add the extra drives. Though the doc's say the N40L will only take a maximum of 8Gb of RAM, you can put 16Gb in it. The RAM listed below is known to work.

Kingston 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333
ECC Unbuffered Server Memory Model KVR1333D3E9SK2/16G
Item #: N82E16820139979 [NewEgg]

 

[SmilingInSeattle]

After reading cyberjock's presentation (slideshow) giving technical details, I agree completely and I'll be looking for some. I'd seen posts, particularly for the N54L that say while HP doesn't officially support 16GB of RAM, it is recognized with no problem. There's fewer posts about the N40L, but since the only real difference between the two models (other than the price) is a faster CPU I can't see any problem. Besides, if it doesn't work, I'll find another machine to put it in, and then I'll probably have to go to plan #2 and build a second NAS. I figure I'm saving just a couple hundred bucks going this route over upgrading the existing pool to 4TB disks, but I'm really concerned about expected long resilvering times as I grow the five disks. )See, I'd recycle the 2TB drives into my offline backup pool, which, in reality, I don't like as it takes a long time to write a full 2TB disk - Deliberately, I currently purchase 500GB drives for backups even though they are not the best $/GB ratio any more,) but after reading technical details in the presentation, I think I would have to swap my 8GB of RAM for 16GB anyway after upgrading to 5 4TB drives in a RAIDZ1 configuration, and, then, I certainly would have to look into converting the RAIDZ1 into RAIDZ2 (wishing I could convert to RAIDZ3 but lacking the SATA ports to do so, making it impractical.)

I'd read similar articles to the RAID5.RAIDZ1 is dead back in 2009 and weighed that information back then, which is why I back up everything (on 500GB NTFS disks because NTFS is an almost universal exchange format (of course, rumor has it that MS is working on moving off NTFS to a more advanced standard, but by the time NTFS becomes obsolete/unsupported it will be long past my life expectancy!

 

[cyberjock]

Yeah. You've kind of noticed that you're somewhat at the limit of your hardware. You want more disk space, and suddenly you are buying more RAM, then you might need more CPU. That's why I tend to do foward-planning with server builds, especially RAM for FreeNAS. You don't need an overly powerful CPU, but RAM will almost certainly be a limiting factor as drives scale up over the next 3-5 years(the expeced lifespan of my server before I want to build a new one).

The HP microservers works great for small servers, but as soon as you start saying thing like "upgrade to bigger drives" and "want more performance" you've guaranteed that you'll outgrow the HP microserver within a year or two. I prefer to build it once and be done with it, so I went with a Xeon e2-1230v2 and 32GB of RAM.

 

[SmilingInSeattle]

Back in 20082009 when I finally, realized, no, admitted, my full sized tower box, at that time with 6 drives in it, was getting too hot, and I had to get bigger ones, and commercial NAS boxes came out, I started researching the possibility of building my own. I've been building my own computers since back in the days of HeathKits, and before that, calculators with discreet components. Putting the parts together, it was hard using components to beat the ready built prices of a Thecus or Drobo, but I was determined as buying a ready built box was just no fun. The N40L was prebuilt, but I couldn't pass up the sale price of $199 at the time. I then spent a lot of time watching disk prices as they were quite cheap when I was procrastinating, and then the floods came in Thailand with the WD and Seagate drive facilities getting flooded out.

My wife, being Korean got me interested in Korean dramas. I found there was a volunteer community that subbed them in English and I started collecting. Back in High School I found World History to be fascinating even though by nature I'm a hard science guy: Mathematics, Physics, Chemistry, Engineering, but I my classes focused on European and Mediterranean history. The korean historical dramas made learning Asian history fun. Yeah, there are four hours of embellishment to one sentence from historical documents, but I find I have a much better understanding about life outside of Democracy. So I now have about 200 televised dramas ranging from 16 one hour sessions to a 200 hour session. As a collector I belong to groups dedicated to keeping these dramas available to others that may stumble upon them. My wife, being an insomniac can't get enough from what is currently produced so she re watches some of them. And now one of my daughters has been hooked. But my collection has really grown since I got the NAS. When I first loaded it I had 1.4TB. I now have 5.2TB and it grows by about 180GB/week. When bought the Proliant, I didn't think I'd really ever fill it up. How wrong I was, no, am!

If I build a new server I think I'll look into blades and a rack, but low power consumption will become the priority!rewatches

 

[cyberjock]

Honestly, I think that my config is pretty amazing.

I have a 24 drive 4U case with M1015 and Intel SAS expander to provide all 24 ports.

E-1230v2 with 32GB of RAM on a Supermicro X9SCM-F-O. It ran at like 36w idle and has plenty of power and RAM for years to come. IPMI was a big plus for me too. :)

 

[gpsguy]

I have both a N40L and N54L. In the latter (ESXi), I've got the aforementioned 16Gb RAM. In my 40, I (only) have 8Gb.

I'd seen posts, particularly for the N54L that say while HP doesn't officially support 16GB of RAM, it is recognized with no problem. There's fewer posts about the N40,,,