Looking for recommendation on multi-HDD external enclosure

[ovig]

Hi guys,

I have a HP MicrosServer running FreeNAS 9.3 very happily... trouble is that I am running out of space on my current pool and have no room to add drives inside the server; I'd need to add an extra 2 discs minimum (for redundancy) but would prefer to have room for 4 or more. USB is a no-no, so eSATA is probably the way to go. And budget is limited (the MicroServer is used for long-term storage to space is much more important than speed). The pool is 4x2TB as RAID-Z2 so I guess I could replace in turn two of the discs by e.g. 4 or 5TB HDDs which would give me another 2 or 3TB of space on the pool; but then I'd end up with 2x2TB discs doing nothing, so I'd rather "add" than "replace".

Having thought about it a bit, and trawled the forum, it seems to me that the best way forward would be to add an IBM M1015 / LSI adapter and connect an external eSATA enclosure with the extra discs. Two minutes of Googling bring back hundreds of 4- 5- or 8-bay enclosures, but they pretty much all seem to have some sort of RAID support and a single eSATA port which may or may not work with the M1015, so:
- is there another option (maybe firewire?)
- does anyone know of an X-bay, X-ports enclosure, thus guaranteeing that I can access the discs as JBOD?
- if not, can someone recommend an enclosure (X-bay but with only one eSATA port at the back) that is known to work with an IBM/LSI adapter?

Thanks in advance,

Olivier

 

[Nick2253]

What you're talking about is a sata port multiplier. Port multipliers have a storied history in storage circles, and they are the bane of any storage administrator worth his salt. They create tons of problems, mostly because they have huge latency and bandwidth issues. Personally, I would not recommend a port-multiplier solution, even for a home setup.

If you still want to go eSATA, I'd search google for "freenas port multiplier" and "freenas esata", and you might find someone who has got this to work. That would possibly give you some direction on enclosure suggestion.

The "enterprise" way to do this is a fiber channel, which is probably way over priced for a home setup.

EDIT:

On further reflection, I'd skip the eSATA and use a SAS expander. You'll have to do some work figuring out what cards will work, but there are a bunch of JBOD SAS expander enclosures available.

 

[ovig]

I was so close - found an enclosure for 4 disks with 4 eSATA ports - no expander in sight, and a very definite 1-disk/1-port setup (http://www.pc-pitstop.com/sata_enclosures/scsata4et.asp)... but whilst they ship worldwide the prices are ludicrous - $180 to ship a $220 enclosure to Great Britain :-(
I guess this would be a viable option for someone in the US

From what I've found so far, SAS enclosures are quite expensive too, probably as they are targeted at the enterprise.

Anyway - the search goes on; thanks for the pointers, Nick.

 

[bestboy]

I'm using an IcyBox IB-544SSK for offsite backup. Works ok.

http://www.raidsonic.de/products/internal_cases/backplanes/index_en.php?we_objectID=1164
http://www.amazon.co.uk/Icy-Box-IB-544SSK-4-Bay-Channel/dp/B006BQYQXE

 

[Arwen]

...
The pool is 4x2TB as RAID-Z2 so I guess I could replace in turn two of the discs by e.g. 4 or 5TB HDDs which would give me another 2 or 3TB of space on the pool; but then I'd end up with 2x2TB discs doing nothing, so I'd rather "add" than "replace".
...

If I understand what you describe above, it won't get you what you want. Basically a RAID-Zx, (1, 2 or 3), VDEV
is going to use the size of the smallest disk, for all disks. Thus, a RAID-Z2 with 2 x 2TB and 2 x 4TB gets you no
additional space. You have to replace ALL the disks to grow a RAID-Z2 VDEV.

Next, adding disks to an existing Pool takes a bit of planning. It's is highly recommended if you add a VDEV to
an existing Pool, it should be the same type. In your case, a 4 disk, RAID-Z2. Preferably even the same sized
disks as existing. Though you probably can get away with larger disks, (like 3TB or 4TB), as long as you
understand speed may get irregular when the Pool gets full.

 

[solarisguy]

@ovig, consider replacing all four of your disks with 6TB ones.

Simple solutions are often the best ones. You can reuse 2TB disks for backups.

 

[gpsguy]

On another note, which Microserver do you have? While the official documentation for the NxxL says it only supports up to 8GB RAM, one can put 16GB in them. HP's docs say the Gen 8 only supports 16GB. I'm not sure if one can exceed that number or not.

That being said, RAM might be a constraining factor, if you want to more and/or larger drives.

 

[ovig]

@bestboy:
that looks good: two quick questions, though - what is the IcyBox sitting in? and they are marketed as dual-channel enclosure - to me that says "one SATA/SAS port for two disks" but there are 4 ports at the back - are yo running them with 4x HDDs and 4 cables going one ea. from a disk to your controller?

@gpsguy:
N36L with 8GB - works fine at present; and I'm working on the "1GB per 1TB" rule so am not worried about adding another 4TB vdev to the existing one, but I might be wrong - tbh the aim is to get as cheaply as possible over the next 6 or 12 months and then replace the whole thing by a larger/more expandable one

@solarisguy:
I'm looking for a cheap upgrade at the moment, and even though the numbers might prove me wrong, it just "feels" like replacing rather than adding will be complicated; 6TB discs are still quite expensive in the UK (best $$/TB I found was 5TB at present) & I haven't got an easy way of adding the new disks temporarily for migrating the data;

 

[solarisguy]

[...] @solarisguy: [...] it just "feels" like replacing rather than adding will be complicated; 6TB discs are still quite expensive in the UK (best $$/TB I found was 5TB at present) & I haven't got an easy way of adding the new disks temporarily for migrating the data;

• I know that feeling, that is why I had offered my advice :)
  
• By only looking at £/TB, you are not looking at your total cost. If you buy 6TB ones, then you can defer buying new disks...
• 1. If you are replacing, then you do not need any extra anything. 2. If you can borrow a PCIe card with SATA ports, you can have the additional disks outside of your PC for the duration of copy. I do not mean on the floor or just wherever, but the issue is more with enough of SATA ports than with the place for disks (e.g. they could be temporarily in another PC case that would at least be responsible for the air flow).

 

[gpsguy]

Some users with the NxxL series have put 6 drives in their server. There used to be a device, I believe it went by the name "X-wing", that allowed one to mount 2x3.5" drives in a 5.25" bay (for example your ODD bay). The closest thing I can find now, is this - Akust 5.25" Bay to 3.5" HDD Adapter

One issue, is that the ODD and eSATA connections appear as an IDE. In order to change them to AHCI, you'd need to load a hacked BIOS on your server. You can search the 'net for more information.


Another possibility would be to put something like this 4 bay 2.5" Icy Dock in your ODD bay and connect it to an HBA, like the M1015. You would be limited to 2.5" HDD's. @jgreco did something like akin to this with his N36L for a different reason, but the end result is similar.

 

[jgreco]

Since it's not yet possible to obtain a drive larger than 2TB in the 2.5 form factor, a nice ServeRAID BR10i may be up to the task, if you can find one substantially cheaper than the M1015.

 

[bestboy]

@bestboy:
that looks good: two quick questions, though - what is the IcyBox sitting in? and they are marketed as dual-channel enclosure - to me that says "one SATA/SAS port for two disks" but there are 4 ports at the back - are yo running them with 4x HDDs and 4 cables going one ea. from a disk to your controller?

Well, the enclosure is supposed to be used internally, but I'm not using it this way. I'm using it externally. It has a very massive and sturdy metal body and can be locked with a key. As I've mentioned I'm using it for offsite backup. What I do is, from time to time I bring it in from where I keep it, hook it up, create manual snapshots of the data and update the encrypted pools by sending the incremental snapshots.
Each of the 4 drive bays in the enclosure has a SATA and a SAS port, which I guess is why they marketed it as dual channel. I only use the SATA ports, tho. My setup looks like this:
I have 2 eSATA ports which I created with a bracket externalizing otherwise unused SATA ports of my mainboard (got a M1015). I hook up 2 drives of the enclosure via the SATA ports to the 2 ports of the bracket via a 50 cm SATA-to-eSATA cable. For power I'm using an external power brick with a molex connector.

Some random points about this setup, which works ok for my use case of an occasional backup:

• The main problem imho is the cabling:
  • The SATA connectors on the enclosure side can become loose easily. There is no mechanical grip like on the eSATA side of the connection.
    
  • It's not really comfortable running external power and such short SATA cables (see next point).
• the total SATA cabling length (internal + external) should not exceed 1 m as per SATA specification
  
• there are also eSATA brackets that externalize power. I did not go for such a model, as my internal power supply is already at its limit
• I have 2 independent pools consisting of 2 drives each in the enclosure, but it should be possible to hook up all 4 drives into a single pool when 2 eSATA brackets are used.
• hot plug works fine. When I hook up the enclosure, FreeNAS promptly finds the newly added drives and I can decrypt the pool from the WebUI.
• When you run shabby external power like I do, make sure to use disks with benign spin-up currents. Read this and this. (I use WD Greens)