expanding controllers

[creepwood]

Long before I run out of space I'd like to know where I'm heading when I do. I have no more vacant onboard SATA-ports on my MOBO so I'll have to expand with IO-card. I need atleast a card that can handle 4 drives.

From what I understand you can use a SAS-card to control standard SATA-drives? Are they too high end? I'm not a wealthy person, got 6 more years of schooling. What should I do? And where can I find if a chipset is actually supported?

 

[Todd]

This is what we're trying to do with our inital build. We bought an Adaptec RAID 6805 which was really pricey but supports something like 256 drives (using SAS expanders). Not sure how the heck you would make more than 20 drives work efficiently without maxing out the PCIe bus but I suppose it's technically possible.

We're struggling to get our Adaptec working right now but we're hell bent on making FreeNAS work.

 

[freeflow]

Dell in the UK are selling thier SAS6 board for around 52 ukp. It requires a 8 lane PCI-e slot (I use a PCI graphics card so have it in the main 16 lane PCI-e slot). It plugged in and worked fine with FreeNAS 8 beta 4 without any configuration needed. This card supports 8 sata disks. You will need to also but 2 SAS to 4 sata connectors. I got these from ebay (china) for around £5ukp each. It did take nearly 23 working days for them to arrive.

http://accessories.euro.dell.com/sn...S&cid=41147&lid=1069636&acd=23975883824506333

 

[matthewowen01]

i have an 8 port SAS card that i use with sata drives. works great.

 

[louisk]

ZFS is capable of upgrading the volume (pool) once you replace existing spindles with larger spindles. For example, you should be able to replace all 4 2T spindles with 3T spindles and FreeNAS should see the new space. Of course, you have to do them one at a time and resilver in between, which, for 2T will take a little while. I don't recall if FreeNAS has implemented this functionality in the GUI or if you would have to do it on the CLI.

 

[matthewowen01]

ZFS is capable of upgrading the volume (pool) once you replace existing spindles with larger spindles. For example, you should be able to replace all 4 2T spindles with 3T spindles and FreeNAS should see the new space. Of course, you have to do them one at a time and resilver in between, which, for 2T will take a little while. I don't recall if FreeNAS has implemented this functionality in the GUI or if you would have to do it on the CLI.

False. Newer versions of ZFS can, not the one that is used by FreeNAS. i've done it under my solaris box but i tried doing it under FreeNAS then realized that the property 'autoexpand' does not exist.

 

[cypher1024]

There are some really cheap (~$10) Dell SAS cards on ebay at the moment. From what I understand, they're basically a cut-down version of the PERC 5/i.
http://www.ebay.com/itm/DELL-MY412-...753?pt=LH_DefaultDomain_0&hash=item4aa73c15f1

The tricky thing is to find the correct SAS -> SATA breakout cable, and figure out some sort of a bracket (mine's being held up by a cable tie until I sort out something more permanent).
The cable should look like this:
http://www.ebay.com/itm/New-32-Pin-...090?pt=LH_DefaultDomain_0&hash=item1e647c4aca

I can confirm that these cards work both with FreeNAS 0.7.2 and 8.0.1 Beta 4.
Also, if you're running low on large PCI-e slots, you can do the PCI-e x1 slot mod, or use one of these adapters to run it in a x1 slot:
http://www.ebay.com/itm/PCI-Express...548?pt=LH_DefaultDomain_0&hash=item27bd165b5c

False. Newer versions of ZFS can, not the one that is used by FreeNAS. i've done it under my solaris box but i tried doing it under FreeNAS then realized that the property 'autoexpand' does not exist.

I think you're talking about something else. louisk is talking about swapping out all of the disks in a vdev for larger items (with a replace/resilver in between), and then having extra space (which FreeNAS is perfectly capable of doing, and has been since ZFS was first implemented in FreeNAS).

What I think you may be referring to is expanding the width of a vdev?

 

[matthewowen01]

I think you're talking about something else. louisk is talking about swapping out all of the disks in a vdev for larger items (with a replace/resilver in between), and then having extra space (which FreeNAS is perfectly capable of doing, and has been since ZFS was first implemented in FreeNAS).

What I think you may be referring to is expanding the width of a vdev?

Please give it a try and let me know how that turns out. I've done it successfully under opensolaris (when i ran it) but can't under Freenas because the version of ZFS is too old. I'd like to be wrong on this but I'm not.... Totally am wrong....

http://forums.freebsd.org/showthread.php?t=10048

 

[cypher1024]

Please give it a try and let me know how that turns out. I've done it successfully under opensolaris (when i ran it) but can't under Freenas because the version of ZFS is too old. I'd like to be wrong on this but I'm not.

I'm really not sure what the autoexpand property is, but I can guarantee that I've increased my pool size by replacing disks one by one with a resilver in between. In fact, I've just finished swapping out the last 1.5TB drives in my main pool for 2TB items, taking the total usable space from ~8TiB to almost 11TiB. The only quirk is that you have to reboot the machine after swapping out the last disk. Until you do the reboot, the old figures for used, free & total space are shown.

I haven't tested this on FreeNAS 8, (my main machine is running 0.7.2.5543 until 8.0.1 Release is out), but I'd be more than happy to test it in a virtual machine if you'd like.

Edit: I was curious so I tried it anyway! The test was run on a virtual machine with 8.0.1. Beta 4 installed. I set up a 3-disk RAID-Z vdev with 4GB disks, yielding an initial total of ~4GiB of usable space. These were then replaced, one by one with 8GB items with a resilver after each.

Pool after setup:

First disk replaced:

After all three disks were replaced, the total space was still the same as when I began:

After a reboot, the new space was available:

If you'd like me to test a different configuration, I can do that too.

Edit #2: I should probably clarify that 8.0.1B4 was the first time I was able to do this sort of upgrade through the WebUI. In past versions of FreeNAS (i.e. 0.7.x), I've had to do this through the command line.

 

[creepwood]

There are some really cheap (~$10) Dell SAS cards on ebay at the moment. From what I understand, they're basically a cut-down version of the PERC 5/i.
http://www.ebay.com/itm/DELL-MY412-...753?pt=LH_DefaultDomain_0&hash=item4aa73c15f1

Oh from what I can make of it (not alot) This is what I'm looking for, but it says it's just compatible with "Compatible with PowerEdge 1950, PowerEdge 2900, PowerEdge 2950 servers and PowerVault NX1950. compatible with SATA & SAS hard drives." Does this mean that it's guarenteed to work with those or will it in reality work with any PC?

I'm not located in America so the shipping might be too high but definetely something to look at. :)

 

[matthewowen01]

Edit: I was curious so I tried it anyway! The test was run on a virtual machine with 8.0.1. Beta 4 installed. I set up a 3-disk RAID-Z vdev with 4GB disks, yielding an initial total of ~4GiB of usable space. These were then replaced, one by one with 8GB items with a resilver after each.

Fantastic, i love being wrong because it means i learn something. Now it's figuring out why i was wrong. so i did some more research and the autoexpand property that was added in a newer release allows for vdevs to expand while live, ie, storage that can never go down can be expanded. Good work, please post a guide when you have some time. i have to redact my redactions.

 

[cypher1024]

Oh from what I can make of it (not alot) This is what I'm looking for, but it says it's just compatible with "Compatible with PowerEdge 1950, PowerEdge 2900, PowerEdge 2950 servers and PowerVault NX1950. compatible with SATA & SAS hard drives." Does this mean that it's guarenteed to work with those or will it in reality work with any PC?

I'm not located in America so the shipping might be too high but definetely something to look at. :)

I'm in Australia, so shipping was slightly more expensive for me too, but it was still very cheap.

Certain motherboards require a slight modification to the card to get them working. It just involves covering or disconnecting two pins.
A list of compatible motherboards and some other tips (including the pin mod) can be found here:
http://forums.overclockers.com.au/showthread.php?t=879827

Both motherboards that I've tried mine in didn't require the mod.

Some people have added active cooling to their cards. The heatsink does get reasonably hot under heavy load (e.g. resilvering a large pool), but with normal use I think it's fine to stick with the passive heatsink.

Fantastic, i love being wrong because it means i learn something. Now it's figuring out why i was wrong. so i did some more research and the autoexpand property that was added in a newer release allows for vdevs to expand while live, ie, storage that can never go down can be expanded. Good work, please post a guide when you have some time. i have to redact my redactions.

That's excellent news! Well, maybe not so excellent for my bank balance because now I'll have to buy a case with hot-swap bays!

 

[matthewowen01]

That's excellent news! Well, maybe not so excellent for my bank balance because now I'll have to buy a case with hot-swap bays!

tell your bank account I'm sorry but i have a Norco and i Love it. 20 hot swap bays, they have one with 24. mine was about $350 but well worth it, cheep fans though i swapped them out for quieter ones.

http://www.newegg.com/Store/BrandSu...12&name=Norco-Technologies-Inc-Server-Chassis

 

[cypher1024]

You're a bad man! I've come very close to ordering an RPC-4220 a few times in the past.
I guess I should start shopping for SFF-8484 to SFF-8087 cables!

 

[cypher1024]

Anyone who has the Dell SAS 5 cards mentioned earlier in this thread might be interested in the instructions I've just posted on making brackets to suit them: http://forums.freenas.org/showthread.php?2519-Dell-SAS-5-Bracket-Fabrication