New user on FreeNAS - Need advice

[KernnK]

Hello !

I'm creating my firt FreeNAS server and would like many advice.

I'm using a PowerEdge R510 with 16GB of RAM, a processor E5620 (12M cache, 2.40Ghz) a expandable RAID controller, and 4 WD 1TB + 4WD 2TB for storage.

I would use 2 mirrored bootable usb-stick (16GB) for FreeNAS system.

My goal is redudancy (one disk failure by RAID/RAIDZ) and performance : my FreeNas server will be connected to ESXi and configured for iSCSI.

I read that :

・ the ZFS filesystem for the storage and iSCSI causes many fragmentation issue. (It is recommended to keep the used space of the pool below 50% )
・ using ZFS with a RAIDZ is not recommended with a RAID controller.
・ FreenNas 9.3 support only ZFS.

After rading this, my initial solution, seem like a bad idea : FreeNas 9.3 with file system storage ZFS and 2 RAIDZ1 of about 3TB and about 6TB (with the raid controller in JBOD mode ).

Should i use FreeNas 9.2 with file system storage UFS and 2 RAID5 of 3TB and 6TB ? Do you think that this solution offers good compromise redudancy / performance ?

Thanks for your answers !

 

[depasseg]

Feel free to use 9.2 or 9.3, HW RAID or JBOD mode, ZFS or UFS, and fill it up to 99.9999%. The recommendations are there for a reason, and that reason is that the non-recommended configurations have had issues where a user has been unable to recover data.

Oh, and for an ESXi datastore, you probably want more than 1 vdev, or your latency will be high. 1 RAID-Z vdev will give you the same IOPS as the slowest drive in it.

 

[KernnK]

Hi Depasseg,

Thanks for your answer. I understand that recommendations are there for a good reason. ;)

But do you think that take the choice of an old version of FreeNas (9.2) with UFS and a HW RAID for storage (1 RAID5 of 4 disk 1TB, and 1 RAID5 of 4 disk 2TB ) for my goal cause others issues ? I'm newbie on FreeNas and I try to avoid the traps.

Do you have any recommendations for use correctly FreeNas like an ESXi datastore (in iSCSI) ?

Thanks by advance.

 

[depasseg]

I try to avoid the traps.

By going back to 9.2 or using UFS, you aren't avoiding traps. You are disabling safeguards. The traps are still there.

Get a cheap HBA (M1015 or similar) and use striped mirrors and you should be good.

 

[Mirfster]

But do you think that take the choice of an old version of FreeNas (9.2) with UFS and a HW RAID for storage (1 RAID5 of 4 disk 1TB, and 1 RAID5 of 4 disk 2TB ) for my goal cause others issues ?

Bit confused OP, are you stating you want to us HW Raid with FreeNas (9.2) or am I reading this incorrectly?

 

[jgreco]

The problem with using UFS and 9.2 is that the iSCSI will be slowish, it can't take advantage of any of the ZFS caching benefits.

iSCSI and ZFS will do better if you can dredge up some more memory. Either way, be sure to use mirrors, not RAIDZ, unless you like slow, and as noted elsewhere, an HBA instead of the RAID controller.

 

[SweetAndLow]

If using iscsi you will want 9.3 because iscsi was improved. You will also want a hba, mirrored vdevs and a good amount of memory.

 

[KernnK]

Hi everybody,

Thanks for all your answers.

By going back to 9.2 or using UFS, you aren't avoiding traps. You are disabling safeguards. The traps are still there.
Get a cheap HBA (M1015 or similar) and use striped mirrors and you should be good. :)

Ok, thanks for this alternative.

My DELL PowerEdge R510 have a RAID controller PERC 6/i.

If i understand correctly, for my solution, use an HBA like M1015 offers betters performance that the initial RAID controller ? It male a really big difference ?

You suggest to use stipped mirrors and it's a good idea but i am limited by the capacity storage of my disk : 3TB and 6TB are the minimal i must have.:/

Bit confused OP, are you stating you want to us HW Raid with FreeNas (9.2) or am I reading this incorrectly?

Hi,

Yes, that's right.

The problem with using UFS and 9.2 is that the iSCSI will be slowish, it can't take advantage of any of the ZFS caching benefits.
iSCSI and ZFS will do better if you can dredge up some more memory. Either way, be sure to use mirrors, not RAIDZ, unless you like slow, and as noted elsewhere, an HBA instead of the RAID controller.

Hi,

Thanks for your answer.

Like i have said, i am limited by the capacity of storage of disks. For this reason i take the choice of RAIDZ or RAID5.

To you, if i understand, the better solution with My DELL PowerEdge R510 is use FreeNas 9.3 with a better capacity of memory, and ZFS ? For you, how many memory are necessary to have good performance ?

Or, alternative solution if i cant's add more memory : i must use striped mirrors and an HBA instead of the RAID controller ?

I've correctly understand ?

If using iscsi you will want 9.3 because iscsi was improved. You will also want a hba, mirrored vdevs and a good amount of memory.

Hi,

Thanks for your answer.

For you, how many memory are necessary ?

About fragmentation problems with iSCSI and ZFS, i cant keep the used space below 50%.

To summarize:

If i take the choice of FreeNas 9.2 and UFS the iSCSI will be slowish.
If i take the choice of FreeNas 9.3 and ZFS + iSCSI, i go have fragmentation problems.

In all case, i must increase memory of my server or make the choice use an HBA in place of Raid Controller.

It's right ?

 

[depasseg]

To summarize:

If i take the choice of FreeNas 9.2 and UFS the iSCSI will be slowish.
If i take the choice of FreeNas 9.3 and ZFS + iSCSI, i go have fragmentation problems.

In all case, i must increase memory of my server or make the choice use an HBA in place of Raid Controller.

It's right ?

Not quite.
The slowish comes from your choice of vdev. If you go RAIDZ it will be slower than striped mirrors, no matter what version or HBA you use.
The fragmentation comes from the iSCSI, regardless of version. iSCSI on 9.3 is faster than on 9.2.
If you use a JBOD mode (and not IT mode) on the integrated HBA, you will likely not be able to run SMART diagnostics (which are recommended), and you might not be able to mount your pool on any other HBA in the future. (The amount of RAM has nothing to do with this.)

 

[KernnK]

Thanks you very much for this explanation. I will try and i going back if i has news questions. ;)

 

[Mirfster]

Bit confused OP, are you stating you want to us HW Raid with FreeNas (9.2) or am I reading this incorrectly?

Hi,

Yes, that's right.

• Can't use Hardware Raid, suggest you invest in a HBA.
• Can't CrossFlash a Perc6/i to IT Mode
• PERC 6/i can't see over 2.2 TB per drive (or something similar, can't recall exactly)
• PERC 6/i doesn't have a JBOD mode

If I am incorrect on this, please feel free to let me know...

 

[KernnK]

No, you have totaly right.

I have just discover this afternoon that i can't use JBOD mode with a Raid Controller PERC 6/i. ( Like i have said, i m a newbie :/ )

I look if i can add a HBA card like M1015, in my PowerEdge R510. Or I should find an other solution. :(

 

[jgreco]

Dell makes HBA cards as well. If you pick up a Dell card, note that you still need to flash it to IT mode.

 

[Mirfster]

M1015 seems to the most recommended around here. If you want to stay with Dell, you can get a H200 (I use this in my Dell C2100/FS12-TY Servers; but is the "Mezzanine" model) or H310. Others go straight for the LSI itself.

Check out jgreco's "Confused about that LSI card? Join the crowd ..." thread for some more info

 

[jgreco]

The Dell card does seem to be more of a PITA to crossflash, by the way. The LSI is the most expensive option, but of course it is easiest to work with because you're buying a product designed to do exactly what we want.

 

[Mirfster]

The Dell card does seem to be more of a PITA to crossflash, by the way

Only for some... :P

My notes:
To just flash it to IT mode without the BIOS:

1) Put the "sas2flsh" into "Advanced Mode" and Erase Existing Flash
Command: sas2flsh -o -e 6
2) Flash H200 to IT Mode:
Command: sas2flsh -o -f 2118it.bin
3) Verify Command: sas2flsh -listall

To flash it to IT mode with the BIOS:

1) Put the "sas2flsh" into "Advanced Mode" and Erase Existing Flash
Command: sas2flsh -o -e 6
2) Flash H200 to IT Mode:
Command: sas2flsh -o -f 2118it.bin -b mptsas2.rom

*** The 2118IR.bin is also here for the "IR" mode...

Reference: http://brycv.com/blog/2012/flashing-it-firmware-to-lsi-sas9211-8i/
Additional Reference: http://pastebin.com/YicqtWb9

 

[Robert Trevellyan]

Can't CrossFlash a Perc6/i to IT Mode

Incorrect, but moot, because ...

PERC 6/i can't see over 2.2 TB per drive (or something similar, can't recall exactly)

Correct.

 

[Mirfster]

Incorrect, but moot, because ...

Really? Hmm, might have to grab one out of the bin and give it a whirl just for kicks...

Sent from my ASUS Transformer Pad TF700T using Tapatalk

 

[Robert Trevellyan]

Really? Hmm, might have to grab one out of the bin and give it a whirl just for kicks...

If I'm correct, it has an LSI 1068E chipset, which can easily be flashed to IT mode. I mean, it must be easy if I was able to do it

But I could be wrong about the chipset, because mine is actually a PERC SAS 6/iR, which came with a used T110 I bought last year. I flashed it to IT mode and ran FreeNAS on it for a while with no issues.

 

[Mirfster]

If I'm correct, it has an LSI 1068E chipset, which can easily be flashed to IT mode. I mean, it must be easy if I was able to do it

But I could be wrong about the chipset, because mine is actually a PERC SAS 6/iR, which came with a used T110 I bought last year. I flashed it to IT mode and ran FreeNAS on it for a while with no issues.

Ah, no worries. Was just thinking that I would have to be really desperate to try and use one now anyways. Doubt I will ever get to trying it out since it lacks support for larger drives.

Sent from my ASUS Transformer Pad TF700T using Tapatalk