Question regarding replacing a PERC H700 in an R510

[sunspot]

Hello,

I recently bought a used Dell Poweredge R510 which came equipped with a PERC H700. I have never seen a controller that couldn't do pass-through or JBOD, so needless to say I'm pretty bummed with what I ended up with. I'm aware that the instructions here state to not get an H700, I found these instructions after the fact.

My question is, what's a good controller to replace this with? I was reading that an H200 would be a good solution, but the post was dated and I was wondering if there would be a more modern, speedier solution.

Also, I want to ensure that the H200, or whatever is (hopefully) recommended, will in fact fit and work with an R510.

Lastly, some of the disks I wish to use do not have TLER. Which has never been a problem for me in a JBOD scenario, I want to also make sure that whatever controller I get will not cause issues if this feature isn't available. I assume that if it does passthrough fine then it shouldn't be a problem but you guys would know better than I if there's some hiccup I'm not aware of here.

EDIT: Not sure if this is relevant but the existing card can/is currently supporting 14 SAS/SATA drives. So the new one would need at least that much capability as well. I'm installing two SSD's in RAID0 for the OS, so ideally that would be supported at maximum throughput.

Thanks in advance,

Sunspot

 

[Chris Moore]

The Dell H310 controller can be flashed to IT mode and I am using them in all my FreeNAS builds. There are some newer cards, but it depends what disks you are using if there is any use in getting them. The only way you actually need a newer card is if you are using SAS 12GB/s disks.

 

[sunspot]

A 12GB is definitely a possibility but it won't (likely) ever be a SAS drive but just a plain SATA like a WD Gold or something. Would that work?

Can I ask what IT mode is? And where do you get the tools to flash it/etc?

 

[sunspot]

This says that it's only 8 port? I assume that means it will only handle 8 drives? Or am I mistaken? My current setup has 14 drives.

https://www.ebay.com/itm/New-Dell-PERC-H310-Adapter-8-Port-6Gb-s-SAS-RAID-Controller-Card-3P0R3

 

[Chris Moore]

A 12GB is definitely a possibility but it won't (likely) ever be a SAS drive but just a plain SATA like a WD Gold or something. Would that work?

Can I ask what IT mode is? And where do you get the tools to flash it/etc?

Sorry, I mistakenly capitalized the b in GB/s for that.
Anyhow, 12Gb/s is not a possibility with SATA, only with SAS. Right now (December 2017) the fastest kind of SATA is the SATA III (revision 3.x) interface, formally known as SATA 6Gb/s. It is the third generation SATA interface, running at 6.0Gb/s. The bandwidth throughput, which is supported by the interface, is up to 600MB/s. This interface is backwards compatible with the older / slower SATA interfaces.

Now, a thing to understand about any of the drive interfaces is that it is the maximum potential, not what the drive can actually deliver. If you read the drive spec sheet of the drives, you will find that most of them deliver mechanical read / write speeds that are in the 100 to 300 MB/s range, which is half or less than the rated speed of the interface. The only kind of drive that is fast enough to saturate the SATA III interface is a SSD, and only some models of those, and I am totally ignoring the faster solid state drive technology that does not use a SATA interface, because we are discussing SATA and not those other interfaces.

That brings us to SAS, and the fact that a SAS (Serial Attached SCSI) HBA (Host Bus Adapter) can talk to SATA drives. You can get SAS adapters that are rated at 12Gb/s (SAS3), and SAS drives that are rated at 12Gb/s, but you are still limited by the mechanical speed of the drive (in mechanical drives) which is sometimes significantly faster with the SAS drives because of better hardware in the drive, but they are also usually much more expensive than SATA drives. If you choose to use SATA drives, there is usually no advantage to the newer, faster controllers. You can pick a SAS 6Gb/s (SAS2) card (like the Dell H310) and be able to address all the speed a SATA drive is able to manage.

IT Mode. The Dell H310 is technically a very low-end 'RAID' controller and the firmware that is loaded on it is called IR (the R being for RAID) but, for FreeNAS, we don't want that. What we do is replace that factory (Dell modified) firmware with the LSI/Avago firmware that is IT (Initiator Target) mode and allows the card to simply act as a 'dumb' interface with no RAID capability. Here is a link to more.
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

This says that it's only 8 port? I assume that means it will only handle 8 drives? Or am I mistaken? My current setup has 14 drives.

The SAS controller has 8 'lanes' but the H310 (which uses a LSI SAS2008 chip) can (after being flashed with the LSI IT mode firmware) control up to either 128 or 256 drives by way of SAS expanders. Sorry, all from memory and I can't remember the exact number. The PERC H700 also has 8 lanes (with two x4 internal mini-SAS SFF8087 connectors) just like the H310. I don't know the specifics of the server you have, but it may have a SAS expander in it that the card is connected to or some of the drives may be connected to SATA ports on the system board. Anyhow, don't get the 'mini card' version of the H310, because it quits working if you flash the non-Dell firmware.
You might want to read this introduction to SAS as it probably explains some of this better than I did:
https://forums.freenas.org/index.ph...imer-on-basic-sas-and-sata.26145/#post-165190

Also some useful info to be found here:
https://www.servethehome.com/lsi-sas-2008-raid-controller-hba-information/

 

[Chris Moore]

Lastly, some of the disks I wish to use do not have TLER. Which has never been a problem for me in a JBOD scenario, I want to also make sure that whatever controller I get will not cause issues if this feature isn't available.

PS. TLER is an ongoing debate with ZFS. The general consensus is that having it is better than not having it, but it is not such a showstopper that you need to rush out and buy all new drives.

 

[sunspot]

Thanks Chris, you've been incredibly helpful.

My server is a dell poweredge R510

Okay here is one I found on ebay: https://www.ebay.com/itm/New-Dell-P...441293&hash=item2f0e0adf81:g:3cwAAOSwZW5Z95kd

What do you think?

The only problem is.. mine can't have that backplate as it sits inside the chassis near the drives in the front and the cables will not reach if I put it all the way to the back. Can I just pop that backplate off? I would assume so. Also, will a 4x PCI-E lane port work with this card? It's what the PERCH700 was using :\ Also will it be enough with 12 drives being WD Blacks 7200rpm, and 2 being Samsung EVO SSDs (another factor here is I want to put the 2 EVOs in RAID1 so maybe IT mode alone wouldn't work for me here?), would that be enough to support the throughput on all of these drives? And yes, there are two SAS cables that connect up to it. As for the extra drives there seems to be an IDE cable running to the backplane? which confused me. I'm well versed in desktop hardware but server hardware is a new beast for me.

Also, according to the specs from dell: http://i.dell.com/sites/doccontent/...heets/Documents/dell-perc-h310-spec-sheet.pdf
It looks like the card handles 32 disks in non RAID stock. Would flashing really be necessary? Is there an advantage over stock?

As an aside, I'm planning on using BTRFS instead of ZFS. I realize it's not as stable/etc. I just love BTRFS :\

EDIT: I've edited post too many times to keep track, please read the latest and not what may have been sent to you via email as my first post! Thanks Chris!

 

[sunspot]

Here are links to pictures of the server, hopefully this helps paint a better picture. The ssds are currently replaced with a 500gb wd blue for some testing.

https://ibb.co/gOTqCw
https://ibb.co/kVTkeb
https://ibb.co/bsXp5G
https://ibb.co/hfWSsw
https://ibb.co/jYaWKb
https://ibb.co/b6erKb
https://ibb.co/cGsp5G

The SAS/Perc H700 is sitting on the chassis.

Thanks!

 

[Chris Moore]

It looks like the card handles 32 disks in non RAID stock. Would flashing really be necessary? Is there an advantage over stock?

As an aside, I'm planning on using BTRFS instead of ZFS. I realize it's not as stable/etc. I just love BTRFS :\

Using BTRFS with FreeNAS isn't really an option, and if you are not using FreeNAS, I don't know if any of this was applicable.
The reason (with FreeNAS) to flash the card to a different firmware is because it does make a difference. Also, the 35 drive limitation is because of the Dell firmware. The Dell card uses the same controller chip and has the same hardware capability, so it can support more drives once it is flashed with the IT mode firmware. The company has changed hands a couple times; it was originally LSI, then became Avago, and it is now owned by Broadcom, but the principle has not changed. The IT (Initiator Target) mode firmware is ideal for FreeNAS to allow the ZFS file system to have direct control of the drives with no possible interference by the firmware on the card.

 

[Chris Moore]

Here are links to pictures of the server, hopefully this helps paint a better picture. The ssds are currently replaced with a 500gb wd blue for some testing.

https://ibb.co/gOTqCw
https://ibb.co/kVTkeb
https://ibb.co/bsXp5G
https://ibb.co/hfWSsw
https://ibb.co/jYaWKb
https://ibb.co/b6erKb
https://ibb.co/cGsp5G

The SAS/Perc H700 is sitting on the chassis.

Thanks!

The Dell H200 SAS controller has a similar form factor to that H700 and it can be flashed to the same IT mode firmware that the H310 uses. It is possible that the H200 might fit in the same slot the H700 came out of and allow the rear accessible slots to remain free for other purposes.

 

[sunspot]

I was under the impression that I could install FreeNAS over any file system. I guess I'll be reading more into that. I was previously managing my server myself with a simple Ubuntu install and was hoping to setup FreeNAS on this server. But now I see that may not be an option if I hope to keep BTRFS.

Curious, looking at the H310 it looks like it would fit as well. Or am I mistaken there? I know that the cables would have to reach around to the other side of the card but I think that would be doable.

Can I do any RAID with the IT firmware? Like I said I'd like to have RAID1 on the SSD's and the rest be JBOD

 

[Chris Moore]

I was under the impression that I could install FreeNAS over any file system. I guess I'll be reading more into that. I was previously managing my server myself with a simple Ubuntu install and was hoping to setup FreeNAS on this server. But now I see that may not be an option if I hope to keep BTRFS.

Curious, looking at the H310 it looks like it would fit as well. Or am I mistaken there? I know that the cables would have to reach around to the other side of the card but I think that would be doable.

FreeNAS is an operating system, not an application, the boot volume is totally dedicated to nothing but the OS installation and the data drives are all organized into a ZFS structure.
I would say that you should go to the resources section of the forum and read up on FreeNAS before you decide. Sun/Oracle originally developed ZFS (the filesystem) and it has a long history of development behind it. It is a superior filesystem to BTRFS.

 

[sunspot]

FreeNAS is an operating system, not an application, the boot volume is totally dedicated to nothing but the OS installation and the data drives are all organized into a ZFS structure.
I would say that you should go to the resources section of the forum and read up on FreeNAS before you decide. Sun/Oracle originally developed ZFS (the filesystem) and it has a long history of development behind it. It is a superior filesystem to BTRFS.

The fact that it's an operating system is the exact reason I would assume it can be installed independent of the FS. It sounds like it's more than an OS, it's a FS as well. Like how windows requires NTFS, well I suppose some versions of windows still let you choose FAT32. Would you call Linux an application because it can be installed on top of almost any file system?

I already stated that I would be reading more about this in my previous post, as I'm obviously ignorant in my expectations of what FreeNAS can do. I'm sensing some frustration that I had hoped to use BTRFS, both since you stopped answering my real questions, and since you're stating your subjective opinion as fact. I hope you can understand that I'm just trying to get this server working and I truly appreciate all of the help you've provided. I meant no disrespect to ZFS nor FreeNAS, I'm sure it's a lovely FS and I planned to revisit it as an option and recompare the two before I set everything up. Obviously, learning that FreeNAS is limited in this regard, I'll have to weight my options more closely.

Thanks for the help you were providing. I'll pull the trigger on one of these cards and maybe give IT a shot. I was just hoping to find out if it could do some RAID. As like I said I'd like to put the two SSDs in RAID1.

Thanks again,

Sunspot

 

[Chris Moore]

The fact that it's an operating system is the exact reason I would assume it can be installed independent of the FS. It sounds like it's more than an OS, it's a FS as well.

I guess it would be more accurate to say the FreeNAS is an appliance. It is built using FreeBSD as the operating system but they take a lot of things out and put some of their own things in. FreeBSD has integrated support for ZFS as the file system and supports booting from a ZFS storage pool. The boot pool of FreeNAS is often on a mirrored pair of hard drives that are mirrored using ZFS, not some hardware RAID. The storage pool is created after FreeNAS is installed based on your requirements but using ZFS is a requirement as support for any other filesystem has been removed.
The BSD kernel has ZFS integrated where if you want to use ZFS on Linux, you have to add that by downloading and installing it. It just works differently.

I'm sensing some frustration that I had hoped to use BTRFS, both since you stopped answering my real questions, and since you're stating your subjective opinion as fact

I didn't stop answering, I just don't sit here doing nothing but this on my day off, it isn't like this is my job. I think ZFS is superior because I know how much more development has gone into it.

Can I do any RAID with the IT firmware? Like I said I'd like to have RAID1 on the SSD's and the rest be JBOD

Sorry, I overlooked this question. No, the IT mode firmware passes the drive data direct to the OS and does nothing at all to it.