Greater than 2TB with Onboard controllers?

[Letni]

Experts,

How does FreeNAS handle drives larger than 2TB with onboard controllers on older motherboards (non uEFI)? Lets say I have my onboard controller set to ACHI and I'm booting FreeNAS off of USB, Is latest FreeNAS 9.3.1 able to recognize the full ~ 3 TB via the onboard controller, or we still have to use an add-on controller to get larger than 2TB?

 

[BigDave]

Experts,

How does FreeNAS handle drives larger than 2TB with onboard controllers on older motherboards (non uEFI)? Lets say I have my onboard controller set to ACHI and I'm booting FreeNAS off of USB, Is latest FreeNAS 9.3.1 able to recognize the full ~ 3 TB via the onboard controller, or we still have to use an add-on controller to get larger than 2TB?

AFAIK there is no drive size restriction in FreeNAS.
If the hardware supports drives larger than 2TB,
FreeNAS can recognize them and use them.

 

[Mirfster]

+1, what BigDave said.

 

[pirateghost]

FreeNAS doesn't care what size it is. Furthermore, the max size of a controller has nothing to do with uefi

 

[Letni]

Thank you for the replies. These sort of answer the question I have on the matter, but not quite.

I guess the biggest gap I have is around where handoff from the controller (as far as size) to FreeNAS's drivers occurs (somewhere in the BSD kernel). Further reading online, it sounds like people are able to make > 2Tb drivers recognize (in Windows 7SP1 64bit and newer - as a slave drive - IE not booting) even though their legacy BIOS doesn't seem to see over 2.2TB. Usually people are stating an updated Windows controller driver allows them to see the full size of the drive. Obviously booting off of a drive > 2 Tb IS going to require UEFI + GTP partitioning, AND an OS that supports all these requirement (Win7Sp1 64 bit or newer), as documented everywhere, but if the OS can overcome the controller (or bios) limitation (at least that's what its sounding like online in other forums), why can't BSD do so?

Thanks!

 

[pirateghost]

Your understanding is really incorrect. That's not how any of this works.

The limit is at the FIRMWARE/HARDWARE level. Has nothing to do with the OS.

 

[Ericloewe]

2TB limitations are only relevant on MBR-formatted disks. Any halfway modern OS uses GPT-formatted disks with no issues.

Older (very older) BIOSes don't know how to boot from GPT, only from MBR. Booting from GPT is a requirement in FreeNAS, though. Not because of the data disks, but because of the boot device(s). This has nothing to do with the SATA controller.

However, some particular controllers (namely LSI SAS1 controllers) are hardware-limited to 2TB and there's no working around that. Intel SATA is not affected.

 

[Mirfster]

However, some particular controllers (namely LSI SAS1 controllers) are hardware-limited to 2TB and there's no working around that. Intel SATA is not affected.

Dell Perc 6/iR would be a prime example of a piece of hardware that cannot see > 2TB (actually 2.2 to be precise).

 

[Letni]

f

However, some particular controllers (namely LSI SAS1 controllers) are hardware-limited to 2TB and there's no working around that. Intel SATA is not affected.

I agree that LSI SAS1 controllers (namely 1064/1068e) do not allow OS's to see above 2.2TB. I get a feeling that this is more due to the fact this is a "HBA" type device (hardware driven controller) vs onboard ACHI (software driven). In my OP, I was directly asking if onboard (south-bridge type) controllers suffer the same issue, and I think you have confirmed that it would not.. As far as I see it, in order to fully support > 2TB, I think the multiple layers can be enumerated this way:

Non-Booting support:

• File System must support > 2 TB (NTFS,ZFS,ETC)
• Partition support for > 2 TB (GTP format)
• Controller driver-level support for > 2 Tb (if software driven controller.. IE ACHI)
• OS support for Advanced Format 4k - Windows 7SP1 64 bit or newer (drivers won't fix this)
• Controller Support for > 2TB (if hardware driven controller.. IE hardware SATA/SAS)
• Hard drive > 2 TB (of course)

Booting support (when trying to boot modern OS's off of > 2TB hard drives - not FreeNAS per-se):

• File System must support > 2 TB (NTFS,ZFS,ETC)
• Partition support for > 2 TB (GTP format)
• Controller driver-level support for > 2 Tb (if software driven controller.. IE ACHI)
• OS support for Advanced Format 4k - Windows 7SP1 64 bit or newer (drivers won't fix this)
• Controller Support for > 2TB (if hardware driven controller.. IE hardware SATA/SAS)
• OS configured to boot UEFI (not MBR boot)
• UEFI Bios and set to Boot UEFI (not CSM)
• Hard drive > 2 TB (of course)

Thanks!

 

[Ericloewe]

I agree that LSI SAS1 controllers (namely 1064/1068e) do not allow OS's to see above 2.2TB. I get a feeling that this is more due to the fact this is a "HBA" type device (hardware driven controller) vs onboard ACHI (software driven). In my OP, I was directly asking if onboard (south-bridge type) controllers suffer the same issue, and I think you have confirmed that it would not..

A lot of confusion there:

"HBA" is just a generic term for a hard drive interface controller. It can be a SATA, PATA, SAS, SCSI or whatever controller.

AHCI is a standard for SATA controllers.

Intel's SATA controllers (found only within PCHs - or Soutbridges, in older systems) do not have any limitation on drive sizes (that we know of, anyway).

LSI's SAS1 hardware had a very shortsighted design that limited drive size to 2TB (probably some internal 32-bit buffers instead of 48 bits to match LBA addressing).

Non-Booting support:

• 
  ...
• Controller driver-level support for > 2 Tb (if software driven controller.. IE ACHI)
• OS support for Advanced Format 4k - Windows 7SP1 64 bit or newer (drivers won't fix this)
• Controller Support for > 2TB (if hardware driven controller.. IE hardware SATA/SAS)

• Driver and hardware support are to be expected. LBAs are 48 bits. LSI just decided to cheap out and cut their buffers in half rather than leave them 1/4+ empty.
• 4k sectors are completely independent of this issue. Of course, if every single link in the chain talks 4k sectors, we're talking about a ceiling that's 8 times higher than with 512 byte sectors (assuming we're dealing with a shortage of LBAs).

Booting support:

• OS configured to boot UEFI (not MBR boot)
• UEFI Bios and set to Boot UEFI (not CSM)

Wrong. FreeNAS boots from a GPT-partitioned device using BIOS mechanisms. My old desktop's motherboard, a Gigabyte X58A-UD3R, received a BIOS update that added GPT boot support - but it wasn't completely new, UEFI firmware.

Of course, the UEFI standard requires GPT booting to be available, but it doesn't mean legacy BIOSes can't boot from GPT.

 

[Letni]

Wrong. FreeNAS boots from a GPT-partitioned device using BIOS mechanisms. My old desktop's motherboard, a Gigabyte X58A-UD3R, received a BIOS update that added GPT boot support - but it wasn't completely new, UEFI firmware.

Of course, the UEFI standard requires GPT booting to be available, but it doesn't mean legacy BIOSes can't boot from GPT.

This was not in reference to FreeNAS booting, this was in reference to a > 2TB drive booting an OS where the OS partition/filesystem is > 2.2TB.. I agree that GTP formatted disks can boot legacy BIOS/CSM, but only if the boot drive is < 2.2 TB

"HBA" is just a generic term for a hard drive interface controller. It can be a SATA, PATA, SAS, SCSI or whatever controller.

I agree HBA is Generic term, but LSI controllers support hardware RAID which means there is a lot more processing going on in the controller CPU vs onboard ACHI where a lot of the controller tasks are offloaded to the actual system CPU. I'm deriving this is why LSI has size limitation on its older chipsets.

Still glad to know that I can purchase 3 or 4TB HD's to start replacing my 2TB drives in my FreeNAS box (Given the drives are plugged into onboard ICH9), given i'm running low on space.

 

[Ericloewe]

This was not in reference to FreeNAS booting, this was in reference to a > 2TB drive booting an OS where the OS partition/filesystem is > 2.2TB.. I agree that GTP formatted disks can boot legacy BIOS/CSM, but only if the boot drive is < 2.2 TB

That would make no sense. If the BIOS can deal with GPT, limiting it to 2TB drives would be an artificial restriction.
The aforementioned motherboard's BIOS update also clearly states that it adds support for booting from 2+TB drives.

I agree HBA is Generic term, but LSI controllers support hardware RAID which means there is a lot more processing going on in the controller CPU vs onboard ACHI where a lot of the controller tasks are offloaded to the actual system CPU.

There's very little processing of any kind going on in an HBA. The AHCI standard is a fancy document that says how a SATA controller should DMA the drive's data in and out of RAM.

An SAS controller is going to be similar, with a little more overhead.

The issue with LSI SAS1 controllers is most likely in the PHY part of the controller. A SATA controller could suffer from the exact same cost-cutting measure.

 

[Michael Kammer]

Is it possible to add on a secondary controller on a server that has the 2TB limit?

I have a Penguin Relion 1600 server with 4 x 2TB drives. I have tried larger drives, but that controller is maxed at recognizing 2 TB per drive.

I would like to add on an additional EXTERNAL array if possible.
Has anyone had any success doing this?

Is this just asking for trouble?

 

[danb35]

Is it possible to add on a secondary controller on a server that has the 2TB limit?

I don't see why not--one of the LSI HBAs should work fine with any reasonably-modern hardware, and has no problem with the 2 TB limit.

 

[Stux]

I don't see why not--one of the LSI HBAs should work fine with any reasonably-modern hardware, and has no problem with the 2 TB limit.

I can't see a reason why it wouldn't work on ancient PCIe1 hardware either.

For those wondering: 2^32 x 512 bytes per LBA is 2.2TB

http://www.informit.com/articles/article.aspx?p=2028834&seqNum=7

 

[Michael Kammer]

Just a follow up: Yes, both internal and external controllers worked. I removed the "daughter-board" internal controller, and added a LSI 9201-16E external, and a half height LSI internal 9211-8i controller.
I did make sure to "cross-flash" the controllers to JBOD firmware (It did take a bit of time to learn how to do this). Both controllers were about $36 each on Ebay.
I ran the external controller to an external 4 drive 8088 SAS chassis for $110 (again on Ebay), and I am running 4x6TB SATAIII HGST NAS drives.
The internal controller, I ran to the SAS/SATA array, and I added 4 SATA III drives which appear to be running at SATA-II speed I suspect.
So I now am running 4 internal and 4 external 6TB drives.
The nice thing about these old servers is that DDR-2 ECC ram can be had very inexpensively. I upgraded the unit to 32GB of ECC ram, again for $36 on Ebay.
I also updated the system to FreeNAS 11.1 and installed the new Plex media server software which appears to be running just fine!
Thanks for the help all.....