New disks are not recognized - FreeNAS-11.2-RELEASE-U1

[maozefang]

Hi all,

Long time visitor of the forum, first-time poster. Up until today, the information available from the community was enough to get me unstuck. However this time I cannt seem to find an answer to my problem.

Three years ago, I put together a small server with leftover parts from a previous Home Theatre PC.
FreeBSD 11.2-STABLE
Base Board: MSI - Z77MA-G45 (MS-7759) Version 1.0
CPU: Intel(R) Core(TM) i5-3350P CPU @ 3.10GHz
RAM: 4 x 8GB Corsair Vengeance (CMZ4GX3M1A1600C9) @1333 MT/s
da0: drive: Corsair Voyager VEGA - 32GB
ada0: TOSHIBA DT01ACA300 MX6OABB0 - 3TB
ada1: TOSHIBA DT01ACA300 MX6OABB0 - 3TB
ada2: WDC WD60EFRX-68L0BN1 82.00A82 - 6TB

Freenas is installed on the 32 GB USB key, the 2 x 3TBs drives are mirrored and were the initial pool while the 6TB, added later on, is used mostly to store my videos. In need to expand storage capacity I am now trying to add 2 x 4TB WD Reds (likely striped). Unfortunately, the new drives are not detected neither under the Disks nor under the pool extension page. As a test, I also tried a new 3TB Seagate Barracuda that I had laying around, same result :(

I have searched on the forum and google but I still cannot figure out why the new drives are not detected. The only threads I found were pointing at hardware incompatibility, but this was usually on new builds which is not the case for mine. If anyone could help it would be much appreciated.

I have run a few commands, output listed below:

Code:

camcontrol devlist
<TOSHIBA DT01ACA300 MX6OABB0>      at scbus0 target 0 lun 0 (pass0,ada0)
<TOSHIBA DT01ACA300 MX6OABB0>      at scbus1 target 0 lun 0 (pass1,ada1)
<WDC WD60EFRX-68L0BN1 82.00A82>    at scbus2 target 0 lun 0 (pass2,ada2)
<Corsair Voyager VEGA 000A>        at scbus5 target 0 lun 0 (pass3,da0)

glabel status
                                      Name  Status  Components
gptid/77180a80-87fb-11e6-983c-d050991d0a0f     N/A  ada0p2
gptid/764974ec-87fb-11e6-983c-d050991d0a0f     N/A  ada1p2
gptid/ad1ae1ae-5ded-11e8-81e8-8c89a5c4a017     N/A  ada2p2
gptid/9ac2af8f-5d6e-11e4-ad99-d050991d0a0f     N/A  da0p1
gptid/9acf3275-5d6e-11e4-ad99-d050991d0a0f     N/A  da0p2
gptid/76fdd7ab-87fb-11e6-983c-d050991d0a0f     N/A  ada0p1

gpart status
  Name  Status  Components
ada0p1      OK  ada0
ada0p2      OK  ada0
ada1p1      OK  ada1
ada1p2      OK  ada1
ada2p1      OK  ada2
ada2p2      OK  ada2
 da0p1      OK  da0
 da0p2      OK  da0

gpart list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   efimedia: HD(1,GPT,76fdd7ab-87fb-11e6-983c-d050991d0a0f,0x80,0x400000)
   rawuuid: 76fdd7ab-87fb-11e6-983c-d050991d0a0f
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada0p2
   Mediasize: 2998445412352 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,77180a80-87fb-11e6-983c-d050991d0a0f,0x400080,0x15d10a308)
   rawuuid: 77180a80-87fb-11e6-983c-d050991d0a0f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2998445412352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 5860533127
   start: 4194432
Consumers:
1. Name: ada0
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e3

Geom name: ada1
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 5860533134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,762809f0-87fb-11e6-983c-d050991d0a0f,0x80,0x400000)
   rawuuid: 762809f0-87fb-11e6-983c-d050991d0a0f
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada1p2
   Mediasize: 2998445412352 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,764974ec-87fb-11e6-983c-d050991d0a0f,0x400080,0x15d10a308)
   rawuuid: 764974ec-87fb-11e6-983c-d050991d0a0f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2998445412352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 5860533127
   start: 4194432
Consumers:
1. Name: ada1
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 11721045127
first: 40
entries: 152
scheme: GPT
Providers:
1. Name: ada2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,ad0b7fa3-5ded-11e8-81e8-8c89a5c4a017,0x80,0x400000)
   rawuuid: ad0b7fa3-5ded-11e8-81e8-8c89a5c4a017
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada2p2
   Mediasize: 5999027552256 (5.5T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,ad1ae1ae-5ded-11e8-81e8-8c89a5c4a017,0x400080,0x2ba60f400)
   rawuuid: ad1ae1ae-5ded-11e8-81e8-8c89a5c4a017
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 5999027552256
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 11721045119
   start: 4194432
Consumers:
1. Name: ada2
   Mediasize: 6001175126016 (5.5T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5

Geom name: da0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 60555230
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da0p1
   Mediasize: 524288 (512K)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 4210688
   Mode: r0w0e0
   efimedia: HD(1,GPT,9ac2af8f-5d6e-11e4-ad99-d050991d0a0f,0x2020,0x400)
   rawuuid: 9ac2af8f-5d6e-11e4-ad99-d050991d0a0f
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: (null)
   length: 524288
   offset: 4210688
   type: freebsd-boot
   index: 1
   end: 9247
   start: 8224
2. Name: da0p2
   Mediasize: 30999539712 (29G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 4734976
   Mode: r1w1e2
   efimedia: HD(2,GPT,9acf3275-5d6e-11e4-ad99-d050991d0a0f,0x2420,0x39bdbb8)
   rawuuid: 9acf3275-5d6e-11e4-ad99-d050991d0a0f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 30999539712
   offset: 4734976
   type: freebsd-zfs
   index: 2
   end: 60555223
   start: 9248
Consumers:
1. Name: da0
   Mediasize: 31004295168 (29G)
   Sectorsize: 512
   Mode: r1w1e3

 

[Chris Moore]

The only threads I found were pointing at hardware incompatibility, but this was usually on new builds which is not the case for mine.

It could still be some incompatibility.
What kind of drives are you attempting to add?
You may need to use a different controller card to run your storage drives. It is also a possibility that the ports you are attempting to use have been disabled in the BIOS.

 

[maozefang]

It could still be some incompatibility.
What kind of drives are you attempting to add?
You may need to use a different controller card to run your storage drives. It is also a possibility that the ports you are attempting to use have been disabled in the BIOS.

Hi Chris,
thanks for the quick reply.

I am trying to add 2 x 4TB Western Digital Red Pro.

Other than hooking up a monitor a reboot to access Bios, is there a way to check enables/disabled ports?

I ran a lspci and this is what came up:

Code:

 lspci
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)
00:1f.0 ISA bridge: Intel Corporation SATA Controller [RAID mode] (rev 04)
00:1f.0 ISA bridge: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series]
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series]
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

 

[Chris Moore]

Other than hooking up a monitor a reboot to access Bios, is there a way to check enables/disabled ports?

Not with that system board.
It looks like a good Intel chipset, but it is possible for some of the ports to have been configured differently or even disabled in the BIOS.

00:1f.0 ISA bridge: Intel Corporation SATA Controller [RAID mode] (rev 04)

This is the thing that stands out to me... All the SATA ports need to be configured as plain SATA ports. That's the first thing to check.

Ethernet controller: Realtek Semiconductor

That is not the best kind of Network adapter, but it isn't the problem right now. It will just make the network run slower.

 

[maozefang]

Not with that system board.
It looks like a good Intel chipset, but it is possible for some of the ports to have been configured differently or even disabled in the BIOS.

I will have a look then, good opportunity to tidy up the cable management in my cabinet.

This is the thing that stands out to me... All the SATA ports need to be configured as plain SATA ports. That's the first thing to check.

I believe you may have well found the cause indeed. This reminded me of an old feature that I enabled on the previous machine this motherboard was pulled from, IntelSmart Response Technology (essentially SSD caching)

Show : Intel Smart Response Technology

Technology Brief
SSD Requirements
Intel Smart Response Technology requires a PCIe* or SATA based SSD in conjunction with a traditional HDD or a SSHD.

Discrete SSD Cache Requirements
Any SATA SSD can be used as a cache provided it has a minimum capacity of 16GB. The system SATA controller must be set to RAID mode via the system BIOS and the SSD attached to an internal PCIe or SATA port of the computer. There is no support for Intel Smart Response Technology in AHCI mode. Please refer to the platform specific design guides for PCIe and SATA supported lanes.

That is not the best kind of Network adapter, but it isn't the problem right now. It will just make the network run slower.

I have to admit it is a bit sluggish. Would you recommend a NIC? I will have a look at the hardware requirement guide. At that point I am wondering if I should not actually be looking at updating the entire platform. It will depend cost-wise I believe

Thank you for the pointers! I will try a few things and post my results later today.

 

[Chris Moore]

I have to admit it is a bit sluggish. Would you recommend a NIC?

Most Intel NICs work well in FreeNAS.

At that point I am wondering if I should not actually be looking at updating the entire platform.

You might want to take a look at this option:

https://www.ixsystems.com/community/resources/specific-build-components-list-up-to-32gb-ram.109/

 

[maozefang]

Well...still at a dead end I am afraid

Not with that system board.
It looks like a good Intel chipset, but it is possible for some of the ports to have been configured differently or even disabled in the BIOS.

This is the thing that stands out to me... All the SATA ports need to be configured as plain SATA ports. That's the first thing to check.

Switched the configuration back to default in the BIOS. SATA in AHCI mode and no ports disabled.

00:1f.0 ISA bridge: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
00:1f.0 ISA bridge: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller (rev 04)

That is not the best kind of Network adapter, but it isn't the problem right now. It will just make the network run slower.

I did pick up an Intel NIC along the way. Small consolation :)
Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

 

[Chris Moore]

I can't imagine why those drives would not be detected but it could be some peculiarity of the system board. I had a board that disabled some hardware automatically if other hardware was enabled because there were not enough PCI-E lanes for everything to work at once.
It might be worth trying one of these SAS controllers. You can run all your data drives from one controller:
https://www.ebay.com/itm/Dell-H310-6Gbps-SAS-HBA-w-LSI-9211-8i-P20-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601
I used to have one like this in my server and it works fine, but it does get some heat to it, so it needs airflow.
It also needs a set of cables like this to connect the drives:
https://www.ebay.com/itm/Lot-of-2-M...-Forward-Breakout-Internal-Cable/371681252206

 

[gpsguy]

We don't recommend adding stripes with mirrors in the same pool. With your current configuration, should the 6TB drive fail, you'll loose all your data.

the 2 x 3TBs drives are mirrored and were the initial pool while the 6TB, added later on, is used mostly to store my videos. In need to expand storage capacity I am now trying to add 2 x 4TB WD Reds (likely striped)

As to your problem with detecting the drives, you might want to check and see if there's a newer BIOS for your motherboard.

 

[Chris Moore]

the 2 x 3TBs drives are mirrored and were the initial pool while the 6TB, added later on, is used mostly to store my videos. In need to expand storage capacity I am now trying to add 2 x 4TB WD Reds (likely striped).

I wasn't going to address this until later, but gpsguy brought it up. Is the 6TB drive in the same pool with the mirror set? Do you understand about a single drive failure destroying the whole pool?
There is a slideshow about these things that you might aught to look at:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

 

[maozefang]

We don't recommend adding stripes with mirrors in the same pool. With your current configuration, should the 6TB drive fail, you'll loose all your data.

Hi gpsguy. Thank you for the recommendation. The 6TB is on its own pool which. The idea was to extpand with a 4TB since the data on on the 6TB is disposable and/or recoverable from physical disks - my intention here was to go with maximum capacity rather than data preservation. The 3TBs are mirrored and in a separate pool, that's where I keep things I do not want to lose and also where my iocage jails/plugins are housed.

I see now how it was not clear in my intial posts ;)

As to your problem with detecting the drives, you might want to check and see if there's a newer BIOS for your motherboard.

BIOS appears to already be up to the latest major release from 2013.

You can run all your data drives from one controller:
https://www.ebay.com/itm/Dell-H310-6Gbps-SAS-HBA-w-LSI-9211-8i-P20-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601

thanks Chris you answered my next question. I was wondering if an HBA would succesfully bypass the SATA port quirks on the motherboard.
I believe that could be the most economical alternative rather than a new board/CPU setup.
I will see if I can source it from ebay Canada

 

[Chris Moore]

I will see if I can source it from ebay Canada

If you get it somewhere else, there are several models that are basically the same, so you might want to be flexible, just be aware that you need to ensure that the firmware is flashed to IT mode, even if you need to do it yourself and on the latest version. Here is some guidance on models:

Don't be afraid to be SAS-sy
https://forums.freenas.org/index.php?resources/don't-be-afraid-to-be-sas-sy.48/

Hardware Recommendations Guide Rev. 1e) 2017-05-06
https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/

Hardware Recommendations by @cyberjock - from 26 Aug 2014 - and still valid
https://forums.freenas.org/threads/hardware-recommendations-read-this-first.23069/

 

[gpsguy]

Sorry for going off-tangent.

We often see users make this mistake, but it doesn't apply to your pool layout.

The 6TB is on its own pool which

 

[Chris Moore]

I will see if I can source it from ebay Canada

Sorry for the delay. I had to go looking for this. This is a thread I wanted to share to you earlier but I couldn't find it:

Confused about that LSI card? Join the crowd ...
https://www.ixsystems.com/community/threads/confused-about-that-lsi-card-join-the-crowd.11901/

 

[maozefang]

Sorry for the delay. I had to go looking for this. This is a thread I wanted to share to you earlier but I couldn't find it:

Confused about that LSI card? Join the crowd ...
https://www.ixsystems.com/community/threads/confused-about-that-lsi-card-join-the-crowd.11901/

That's great info and was much needed. I think I will focus my efforts on finding a M1015 as it appears to be a popular and reliable choice in the community.

I had found an offer for a used LSI 9750-4i in my area and spent the afternoon trying to figure out if it would work. (another post from jgreco indicates it would not be the best fit https://www.ixsystems.com/community/threads/freenas-on-high-end-home-built-platform.12591/)

 

[maozefang]

Sorry for going off-tangent.

We often see users make this mistake, but it doesn't apply to your pool layout.

No worries at all. I actually appreciate you taking the time to offer best practices. I do try to follow the guides and manual but i am bound to miss steps for sure.

Once I am successful at having Freenas detect those new drives, I will be onto another challenge in switching from my onboard network controller to that new NIC I purchased following Chris's advice. Something is telling me it will not be a walk in the park :p

 

[maozefang]

Hi guys,

Unfortunately the issue persists. I flashed and installed a Dell H310 (following this guide) and the disks are still not visible. I tried different cables, switched out power supplies to rule those out.

May have to call it on this platform. I will keep the new hard drives on the shelf until I can put together a machine based on the Hardware Recommendations.

Thank you for all the help though, greatly appreciated.

 

[Heracles]

Hi Maozefang,

Should you wish to keep pushing for your diagnosis, you can do something like this :
--Power off FreeNAS
--Unplug the drives used by FreeNAS (pool and boot)
--Plug only the new drives that you try to get recognized
--Make yourself a new FreeNAS installer
--Add a temporary 8+ Gig USB stick
--Boot the server with the Installer and install a brand new FreeNAS on the temporary stick
--See if that new temporary drive has better luck recognizing the drives

You can try re-using the same connections as the existing drives as well as with the extra connections you are trying to use.

If the new drives are not recognized when plugged in the same cables and ports as the existing drives, you know the problem is around the drives.
Should the drives works only when plugged in the same cable / ports as the existing drive, the problem is probably related to the board.
Should you get the new drives recognized when plugged in different ports, then you may have fix your setup. Put back you server in its original state (boot drive and drives with the pool) and reboot your actual FreeNAS instance.

You can do this kind of testing with a different operating system like Linux. If Linux can see the drives on the extra ports, then the problem is not in the hardware.

Good luck doing your diagnosis,

 

[gpsguy]

What are teh spec's on your power supply?

You might want to try a molex to SATA power connector. Some users have run into issues related to SATA revision 3.3.

https://en.wikipedia.org/wiki/Serial_ATA#SATA_revision_3.3

 

[Chris Moore]

WD60EFRX

Are these shucked USB drives? That makes a big difference.

Take a look at this video.
https://www.youtube.com/watch?v=9W3-uOl4ruc