Trying to expand 7 drive pool but can't detect new drives

[theBiggVan]

Hi All,
New to freenas and NAS in general. I have setup a nas using initially 6 drives (4tb Ironwolfs). One of them was DOA so i had that replaced with another and ended up recreating the pool using RAID Z2 instead of Z because of all the horror stories. So 5 drives in RAIDZ2 then expanded with the 6th drive once it finally arrived. purchased and expanded with a 7th drive to the pool with no issue. Purchased another drive (these are all the same 4tb Ironwolf NAS drives) this time used from ebay. installed and went to view disks and only saw the original 7 drives. Switched around some available cables both power and sata and still not seeing it so i assumed it was defective. Just sent that out for return this morning and received a new drive this evening i purchased from amazon. Install the drive and same thing. the new drive does not show up in the "View Disks". Tried to manually expand and the drive does not show up like the 6th or 7th one did before. See build below.

Am i doing something wrong or is it possible that i just have two bad drives in a row? am i not able to expand a pool pass 7 drives? DO i have to recreate the pool with the intended 10 drives? Even if i cant expand to more than 7 drive wouldnt i still beable to see a drive in the "View Disks" section? or do i need to format the drives before freenas can recognize it? Thank you in advance for any help.

CPU: i3 4170
MOBO: Supermicro x10SAE
RAM Crucial 16GB 2x8 CT2KIT102472BD160B ( plan to anyway but do i need to add more before expanding pass 17TB?)
PSU: Corsair VS600 (I went through a few PSU calculators for 10 drives and my build. Do i need more power than this?)
Drives: ST4000VN008
SATA Card: Syba SI-PEX40064
FreeNas build: FreeNAS-9.10.2-U3 (e1497f269)

 

[danb35]

How are you expanding the pool? Because if your answer is anything other than "backing up the data, destroying the pool, and recreating it with n + 1 disks", you don't have a seven-disk RAIDZ2 pool; you have a five-disk RAIDZ2 pool with two additional disks striped in. That's technically known as Hating Your Data. Expanding with additional disks will make the situation even worse.

You're certainly able to have more than seven disks in a pool; I currently have 18 (see system specs below). And though your motherboard isn't ideal for a FreeNAS box, it should work.

To answer your direct questions, can you post a couple of screenshots? One of your volume status page (Storage button on top, select your pool (first line in the list with that name), Volume Status button at the bottom (looks like a blank sheet of notebook paper)), and one of the View Disks screen (Storage button on top, View Disks button). And if the newly-added disk doesn't show, also post the output of camcontrol devlist at the shell.

Edit: How are your disks attached? That is, which ones are attached to which ports? Your motherboard has six that are controlled by one controller and two that are controlled by a second, and then you have an add-in SATA controller (which are nearly universally garbage). Which ports are the existing, detected disks plugged into? Where is the newly-connected, not-detected disk plugged into?

 

[Spearfoot]

"So 5 drives in RAIDZ2 then expanded with the 6th drive once it finally arrived. purchased and expanded with a 7th drive to the pool with no issue."

Yikes! You can't expand a RAIDZ pool by adding individual drives to it; I'm afraid you may have striped the 6th disk with your 5-disk RAIDZ2 array. If that's the case, you'll have to destroy your pool and re-create it.

Show us the pool's status -- from the GUI, go to Storage; select the pool; click Volume Status (the right-most of the three buttons at the bottom of the screen). Then post a screenshot of the results here.

I've attached a couple of screenshots from one of my systems to help you navigate, just in case my instructions don't make any sense:


EDIT: Ha! I see @danb35 beat me to it...

 

[SweetAndLow]

Can't add drives to a vdev. You probably created a new vdev with a single disk.

Sent from my Nexus 5X using Tapatalk

 

[theBiggVan]

How are you expanding the pool? Because if your answer is anything other than "backing up the data, destroying the pool, and recreating it with n + 1 disks", you don't have a seven-disk RAIDZ2 pool; you have a five-disk RAIDZ2 pool with two additional disks striped in. That's technically known as Hating Your Data. Expanding with additional disks will make the situation even worse.

You're certainly able to have more than seven disks in a pool; I currently have 18 (see system specs below). And though your motherboard isn't ideal for a FreeNAS box, it should work.

To answer your direct questions, can you post a couple of screenshots? One of your volume status page (Storage button on top, select your pool (first line in the list with that name), Volume Status button at the bottom (looks like a blank sheet of notebook paper)), and one of the View Disks screen (Storage button on top, View Disks button). And if the newly-added disk doesn't show, also post the output of camcontrol devlist at the shell.

Edit: How are your disks attached? That is, which ones are attached to which ports? Your motherboard has six that are controlled by one controller and two that are controlled by a second, and then you have an add-in SATA controller (which are nearly universally garbage). Which ports are the existing, detected disks plugged into? Where is the newly-connected, not-detected disk plugged into?

For starters backing up and recreating is no issue. I have plenty of local storage to do that. just wanted to make sure i can have a 10+ RAIDZ2 pool. Guess ill just have to wait until i have all the drives i want then recreate. Attaching Volume status and camcontrol devlist.

So it looks like the last two drives were striped. destroy and recreate it is but still not detecting new drives.

6 drives connected to the first onboard sata controller. 1 connected to the second onboard sata controller. connected 1 to the sata card to make sure it was working and it is NOT the one that is not being detected.

When you say the mobo is not ideal do you mean because of the number of sata ports?

and again thank you for the help

 

[danb35]

It's absolutely possible to create a 10-disk RAIDZ2 vdev, but you really don't want to go over 12 disks in one vdev. If you need more disks than that in a pool, they should be split across two or more vdevs (I'm using three, six-disk vdevs for my pool).

The View Disks and camcontrol devlist are in agreement; your system is only seeing seven disks. Where is the one attached that the system isn't seeing? And does your system see the disk when you boot? I.e., does it show in the BIOS or POST screens? If it doesn't show there, try swapping cables (power and data), and try moving it to a different port. If it still isn't showing, as unlikely as it sounds, you would appear to have a second DOA disk.

When you say the mobo is not ideal do you mean because of the number of sata ports?

The X10SAE is a workstation board. It has features you don't really want in a server (like audio), which can in unusual cases cause problems, and it doesn't have features that really are helpful in a server (like IPMI). A better choice in the X10 series would have been the X10SL7-F, which has IPMI, as well as an onboard LSI HBA. One of those boards could handle all 36 bays in my chassis, though its RAM capacity is a little low for that. IPMI is great for remote management.

You'd asked earlier about your power supply, and it probably is a little low-capacity for 10 disks. @jgreco has written up some guidance on power supply sizing that would be worth taking a look at.

Edit: And in your current configuration, you effectively have no redundancy. If one of those two striped disks fails, your entire pool is toast and all your data (well, all of it that isn't backed up) will be lost.

 

[theBiggVan]

It's absolutely possible to create a 10-disk RAIDZ2 vdev, but you really don't want to go over 12 disks in one vdev. If you need more disks than that in a pool, they should be split across two or more vdevs (I'm using three, six-disk vdevs for my pool).

The View Disks and camcontrol devlist are in agreement; your system is only seeing seven disks. Where is the one attached that the system isn't seeing? And does your system see the disk when you boot? I.e., does it show in the BIOS or POST screens? If it doesn't show there, try swapping cables (power and data), and try moving it to a different port. If it still isn't showing, as unlikely as it sounds, you would appear to have a second DOA disk.


The X10SAE is a workstation board. It has features you don't really want in a server (like audio), which can in unusual cases cause problems, and it doesn't have features that really are helpful in a server (like IPMI). A better choice in the X10 series would have been the X10SL7-F, which has IPMI, as well as an onboard LSI HBA. One of those boards could handle all 36 bays in my chassis, though its RAM capacity is a little low for that. IPMI is great for remote management.

You'd asked earlier about your power supply, and it probably is a little low-capacity for 10 disks. @jgreco has written up some guidance on power supply sizing that would be worth taking a look at.

Edit: And in your current configuration, you effectively have no redundancy. If one of those two striped disks fails, your entire pool is toast and all your data (well, all of it that isn't backed up) will be lost.

The drive not being detected was connected to the second onboard controller.

Disconnected a drive to test the one that is not being detected and freenas still shows the disconnected drive (ada0 SN: ZDH13JHK) and still doesn't show the undetected one in view disks. Same for volume status. showing same 7 drives as if nothing changed.

Also thank you for all your help. Backing up now. i guises its better to have learned from my mistake now than later. Going to stick with 8 drives for now assuming the 8th one is not DOA. Maybe future mobo upgrade and drive expansion.

 

[gpsguy]

As others have said, even though your Syba controller appears to work, I'd dump it.

Get something like an IBM M1015 instead and flash it to IT mode.

 

[Stux]

8 drive RaidZ2 is a great compromise between reliability, performance and storage efficiency. I use it myself ;)

If you ever want more space... add another 8 drives... and perhaps upgrade to a 24 bay enclosure ;)

 

[danb35]

Disconnected a drive to test the one that is not being detected and freenas still shows the disconnected drive (ada0 SN: ZDH13JHK) and still doesn't show the undetected one in view disks. Same for volume status. showing same 7 drives as if nothing changed.

This sounds very odd. Can you give a bit more detail about what you did? Did you unplug the one disk and plug in the other while your server was running? Or did you shut it down, unplug ada0, plug in the questionable disk in where ada0 had been, and then power up? And if the latter, are you sure that ada0 was still showing the same serial number as before?

Also, I didn't see that you addressed my earlier question:

And does your system see the disk when you boot? I.e., does it show in the BIOS or POST screens?

 

[danb35]

perhaps upgrade to a 24 bay enclosure ;)

That's what I was looking to do about a year ago--my 12-bay chassis was full, and my pool was pretty full as well. Increasing capacity would have meant (1) replacing six disks with larger disks, resilvering with each addition, and (2) tossing out six perfectly good (well, still working, anyway) disks. (1) seemed painful, and (2) seemed wasteful--so instead of tossing out six working disks, I bought a whole new (to me) server. Hmmm, maybe not the most economical decision. But I still have 18 bays free, so I doubt I'll run out of space anytime soon (and if I should happen to fill it, it has external SAS ports for me to add a disk shelf if needed or desired).

 

[theBiggVan]

This sounds very odd. Can you give a bit more detail about what you did? Did you unplug the one disk and plug in the other while your server was running? Or did you shut it down, unplug ada0, plug in the questionable disk in where ada0 had been, and then power up? And if the latter, are you sure that ada0 was still showing the same serial number as before?

Also, I didn't see that you addressed my earlier question:

OK just returned that drive. Apparently im pretty unlucky. Hopefully the next drive will be ok. (Maybe Ironwolfs weren't such a good idea. ANybody have any majopr trouble with them?) couldnt find the drive in the NAS build BIOS. So i took the drive out of my NAS and dropped it in my main PC using a known working sata cable. (turned the NAS off before i made any changes. always do) The drive wouldnt initialize in windows. Kept getting an I/O error. So thats gone and now im waiting for another. Also did a factory reset last night and holding off on creating anything until i get my 8th drive. If i expand in the future now i know have to destroy and recreate. May not go past 11 since that all my case holds. May upgrade the mobo in the future and find a more reasonable case but once i get my 8 drives up and running i think im done for now as far as changes.

swapped a drive from my main into the nas and it detected it with no issues so im good for now. removed the sata card and sticking to the 8 onboard ports.


Thank you very much for all your hjelp and guidance. Really appreciate it.

 

[theBiggVan]

This sounds very odd. Can you give a bit more detail about what you did? Did you unplug the one disk and plug in the other while your server was running? Or did you shut it down, unplug ada0, plug in the questionable disk in where ada0 had been, and then power up? And if the latter, are you sure that ada0 was still showing the same serial number as before?

I always shut it down before i move anything. Cant remember the position but i know for sure it was showing the serial number of the drive that was disconnected.

 

[danb35]

You certainly can safely expand your pools, if you do it correctly. "Correctly" entails adding redundant vdevs. For example, I created my pool with one six-disk RAIDZ2 vdev, and expanded it with a second six-disk RAIDZ2 vdev, and then with a third. But single disks just doesn't work safely. Odd that you'd have such bad luck with the disks, but before you get a lot of data on them is the time to find out. Make sure you burn in/test them before recreating your pool.

 

[ccssid]

psu ? Take a look at my signature. Never a problem. Although, if I had to do over again ...maybe...but not necessarily ....I might consider a 650 watt seasonic.