zpool status showing drive names differently for different component raidz2(s)

[allegiance]

I have a zpool that was setup a very long time ago, and has been added to over the years by adding new raidz2 collections of drives to it. I am currently now running FreeNAS 9.10.1-US. I believe the first raidz2 was added via command line, the second was added via the GUI in a version before 9.10.1, and the most current one added using the GUI with 9.10.1. When I do a zpool status, I get different formats for the physical drive names in the status between the different zpool2s. Plus, my very first zpool2 complains about block size. These are all 4Tb WD Red drives. Below is what I see when doing a pool status:

pool: Shared

state: ONLINE

status: One or more devices are configured to use a non-native block size.

Expect reduced performance.

action: Replace affected devices with devices that support the

configured block size, or migrate data to a properly configured

pool.

scan: scrub repaired 0 in 115h29m with 0 errors on Thu Jan 19 19:29:32 2017

config:


NAME STATE READ WRITE CKSUM

Shared ONLINE 0 0 0

raidz2-0 ONLINE 0 0 0

gptid/0b184901-4126-11e5-a934-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/a71c4aec-4cdc-11e5-b035-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/0a208c45-49c0-11e5-a8c9-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/c2797b00-45db-11e5-b798-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/41abb3b5-5af1-11e5-964e-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/0b61db0f-5711-11e5-90ec-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/5fe6acb4-6154-11e5-aef2-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/193e97fb-50d0-11e5-9a9d-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/6e9292a8-646a-11e5-baa7-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

gptid/89dfd2f2-5fbb-11e5-9dd4-f46d042cc584 ONLINE 0 0 0 block size: 512B configured, 4096B native

raidz2-1 ONLINE 0 0 0

gptid/af1cc225-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

gptid/aff1afd8-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

gptid/b0cc30aa-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

gptid/b1a7a1e0-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

gptid/b27cccdf-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

gptid/b351f73f-3158-11e4-aa97-f46d042cc584 ONLINE 0 0 0

raidz2-2 ONLINE 0 0 0

da0 ONLINE 0 0 0

da1 ONLINE 0 0 0

da2 ONLINE 0 0 0

da3 ONLINE 0 0 0

da4 ONLINE 0 0 0

da5 ONLINE 0 0 0


I need to soon add another raidz2 (6 more drives), and was wondering:
A) how I should do it (GUI or terminal)
B) should I somehow "fix" my current raidz2 setups so they match when reporting drives (how do I do this?)
C) Worry or not worry about any or all of this

Thank you. Sorry if I am using some of the nomenclature incorrectly. This thing runs pretty solid, so I rarely have to mess with it. It is a backup server, so all data is live elsewhere, but would be a paid to restore or backup/restore all 40+ Tb of data.

 

[nojohnny101]

A) how I should do it (GUI or terminal)

Always do things from the GUI. FreeNAS generally doesn't like when you do things behind its back. Unless you can't do what you want to do in the GUI, there is not reason to go the CLI.

B) should I somehow "fix" my current raidz2 setups so they match when reporting drives (how do I do this?)

I'm not 100% sure on this (someone else will chime in) but I don't believe the different "labels" matter. They are both valid for FreeNAS and don't indicate a problem as far as I understand.

C) Worry or not worry about any or all of this

I see no case for alarm. (put your code in "code tags" though and it would be far easier to read).

 

[Arwen]

Have you rebooted since you added the 3rd RAID-Z2?

It's possible that a simple export and import, (which occurs during reboot),
will change the 3rd RAID-Z2's labeling to match it's companions.

There may be a way to fix your first RAID-Z2's block size issues. If you
can, run this command and let us know the result;

zpool get ashift Shared

Since you have a RAID-Z2, it may be possible to replace each disk in the
first RAID-Z2 one at a time, using the new 4096 blocking. It will cause a
re-silver, (annoying, yes), but you can do it at your convienance. Note that
you can likely replace the 512 byte disk with it's self as a 4096 byte disk.

But first, lets just to verify you have 4096 byte block for your pool.

 

[rs225]

Good news: You can probably do nothing. This is a pool used for bulk written backup data, so your performance impact is minimal.

Bad news: The only way to fix it is to replace every single WD Red with something like a WD RE, or recreate the pool from scratch. A newly created pool should automatically detect and set it for 4K/ashift=12.

 

[Robert Trevellyan]

raidz2-2

Looks like this vdev is using whole disks, so my guess is that's the one that was created using the CLI. The only way I know to change that is to backup everything and rebuild the entire pool.

 

[allegiance]

Looks like this vdev is using whole disks, so my guess is that's the one that was created using the CLI. The only way I know to change that is to backup everything and rebuild the entire pool.

raidz2-2 was definitely added using the GUI, it was pretty recent so I distinctly remember doing it that way.

 

[allegiance]

Good news: You can probably do nothing. This is a pool used for bulk written backup data, so your performance impact is minimal.

Bad news: The only way to fix it is to replace every single WD Red with something like a WD RE, or recreate the pool from scratch. A newly created pool should automatically detect and set it for 4K/ashift=12.

Thank you. I was thinking that too, as far as the performance being non-critical on this. The backups are deposited and accessed over the internet to boot, so the disks are definitely not the slowest point in the chain.

 

[allegiance]

Always do things from the GUI. FreeNAS generally doesn't like when you do things behind its back. Unless you can't do what you want to do in the GUI, there is not reason to go the CLI.


I'm not 100% sure on this (someone else will chime in) but I don't believe the different "labels" matter. They are both valid for FreeNAS and don't indicate a problem as far as I understand.


I see no case for alarm. (put your code in "code tags" though and it would be far easier to read).

I guess my main concern with the drives being labeled da0, da1, etc is I fear it will make it more difficult to track down a bad drive if one fails, and/or will the drive labels change on reboot

 

[allegiance]

Have you rebooted since you added the 3rd RAID-Z2?

It's possible that a simple export and import, (which occurs during reboot),
will change the 3rd RAID-Z2's labeling to match it's companions.

There may be a way to fix your first RAID-Z2's block size issues. If you
can, run this command and let us know the result;

zpool get ashift Shared

Since you have a RAID-Z2, it may be possible to replace each disk in the
first RAID-Z2 one at a time, using the new 4096 blocking. It will cause a
re-silver, (annoying, yes), but you can do it at your convienance. Note that
you can likely replace the 512 byte disk with it's self as a 4096 byte disk.

But first, lets just to verify you have 4096 byte block for your pool.

[root@freenas01] ~# zpool get ashift Shared
bad property list: invalid property 'ashift'
Hopefully not an indication of a problem, but an indication that I did something wrong in the command?

I had the same thought about replacing each disk one by one too - good to hear it from another source.

 

[SweetAndLow]

Once a vdev is created with a certain ashift value it can not be changed. So you are stuck with the 512 block size. Your pool with the strange labels was created using the cli and was done differently then how freenas does it. you can probably give those drives gptid's but you can not adjust the partitions to match what freenas does.

If this was my pool I would plan on a way to backup all the data and rebuild so all the vdevs are the same. Who knows what problems you might run into or it might just be happy and keep working.

 

[Arwen]

[root@freenas01] ~# zpool get ashift Shared
bad property list: invalid property 'ashift'
Hopefully not an indication of a problem, but an indication that I did something wrong in the command?
...

This might be a difference between OpenZFS on FreeBSD / NAS and ZFS on Linux. With Linux you
can use the command as I've listed it. But it appears FreeBSD / NAS does not have it, (yet?). I don't
have access to my FreeNAS right at the moment, (but am using a Linux workstation with ZFS).

...
I had the same thought about replacing each disk one by one too - good to hear it from another source.

@SweetAndLow may have it correct on the vDev being fixed at creation time to 512b or 4096b block
size. I thought read somewhere that someone fixed their pool by doing the replacement, but it may have
been different circumstances.

 

[allegiance]

Here's a new snag (related or not, not sure). I went to add 6 new drives (raidz2) to extend the zpool via the GUI, and I get the follow error:

• You are trying to add a virtual device consisting of 6 device(s) in a pool that has a virtual device consisting of 10 device(s)

So, I dug back through my notes and discovered that I had this issue the last time I extended - so Robert Trevellyan was right, and I was wrong (looking at the wrong set of notes, and poor memory) - raidz2-2 was added via CLI, not GUI. According to my notes, I ran into this error when I tried to add raidz2-2, so I ended up adding/extending it through the CLI.

Can I not mix raidz2 of different drive sizes in the same pool? raidz2-0 (10 drives) and raidz2-1 (6 drives) were added using the GUI, but I think in v8 of FreeNAS. I tried adding raidz2-2 (and now raidz2-3) in v9 via the GUI, but it doesn't seem to work. I was going to try it using "Manual Setup" in the GUI, but I hesitated when I saw the red "Existing data will be cleared" on the "Add Volume" button

 

[bigphil]

[root@freenas01] ~# zpool get ashift Shared
bad property list: invalid property 'ashift'
Hopefully not an indication of a problem, but an indication that I did something wrong in the command?

Doesn't work like that anymore...you may get the values from this command: "zdb -U /data/zfs/zpool.cache | grep ashift"
(you may find the cache file path by: "zpool get cachefile"

 

[Robert Trevellyan]

Can I not mix raidz2 of different drive sizes in the same pool? ... I was going to try it using "Manual Setup" in the GUI, but I hesitated when I saw the red "Existing data will be cleared" on the "Add Volume" button

You can mix vdev layouts in the GUI, using Manual Setup as you suspected. You need to select a volume to extend, rather than creating a new volume.

 

[allegiance]

You can mix vdev layouts in the GUI, using Manual Setup as you suspected. You need to select a volume to extend, rather than creating a new volume.

Yes, that did the trick. Next I will try removing and replacing the discs one at a time in raidz2-2 and see if that fixes the drive name discrepancy. Thanks all for your help and advice!