Removed Disk from Pool and added bigger Samsung 980TB - not recognized

[SKB]

Hi,
the plan was, to make my VM pool bigger - from 500GB to 1TB.

I have a pool for VM's, which had 2 disk mirrored - i removed 1 of the disks (500GB) and added 1 new (1TB).

It was resilvering in around 15 mins - but after restarting and attaching the next 1TB it told me, that the boot-pool, which is on a SSD has some i/o failures, which is/was not true.

So i reverted back to the old setup with 2 500GB NVME's but only one of them is recognized. Its SDC.

root@truenas[~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 2G 0 part
│ └─md127 9:127 0 2G 0 raid1
│ └─md127 253:0 0 2G 0 crypt [SWAP]
└─sda2 8:2 0 929.5G 0 part
sdb 8:16 0 931.5G 0 disk
├─sdb1 8:17 0 2G 0 part
│ └─md127 9:127 0 2G 0 raid1
│ └─md127 253:0 0 2G 0 crypt [SWAP]
└─sdb2 8:18 0 929.5G 0 part
sdc 8:32 0 476.9G 0 disk
├─sdc1 8:33 0 2G 0 part
└─sdc2 8:34 0 474.9G 0 part
sdd 8:48 0 232.9G 0 disk
├─sdd1 8:49 0 1M 0 part
├─sdd2 8:50 0 512M 0 part
├─sdd3 8:51 0 216.4G 0 part
└─sdd4 8:52 0 16G 0 part
sde 8:64 0 931.5G 0 disk
├─sde1 8:65 0 2G 0 part
└─sde2 8:66 0 929.5G 0 part
zd0 230:0 0 50G 0 disk
zd16 230:16 0 50G 0 disk
zd32 230:32 0 10G 0 disk
zd48 230:48 0 20G 0 disk
zd64 230:64 0 20G 0 disk

After that, I erased both new 1TB Samsung 980 via BIOS and they are also showing up in BIOS. Further i formated the other 500GB Disk, to be able, to add it back to the pool, but as you can see, its not showing up anymore. All 3 Samsung NVME's are not showing up anymore. Only the "old" NVM which now holds alone my VM's.

What i also did, for a quick test, i installed a fresh Truenas Scale on another SSD, booted it up and id showed all NVME's. In there, i again formatted them, uploaded my configuration from the other on and rebooted. Now, its again only showing the SDC and none of the Samsung NVME's

The MB is Asrock X570M Pro with AMD Ryzen 5 4600G

Could you assist here, please?

 

[artlessknave]

if I am reading the spec page correctly, the m.2 slots on that gaming motherboard have some sort of RAID thing.
I would bet that's causing you issues. this motherboard is not a good choice for TrueNAS. this is exactly why gaming motherboards are highly discouraged, though they usually aren't this bad.

you could try and see if you can find a way to disable any of that kind of setting, otherwise, if you must use this board, get one of the recomended LSI SAS controllers and avoid the m.2.

 

[SKB]

Hi @artlessknave ,
thanks for replying. Both of the ports were working before.
The recognize both M2 1TB NVME.
Even TrueNAS in the installation boot up from USB.

Further, TrueNAS recognize both of then in a fresh install.
But not after uploading my configuration backup.

So, in my eyes, it has nothing to do with the mainboard. It is config related.

Next one, I opened the freenas dB file and could find those disks mentioned in there. But I am not able, to find further information.
I think, the system is "ignoring" or "blocking" them somehow.

 

[SKB]

Ok, i somehow managed to get the disk visible again in TrueNAS, but now i have the problem, that i am not able to attach it to the VDEV as a mirror, described here (https://www.truenas.com/community/t...or-by-adding-a-drive.14880/page-3#post-453290), as i TrueNAS Scale does not have gpart aboard.

Some ideas?

 

[artlessknave]

it sounds like you are trying to hack it to get it "working". this is not a good idea, and typcially results in issues down the line.

 

[SKB]

Ok, but do you have any idea please, to get the disk attached to zfs again, please?

 

[Patrick M. Hausen]

Use the UI?

 

[SKB]

In the UI, there is only the option, to expand this. Or,did i overlook something?

 

[artlessknave]

supported hardware is supported for a reason; because it works without having the exact issues you are having.

at this point, i consider your problem to be your hardware choices. I do not have the same hardware so there is no way I can test it, and I frankly, wouldn't care to.

maybe someone else will be interested in trying.

 

[artlessknave]

Use the UI?

they are having disks disappear.

 

[Patrick M. Hausen]

You can "extend" a vdev in the CORE UI, don't know about SCALE, but it should work just the same. That function though labelled misleadingly attaches another mirrored device to an existing vdev. Only make triple sure not to add an additional vdev instead, that will build a striped pool - not what you want.

As for the CLI way - you need to create a GPT partition table with the TrueNAS specific layout. How that is done on Linux I would need to search. The reason there is no gpart command is that these tools are named differently on Linux. That's why I suggested the UI.

Edit: "extend", not "expand".

 

[SKB]

You can "expand" a vdev in the CORE UI, don't know about SCALE, but it should work just the same. That function though labelled misleadingly attaches another mirrored device to an existing vdev. Only make triple sure not to add an additional vdev instead, that will build a striped pool - not what you want.

As for the CLI way - you need to create a GPT partition table with the TrueNAS specific layout. How that is done on Linux I would need to search. The reason there is no gpart command is that these tools are named differently on Linux. That's why I suggested the UI.

Thanks for your time here. Much appreciated.

Before i do something wrong, is this the scenario, which should work?

Is that correct?

 

[Patrick M. Hausen]

DON'T ADD A VDEV!!!

Go to pool status, click on the menu to the right of the existing vdev, pick "extend". Sorry for the incorrect "expand" in my last post.

 

[SKB]

DON'T ADD A VDEV!!!

Go to pool status, click on the menu to the right of the existing vdev, pick "extend". Sorry for the incorrect "expand" in my last post.

Thanks. Will try to find it. I am just out for few mins. Head refresh ;)

 

[SKB]

DON'T ADD A VDEV!!!

Go to pool status, click on the menu to the right of the existing vdev, pick "extend". Sorry for the incorrect "expand" in my last post.

Hm, ok, to be honest, that was, what i clicked. Is there now an option, to remove the VDEV from that again, please?
i read your reply as expand :(
Thanks!

 

[SKB]

root@truenas[~]# zpool import
pool: Virtual_Machines
id: 10106285119122570150
state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

Virtual_Machines UNAVAIL insufficient replicas
41fb82ba-1c46-4da2-818c-8bd7c917c698 ONLINE
7caea7a5-dd75-4525-aa2a-e21eda8167f4 UNAVAIL

This is, what CLI says now and the GUI shows this:

Please help!

 

[WI_Hedgehog]

What @artlessknave said is unfortunately correct, systems like yours are susceptible to random drive loss under TrueNAS. We're not trying to be server snobs or unhelpful, it's rather the opposite, we want you to succeed. (Similar thread on this MB)

Gaming MBs tend to work under certain conditions and not others, just like some Datacenter servers might play some games if you are persistent, but it's generally more trouble than it's worth because a Datacenter server just won't do what you want the way you want it to and when it gets close to what you want its a delicate kludge that breaks easily. Same here with your gaming rig trying to run Datacenter software.

Ubuntu Server might work better with this hardware, and it will run ZFS, though LVM with ext4 is probably a better choice.

 

[Patrick M. Hausen]

Is there now an option, to remove the VDEV from that again, please?

No, there isn't. That's why I also wrote in my first reply:

Only make triple sure not to add an additional vdev instead, that will build a striped pool - not what you want.

 

[SKB]

What @artlessknave said is unfortunately correct, systems like yours are susceptible to random drive loss under TrueNAS. We're not trying to be server snobs or unhelpful, it's rather the opposite, we want you to succeed. (Similar thread on this MB)

Gaming MBs tend to work under certain conditions and not others, just like some Datacenter servers might play some games if you are persistent, but it's generally more trouble than it's worth because a Datacenter server just won't do what you want the way you want it to and when it gets close to what you want its a delicate kludge that breaks easily. Same here with your gaming rig trying to run Datacenter software.

Ubuntu Server might work better with this hardware, and it will run ZFS, though LVM with ext4 is probably a better choice.

That might be the reason, why this system was running smoothly the last year till the stupid human have had the idea, to upgrade the space.

That's the real reason, why it didn't work. Not that it's a gaming MB.

Thanks for not helping me to succeed.
I installed everything new on the new disks.

 

[SKB]

No, there isn't. That's why I also wrote in my first reply:

Yeah, I noticed that.

Re-Install helped.
Thanks for pointing that.