SOLVED Problem with "non-unique serial numbers disks"

[waldauf]

Hello!

I'm new to TrueNAS but eager to learn. :) I just installed my first TrueNAS SCALE and wanted to create a pool. But I got stuck on this message:
Warning: There are 2 disks available that have non-unique serial numbers. Non-unique serial numbers can be caused by a cabling issue and adding such disks to a pool can result in lost data.

Is there a way to find out which disks have problems? Below I put information about my HW.

About TrueNAS:
OS Version:TrueNAS-SCALE-23.10.0.1
Product:ODROID-H3+
Model:Intel(R) Pentium(R) Silver N6005 @ 2.00GHz
Memory:8 GiB
System Serial:Default string


Information about my disks: Seagate IronWolf Pro, 3,5" - 4TB (ST4000NT001)

admin@truenas[~]$ sudo fdisk -l
Disk /dev/mmcblk0: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 34A18C3F-00F0-4DD0-BE0E-24F80B1A924F

Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 6143 2048 1M BIOS boot
/dev/mmcblk0p2 6144 1054719 1048576 512M EFI System
/dev/mmcblk0p3 1054720 61071326 60016607 28.6G Solaris /usr & Apple ZFS


Disk /dev/mmcblk0boot0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mmcblk0boot1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdf: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3974581D-0C31-C049-95FE-9D432C551AED

Device Start End Sectors Size Type
/dev/sdf1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sdf9 7814019072 7814035455 16384 8M Solaris reserved 1


Disk /dev/sde: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 1F8EF0D8-C50B-CE4C-90D5-DB63B49932E0

Device Start End Sectors Size Type
/dev/sde1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sde9 7814019072 7814035455 16384 8M Solaris reserved 1


Disk /dev/sdc: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 41088C02-A3AA-6847-8DDD-B707C02E2F24

Device Start End Sectors Size Type
/dev/sdc1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sdc9 7814019072 7814035455 16384 8M Solaris reserved 1


Disk /dev/sdd: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 1FFC850C-834B-9F47-AB12-D0451E39C9A6

Device Start End Sectors Size Type
/dev/sdd1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sdd9 7814019072 7814035455 16384 8M Solaris reserved 1


Disk /dev/sda: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: E6841B6A-EF0B-8141-885C-9C3B450C292E

Device Start End Sectors Size Type
/dev/sda1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sda9 7814019072 7814035455 16384 8M Solaris reserved 1


Disk /dev/sdb: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NT001-3M21
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 652BCFEA-E77D-D54F-8FC3-6972BE6ECDAF

Device Start End Sectors Size Type
/dev/sdb1 2048 7814019071 7814017024 3.6T Solaris /usr & Apple ZFS
/dev/sdb9 7814019072 7814035455 16384 8M Solaris reserved 1

 

[waldauf]

I forgot to mention I use this M.2 - SATA adapter: Bewinner M.2 to SATA3.0 Expansion Card 6 PortsSATA 3.0.

 

[elvisimprsntr]

Welcome!

1. ODROID-H3+ <-- Not really a suitable platform for a NAS

https://www.amazon.com/Odroid-Quad-Core-Processor-Intel-Jasper/dp/B0BM6ZJFDH

2. Cheap SATA expanders with ASM1166 controller <-- Problem

https://www.amazon.com/dp/B0B6RQHY4F

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

In the last year or two, we've had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use port multipliers, and use cheap SATA controllers only after extensive research. SATA controllers and SATA...

www.truenas.com

You can verify with the following command

find /dev/sd? -exec smartctl -a {} \; | grep -i serial

 

[waldauf]

My goal is to have a small NAS with RAIDZ-2 for my home use (network storage for pictures, documents and backups; without deduplication). First I tried to find a small PC (ideally a SBC) with SATA ports. Finally I found this Odroid H3+ with M.2 - SATA adapter. I like it for it's size. Then I continued with choosing OS NAS. Now I see that I should have included the selection of the OS in my analysis as well.

If you have tips about HW I appreciate your hints for small house use. I thought about Xiwai NGFF NVME M-Key PCI Express to SATA 3.0 6Gbps 5 Ports Adapter Converter Hard Drive Expansion Card JMB585 2280 (should be better compatible with H3+), is it the same situation as Bewinner or better?

Finally - is it feasible to run TrueNAS (SCALE/CORE) on H3+ or it's totally bad idea with unsure stability?

 

[elvisimprsntr]

@waldauf

It's not TrueNAS itself that does not support port multipliers, it is the ZFS file system.

The JMB585 is just another port multiplier, which will likely result in the same behavior and not suitable for ZFS.

Looks like the ODROID-H3+ has two on-board SATA ports. You could try connecting the two drives in your RAIDZ2 array to those, but it looks like you were trying to connect 6 drive total, not including the MMC boot drive (which BTW is not recommended either)

Your other options would be to spin up your own NAS from a non-ZFS based Linux distribution or another NAS distribution, or pick up a new/used OEM NAS appliance to install TrueNAS. Many have installed TrueNAS CORE/SCALE on QNAP NAS appliances, including myself.

 

[somethingweird]

that board has a realtek NIC - that another possible performance issue. Why not an intel ATOM board? or premade from IX system (hehe)

 

[Ericloewe]

The recommended hardware is as it for very good reasons, and every little bit of unsuitable hardware adds a little bit of trouble. And then the troubles multiply. In particular, if you examine the super-cheap stuff, it becomes evident how the price was "optimized". Since storing data is about keeping it all intact, without errors, forever, the margin for error is minimal.

That said, instead of examining the server to find the source of your immediate issue, I'd examine the disks. Duplicate serial numbers are not something any AHCI controller would cause to just two disks... Were these disks bought used by any chance? Just the other day, someone showed up with "refurbished" disks whose SMART data had been tampered with and the serial number reset to zero in the process.

It's not TrueNAS itself that does not support port multipliers, it is the ZFS file system.

This is very inaccurate. ZFS does not care, nor is its role to care. The problems with SATA port multipliers are numerous, however:

1. They're not supported by Intel, which is effectively a death sentence support-wise
2. They're built down to a price to enable the cheapest, crappiest stuff you can hire Super China Happy Sun Technology Shenzhen Limited and Company to build for you. They're not used in any serious equipment, so they're not actually seriously scrutinized.
3. Following from the previous point, any piece of equipment that integrates a port multiplier should automatically be classed as garbage.
4. Given this picture, why should you as a driver author care one bit about proper support for these things?

SSD expanders avoid these issues by:

1. Being part of the spec from the beginning
2. Being pretty expensive and fairly power-hungry ICs
3. Being routinely designed into servers
4. Being broadly supported by the whole ecosystem
   
   

   JMB585 is just another port multiplier,

No, it is not. It is an AHCI SATA controller. The problem with it is that the market is filled with AHCI controllers that don't work very well, from reputable companies (Marvell for instance) and slightly less-reputable ones like JMicron. Figuring out which ones are good is a pain and the market is not conducive to the stable supply chain needed for a serious recommendation.

Cheap SATA expanders with ASM1166 controller <

Same deal. Not a SATA port multiplier. Also not a good idea for the same reasons, though.

 

[waldauf]

I connected 2 random disks to SATA on the H3's board and I still see the same error "non-unique serial numbers disks":
root@truenas[~]# find /dev/sd? -exec smartctl -a {} \; | grep -i serial
Serial Number: WX10D8XB
Serial Number: WX10D91N

Disks are new, not reused.

Just wonder - as new in ZFS - extender M.2-SATA does cause just slow down of RAIDZ-2 or it also can break the data because the controller on the extender doesn't manage to operate 4 SATA simultaneously (2 SATAs will be used from Odroid itself)?

 

[waldauf]

First of all, thank you for your willingness to help.

I spent some time reading recommended articles and some other ones. I also found this guide TrueNAS Community Hardware Guide. I made a few steps back and now I'm thinking in this way:
* I can keep my Odroid as my backup solution with RAID 1 or replace my RPi with it. Or sold it.
* Build my RAIDZ-2 on the Mini ITX system with my 6 SATA disks.

I found these MBs - can I ask you for help with which to choose?

• -A- The most expensive 4 cores: Supermicro MBD-A2SDI-4C-HLN4F-O (url)
• -B- Cheaper 2 cores: Supermicro MBD-A2SDI-2C-HLN4F-O (url)
• -C- AliExpress, cheaper, 4 cores with Pentium N6005: Topton NAS Motherboard N6005/N5105 (url)
• -D- AliExpress, the cheapest: JASPER SEE NAS Motherboard Intel Celeron N5105 (url)

I have also a few additional questions:

1. Is Intel Atom enough for NAS? 2 or 4 cores? I intend to have an economical solution from the perspective of kWh.
2. As I wrote I bought 6x 4 TB and I would like to set them to RAIDZ-2. Is it possible to turn on deduplication with 16 GB RAM? It's not my goal to have dedup enabled but if there is space to do it, so I'm asking myself why not.

 

[waldauf]

I just found out that Pentium N6005 and Celeron N5105 don't support ECC memory. So they are not an option.

 

[somethingweird]

As I wrote I bought 6x 4 TB and I would like to set them to RAIDZ-2. Is it possible to turn on deduplication with 16 GB RAM? It's not my goal to have dedup enabled but if there is space to do it, so I'm asking myself why not.

Your answer is in https://www.truenas.com/docs/references/zfsdeduplication/

 

[waldauf]

Finally, I bought this HW which I hope is sufficient for Home NAS:

• SUPERMICRO 1U 4x3,5" HS (12G) chasis ‚350W
• SUPERMICRO MB s1151 E-21xx,4xDDR4,8xSATA,2xPCI-E,2xM.2 INTEL 4-core
• Xeon E-2224 3.4GHZ/8MB/FCLGA1151/71W
• SAMSUNG 16GB DDR4 2666 1Rx8 ECC UDIMM

MB is possible to extend with additional RAM or change CPU.


Can I ask you for advice on running TrueNAS on Odroid with two disks directly connected to inner SATA connectors? I'm facing the same problem as I had with M.2-SATA extender: "non-unique serial numbers disks"
root@truenas[~]# find /dev/sd? -exec smartctl -a {} \; | grep -i serial
Serial Number: WX10D8XB
Serial Number: WX10D91N

 

[HoneyBadger]

Hey @waldauf

I suspect the issue is actually related to the eMMC on the ODROID board, as it seems to be presenting bootcode sections as multiple physical disks from your fdisk output:

Code:

Disk /dev/mmcblk0: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 34A18C3F-00F0-4DD0-BE0E-24F80B1A924F

Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 6143 2048 1M BIOS boot
/dev/mmcblk0p2 6144 1054719 1048576 512M EFI System
/dev/mmcblk0p3 1054720 61071326 60016607 28.6G Solaris /usr & Apple ZFS


Disk /dev/mmcblk0boot0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mmcblk0boot1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I suspect that the middleware is considering /dev/mmcblk0boot0 and /dev/mmcblk0boot1 to be separate physical devices; but I doubt they'll respond to a SMART query to confirm. Check Storage -> Disks to see if the SERIAL column is populated or blank, which will show you what the middleware sees. Multiple blank lines will trip this error as well. Or for giggles, try booting up TrueNAS on the ODROID with all of the SATA ports disconnected, go into the pool builder, and see if it shows you the same error. I'm betting it will.

 

[waldauf]

Hey @waldauf

I suspect the issue is actually related to the eMMC on the ODROID board, as it seems to be presenting bootcode sections as multiple physical disks from your fdisk output:

Code:

Disk /dev/mmcblk0: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 34A18C3F-00F0-4DD0-BE0E-24F80B1A924F

Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 6143 2048 1M BIOS boot
/dev/mmcblk0p2 6144 1054719 1048576 512M EFI System
/dev/mmcblk0p3 1054720 61071326 60016607 28.6G Solaris /usr & Apple ZFS


Disk /dev/mmcblk0boot0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mmcblk0boot1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I suspect that the middleware is considering /dev/mmcblk0boot0 and /dev/mmcblk0boot1 to be separate physical devices; but I doubt they'll respond to a SMART query to confirm. Check Storage -> Disks to see if the SERIAL column is populated or blank, which will show you what the middleware sees. Multiple blank lines will trip this error as well. Or for giggles, try booting up TrueNAS on the ODROID with all of the SATA ports disconnected, go into the pool builder, and see if it shows you the same error. I'm betting it will.

I confirm that also without connected SATA disks the problem is still there. I replaced Odroid H3+ with Supermicro server so this H3+ will be a backup solution that will not be used by users.

In the end, I can say that the problem wasn't related to M.2-Sata adapter but you warned me about my design which I changed according to your recommendation.

Thank you very much for your help! I can say I'm excited about my new approach to storing/backing up my data with TrueNas.