Unexpected partitioning schema

Status
Not open for further replies.

solarisguy

Guru
Joined
Apr 4, 2014
Messages
1,125
My 9.2.1.5 system is in use, I cannot do any non-trivial testing for a very long time.

I did gpart show, and noticed that ada2 and ada3 are partitioned differently:
Code:
[root@freenas] ~# gpart show
=>        34  1953525101  ada0  GPT  (931G)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128          7        - free -  (3.5k)
 
=>        34  1953525101  ada1  GPT  (931G)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128          7        - free -  (3.5k)
 
=>        34  7814037101  ada2  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037007    1  freebsd-zfs  (3.7T)
 
=>        34  7814037101  ada3  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037007    1  freebsd-zfs  (3.7T)
 
=>        34  7814037101  ada4  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037000    1  freebsd-zfs  (3.7T)
  7814037128          7        - free -  (3.5k)
 
=>        34  7814037101  ada5  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037000    1  freebsd-zfs  (3.7T)
  7814037128          7        - free -  (3.5k)
 
=>        34  7814037101  ada6  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037000    1  freebsd-zfs  (3.7T)
  7814037128          7        - free -  (3.5k)
 
=>        34  7814037101  ada7  GPT  (3.7T)
          34          94        - free -  (47k)
        128  7814037000    1  freebsd-zfs  (3.7T)
  7814037128          7        - free -  (3.5k)
 
=>      63  15124929  da0  MBR  (7.2G)
        63  1930257    1  freebsd  [active]  (942M)
  1930320        63      - free -  (31k)
  1930383  1930257    2  freebsd  (942M)
  3860640      3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  11220000      - free -  (5.4G)
 
=>      0  1930257  da0s1  BSD  (942M)
        0      16        - free -  (8.0k)
      16  1930241      1  !0  (942M)
 
=>      34  30282941  da1  GPT  (14G)
        34        94      - free -  (47k)
      128  30282847    1  freebsd-zfs  (14G)
 
[root@freenas] ~#


ada0 and ada1 (WD10JPVX, AF 512e) are connected to a SATA card. Their partitions were created by FreeNAS GUI using system defaults when creating a RAID-1 pool.

Then I changed in the GUI the swap size on each drive to be zero (0), and created without changing anything else a RAID-Z2 pool out of ada2-ada7 drives. These drives are not identical nor they are connected the same.

Intel motherboard with Xeon, 16GB RAM and C222 Intel chipset. ada2 and ada3 are connected using its SATA III ports. ada4-ada7 are connected using SATA II ports (I do not think that is relevant).

ada2-ada3 are WD4001FAEX (Black) with 512 byte sectors.

ada4-ada7 are WD4003FZEX (Black) with 512 bytes logical, 4096 bytes physical = Advanced Format 512e

I am assuming that gpart senses AF 512e drives, why then it does not partition all the disks in RAID-Z2 the same (to smallest common denominator)? Wouldn't it be simpler or at least more meaningful?

My question is not an academic one. I had realized that I might be forced to replace 512-byte sector disks with AF 512e disks. ZFS could be very unhappy? Anyone with real experience of replacing a disk with a smaller one? What would FreeNAS GUI do as far as the partition on the replacement disk are concerned?

Does this qualify for bug report, so gpart would be using the same partition schema on all RAID-Z* disks?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
You can replace a disk with a smaller one assuming that the partitionable space will be larger than the partition size you actually need.

What you are seeing is the result of setting your swap space to zero(which the GUI tells you that you shouldn't do). Your ada4-7 are 3.5kb larger than ada2 and ada3, so you have unpartitioned space. The size of all of the ZFS partitions on ada2 through ada7 are the same size, and use the same number of LBAs, 7814037000.

One of the great reasons to have swap space is that if you have to RMA one of your disks and it happens to be bigger or smaller(which happens and which you've demonstrated very well) you might not be able to add the disk to the pool. You'd probably be a little pissed if you did an RMA and the replacement disk couldn't be used in your pool. Your disk manufacturer won't care but you sure will. Normally we'd tell you to just temporarily shrink the swapsize to 1.5GB or something so you had more slack space for the zfs partition. But in your case it's already zero so that extra headroom is gone.

Now do you see why we tell people not to mess with settings they don't fully appreciate and why, in particular, we tell people NOT to mess with swap space size? We put it in all caps (STRONGLY DISCOURAGED). What more should we do? :P
 

solarisguy

Guru
Joined
Apr 4, 2014
Messages
1,125
gpart is reporting the same capacity for all the 4TB drives
7814037101 sectors/blocks that are 512-byte each = 4000786995712 bytes​
S.M.A.R.T. is reporting for all six drives
4000787030016 = 4000786995712 + 67 * 512

What got allocated to freebsd-zfs partitions is a multiply of 4K sectors, but only for AFe drives (both the ones with swap, and those without)... That is my peewee, and the logic is either built into gparted or FreeNAS. And I have a feeling that gpart is executing FreeNAS logic.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
FreeNAS' code aligns to 4k sectors and only uses 4k sectors for it's setup.

Oh, and to be completely accurate it doesn't need to be the same number of LBAs as I said above. It does need to match the number of available space that is partition-able though.
 
Status
Not open for further replies.
Top