zfs fudge up, urgent

[creepwood]

Something odd has happened.

I have a 4 x 2TB RAIDZ volume and a second 2 x 250GB mirror for my most important stuff. Short story, it seems that one drive from each volume has switched placed with eachother when my NAS rebooted with one sata cable detached from the large volume. Now I'm having trouble that I don't know how to switch them back. I'm backing up the stuff on the smaller volume right now so that I don't risk anything. How am I'm going to about to fix this? the large volume was degraded at the time of reboot. I attached the cable and rebooted and no more warnings, but as far as I can see the disks switched places in the volumes.

I'm a bit desperate because I would like to not loose the stuff on the large volume.

 

[creepwood]

Volume Path Used Available Size Status Available actions
panda /mnt/panda 4.0 TiB (89%) 492.0 GiB 4.5 TiB HEALTHY
mantis /mnt/mantis 172.9 GiB (76%) 53.5 GiB 226.4 GiB HEALTHY

It says that the total size of panda is 4.5TB but I'm sure that it should be 5.2-5.3TB

 

[creepwood]

Still not wiser

it seems that right disks are in the right pools

Code:

 /# camcontrol devlist
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus0 target 0 lun 0 (pass0,ada0)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus1 target 0 lun 0 (pass1,ada1)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus2 target 0 lun 0 (pass2,ada2)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus3 target 0 lun 0 (pass3,ada3)
<SAMSUNG HD154UI 1AG01118>         at scbus4 target 0 lun 0 (pass4,ada4)
<SAMSUNG SP2504C VT100-33>         at scbus6 target 0 lun 0 (pass5,ada5)
<SAMSUNG SP2504C VT100-33>         at scbus8 target 0 lun 0 (pass6,ada6)
<Kingston DT 100 G2 PMAP>          at scbus10 target 0 lun 0 (pass7,da0)

and when I check volume status it says that the following disks are in the volume, ada0p2, ada1p2, ada2p2, ada3p2 are all online in that volume. Then why am I missing 800MB from 5.3 to 4.5 TB? I did a scrub and zpool status said that it did a 512 resilvered finished 40 minutes ago.

 

[creepwood]

It seems I need a scrub?

also when I do a glabel status ada1 doesn't show up

Code:

 /# glabel status
                                      Name  Status  Components
                             ufs/FreeNASs3     N/A  da0s3
                             ufs/FreeNASs4     N/A  da0s4
                    ufsid/4f86f9dd304b8586     N/A  da0s1a
                            ufs/FreeNASs1a     N/A  da0s1a
                            ufs/FreeNASs2a     N/A  da0s2a
gptid/ae64fa0d-ad1e-11e0-a241-0019dbb34e94     N/A  ada0p2
gptid/aefaf439-ad1e-11e0-a241-0019dbb34e94     N/A  ada2p2
gptid/ad3fb90d-ad1e-11e0-a241-0019dbb34e94     N/A  ada3p2
                               msdosfs/EFI     N/A  ada4p1
                  gpt/EFI System Partition     N/A  ada4p1
gptid/83a6ad8a-1586-4780-9ec5-323e8c022505     N/A  ada4p1
                                 gpt/nashi     N/A  ada4p2
gptid/999c0fb6-b3ce-4cde-a1a7-8f1c649d66a3     N/A  ada4p2
gptid/5e9a9fd1-de2e-11e0-8c77-f46d04d60960     N/A  ada6p1
gptid/5ea69424-de2e-11e0-8c77-f46d04d60960     N/A  ada6p2

 

[creepwood]

Appearantly I've been "losing" storage since the beginning. I went back to check my status mails from as far back as I have. March of this year my volume was the size of 4.9TB, today it's 4.5TB. A month back it was 4.6TB. I guess you're losing space as the time goes by?

I'm still concerned that there is something wrong. I got this in

Code:

Checking status of zfs pools:
  pool: panda
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

	NAME                                            STATE     READ WRITE CKSUM
	panda                                           ONLINE       0     0     0
	  raidz1                                        ONLINE       0     0     0
	    gptid/ad3fb90d-ad1e-11e0-a241-0019dbb34e94  ONLINE       0     0     0
	    ada1p2                                      ONLINE       0     0     0
	    gptid/ae64fa0d-ad1e-11e0-a241-0019dbb34e94  ONLINE       0     0     4
	    gptid/aefaf439-ad1e-11e0-a241-0019dbb34e94  ONLINE       0     0     0

I did a scrub and in the status it said someting about 512 resilvered.

Maybe my volume is whole and fine? But why am I losing size from 5.3TB to 4.5TB is a substantial amount. Also that the glabel status not showing ADA01 bothers me.

 

[paleoN]

I'm still concerned that there is something wrong.

Output of:

Code:

smartctl -q noserial -a /dev/ada0

gpart show /dev/ada1

smartctl -q noserial -a /dev/ada1

You should check the SMART info on the rest of your drives as well.

But why am I losing size from 5.3TB to 4.5TB is a substantial amount.

Code:

zpool list panda

zfs list panda

df -h /mnt/panda

 

[creepwood]

nothing obviously wrong except the output from zpool list panda och zfs list panda.

Code:

zpool list panda
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
panda  7.25T  6.48T   785G    89%  ONLINE  /mnt

Code:

zfs list panda
NAME    USED  AVAIL  REFER  MOUNTPOINT
panda  4.85T   501G  4.03T  /mnt/panda

and for df -h //mnt/panda/

Code:

 df -h /mnt/panda
Filesystem    Size    Used   Avail Capacity  Mounted on
panda         4.5T    4.0T    501G    89%    /mnt/panda

 

[creepwood]

What I'm I looking for on the smartctl tests?

This is for ada0

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (Adv. Format)
Device Model:     WDC WD20EARS-00MVWB0
Firmware Version: 51.0AB51
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Oct  1 00:09:11 2012 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (37260) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   227   174   021    Pre-fail  Always       -       3633
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       266
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   100   253   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   087   087   000    Old_age   Always       -       10088
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       193
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       100
193 Load_Cycle_Count        0x0032   156   156   000    Old_age   Always       -       134729
194 Temperature_Celsius     0x0022   109   102   000    Old_age   Always       -       41
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      9789         -
# 2  Extended offline    Interrupted (host reset)      90%      9628         -
# 3  Extended offline    Completed without error       00%      9429         -
# 4  Extended offline    Completed without error       00%      9406         -
# 5  Extended offline    Completed without error       00%      9046         -
# 6  Extended offline    Completed without error       00%      8687         -
# 7  Extended offline    Completed without error       00%      8663         -
# 8  Extended offline    Completed without error       00%      8425         -
# 9  Extended offline    Completed without error       00%      7998         -
#10  Extended offline    Completed without error       00%      6538         -
#11  Extended offline    Completed without error       00%      5796         -
#12  Extended offline    Completed without error       00%      5101         -
#13  Extended offline    Completed without error       00%      4358         -
#14  Extended offline    Completed without error       00%      3617         -
#15  Extended offline    Completed without error       00%      2898         -
#16  Extended offline    Completed without error       00%      2156         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[creepwood@alexandria] /#

ada1

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (Adv. Format)
Device Model:     WDC WD20EARS-00MVWB0
Firmware Version: 51.0AB51
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Oct  1 00:17:49 2012 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (38760) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   225   170   021    Pre-fail  Always       -       3741
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       266
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   100   253   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   087   087   000    Old_age   Always       -       10088
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       193
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       100
193 Load_Cycle_Count        0x0032   155   155   000    Old_age   Always       -       135542
194 Temperature_Celsius     0x0022   110   106   000    Old_age   Always       -       40
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      9789         -
# 2  Extended offline    Completed without error       00%      9430         -
# 3  Extended offline    Completed without error       00%      9406         -
# 4  Extended offline    Completed without error       00%      9047         -
# 5  Extended offline    Completed without error       00%      8687         -
# 6  Extended offline    Completed without error       00%      8663         -
# 7  Extended offline    Completed without error       00%      8258         -
# 8  Extended offline    Completed without error       00%      7831         -
# 9  Extended offline    Completed without error       00%      7090         -
#10  Extended offline    Completed without error       00%      6371         -
#11  Extended offline    Completed without error       00%      5629         -
#12  Extended offline    Completed without error       00%      4934         -
#13  Extended offline    Completed without error       00%      4191         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

ada2

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (Adv. Format)
Device Model:     WDC WD20EARS-00MVWB0
Firmware Version: 51.0AB51
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Oct  1 00:18:31 2012 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x84) Offline data collection activity
                                        was suspended by an interrupting command from host.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (37080) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   226   171   021    Pre-fail  Always       -       3683
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       339
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   080   080   000    Old_age   Always       -       14992
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       267
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       171
193 Load_Cycle_Count        0x0032   152   152   000    Old_age   Always       -       145416
194 Temperature_Celsius     0x0022   111   100   000    Old_age   Always       -       39
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     14693         -
# 2  Extended offline    Completed without error       00%     14333         -
# 3  Extended offline    Completed without error       00%     14309         -
# 4  Extended offline    Completed without error       00%     13951         -
# 5  Extended offline    Completed without error       00%     13591         -
# 6  Extended offline    Completed without error       00%     13567         -
# 7  Extended offline    Completed without error       00%     12568         -
# 8  Extended offline    Completed without error       00%     11827         -
# 9  Extended offline    Completed without error       00%     11109         -
#10  Extended offline    Completed without error       00%     10367         -
#11  Extended offline    Completed without error       00%      9672         -
#12  Extended offline    Completed without error       00%      8929         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

ada3

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (Adv. Format)
Device Model:     WDC WD20EARS-00MVWB0
Firmware Version: 51.0AB51
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Oct  1 00:18:56 2012 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x84) Offline data collection activity
                                        was suspended by an interrupting command from host.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (37500) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   196   170   051    Pre-fail  Always       -       4768
  3 Spin_Up_Time            0x0027   224   168   021    Pre-fail  Always       -       3800
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       290
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   083   083   000    Old_age   Always       -       13118
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       217
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       121
193 Load_Cycle_Count        0x0032   154   154   000    Old_age   Always       -       138199
194 Temperature_Celsius     0x0022   112   105   000    Old_age   Always       -       38
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   197   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     12818         -
# 2  Extended offline    Completed without error       00%     12459         -
# 3  Extended offline    Completed without error       00%     12435         -
# 4  Extended offline    Completed without error       00%     12076         -
# 5  Extended offline    Completed without error       00%     11716         -
# 6  Extended offline    Completed without error       00%     11692         -
# 7  Extended offline    Completed without error       00%     10453         -
# 8  Extended offline    Completed without error       00%      9736         -
# 9  Extended offline    Completed without error       00%      8993         -
#10  Extended offline    Completed without error       00%      8300         -
#11  Extended offline    Completed without error       00%      7558         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[paleoN]

What I'm I looking for on the smartctl tests?

The smartctl tests look fine. The Load_Cycle_Count is higher than I like, but to be expected with your drives. The 4 checksum errors occurred on ada0 and you thought ada1 was a bit off. At which point I like to look at the disks. Both appear fine. I'm assuming the gpart show command gave you the expected output.

You will want to keep an eye on any further checksum errors. I would look into some decent SATA cables that won't fall out as well. Also, your pool is nearly 90% full. I wouldn't want to fill it any more.

nothing obviously wrong except the output from zpool list panda och zfs list panda.

Actually, the output from all three are correct in their own way. It left as an exercise to the reader to understand why, RTFM.

 

[creepwood]

The chk error could've come from when I accidently started the NAS with a cable out (it wasn't bad connection, I just forgot to connect it again. But I'll look into better cables. I ordered a M1015 SAS card yesterday for expanding my NAS.

I understand that 7.25T is all four disks capacity put together, I understand that I'll get a maximum of n-1 of the capacity. I don't understand why it says 4.85T used and .5T available when the size is 4.5T with another command and when I check properties on the whole volume in windows explorer.

Why am I losing capacity over time?

 

[creepwood]

I just deleted around 500GB of data, and my capacity _dropped_ from 4.5Tb to 4.2Tb. what is going on?

 

[Stephens]

Do you have snapshots enabled?

Also, this has nothing to do with your problem, but I'm curious how you got 6 drives working with the Asus E35M1-M.

 

[creepwood]

Doh! I just found an old snapshot hogging some data. Thank you so much for finding out what was weird.

the two extra small drives are just connected to an old PCI sata controller replacing it with M1015 SAS controller and connecting some more drives. I just use that mirror for the most important stuff on my main volume, rsyncing it.

the only thing I'm concerned about now is that glabel status now, that doesn't show ada1 but all the other drives.

 

[paleoN]

the only thing I'm concerned about now is that glabel status now, that doesn't show ada1 but all the other drives.

Have you rebooted the system recently?

I don't understand why it says 4.85T used and .5T available when the size is 4.5T with another command and when I check properties on the whole volume in windows explorer.

zpool shows the total bytes of storage available in the pool.

zfs shows the total bytes of storage available to the filesystem, after
redundancy is taken into account.

du shows the total bytes of storage used by a directory, after compression
and dedupe is taken into account.

"ls -l" shows the total bytes of storage currently used to store a file,
after compression, dedupe, thin-provisioning, sparseness, etc.

And the great, original [zfs-discuss] post.

 

[paleoN]

How important is it that Size doesn't change in the GUI? Currently it is confusing and this question get asked with some regularity.

Are relatively small changes acceptable? If so it would be simple enough to add up zfs list USED & AVAIL columns to get the total size for the root of the pool. AFAIK, this does change a bit as the pool is filled and more so with lots of very small files.

I'm assuming the zpool list SIZE wouldn't be acceptable for raidzN arrays as that shows the inflated size. Unless I'm overlooking something, this is somewhat non-trivial to obtain. It seems to me you would then need to parse the output of zpool status for a given pool and manually add up the space for any raidzN vdevs. If the pool consists of a stripe, i.e. single disk, or mirrors then the actual or deflated space, i.e. usable, is shown already. No parsing needed.

The following pseudo code should work for all cases:

Code:

SIZE=0
For each top level vdev
 SIZE+=(size of the smallest device * num devices) - ( size of the smallest device * parity level)
Next

 

[leenux_tux]

creepwood,

Depending on how and where your deleting your data from, you could actually be putting it into a "windows" like recycle bin (.recycle) . This is (as far as I understand it) only CIFS thing. So if you "delete" from MS Windows what you think you are deleting goes into this folder (.recycle) and the amount of disk space doesn't change.

Also, 2GB from each disk is automatically taken by FreeNAS for page space. And if you have snapshots setup these can skew what your seeing as well.

 

[creepwood]

The size problem is solved, last scrub without any errors.

Glabel status now is missing one more drive, one of the mirror drives.