SOLVED Slow resilvering speeds

Status
Not open for further replies.
C

craigdt

Explorer
Joined
Mar 10, 2014
Messages
74
Hi All,

I'm experiencing slow resilvering speeds.

I replacing an old 2tb WD Green which had failed with a new Seagate Ironwolf 6tb.

In the past when I had replaced a 2x 2tb WD Green with 2tb WD Red, the resilving speeds were quite fast and if memory serves me, it only took something like 10-12 hours.

I'm planning to replace all the 2tb drives now with Ironwolf 6tb to upgrade the capacity of the NAS.

Note the raidz2-0 pool consist of all WD Red Pro 3tb drives, I've not had any problems with these drives since I've had them.

As you will see in my hdd smart results below, the raidz2-1 pool consists of 3x 2tb WD Green drives and 2x tb WD Red drives and the new 6tb Seagate Ironwolf drive.

I'm looking for advice on how to narrow down why the speeds are slow.

Hardware spec's for the main rig is in my signature

The resilvering process has taken about 7.5 hours now with the following progress
Code:
[root@mainnas] ~# zpool status
  pool: Volume
state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Jun  3 11:50:35 2017
		4.67T scanned out of 20.1T at 186M/s, 24h10m to go
		515G resilvered, 23.26% done
config:

	NAME											STATE	 READ WRITE CKSUM
	Volume										  ONLINE	   0	 0	 0
	  raidz2-0									  ONLINE	   0	 0	 0
		gptid/c4b0da39-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
		gptid/c532471d-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
		gptid/c5b28cc7-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
		gptid/c632bff9-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
		gptid/c6b39932-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
		gptid/c740081c-3049-11e4-a873-002590f0cf92  ONLINE	   0	 0	 0
	  raidz2-1									  ONLINE	   0	 0	 0
		gptid/e7a6c71d-77de-11e5-9f8b-002590f0cf92  ONLINE	   0	 0	 0
		gptid/e8e50969-77de-11e5-9f8b-002590f0cf92  ONLINE	   0	 0	 0
		gptid/e9e4d3b7-77de-11e5-9f8b-002590f0cf92  ONLINE	   0	 0	 0
		gptid/003cde3e-47ff-11e7-8632-002590f0cf92  ONLINE	   0	 0	 0  (resilvering)
		gptid/73f8ac33-7f81-11e5-9b67-002590f0cf92  ONLINE	   0	 0	 0
		gptid/ec9f40a7-77de-11e5-9f8b-002590f0cf92  ONLINE	   0	 0	 0

errors: No known data errors


The graphs for the drives in raidz2-1 show the performance of the drives:
Code:
[Admin@mainnas] ~% iostat ada0 ada1 ada2 ada3 da4 da5
	   tty			ada0			 ada1			 ada2			 ada3			  da4			  da5			 cpu
tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   2	35 56.50 365 20.11  74.84 272 19.90  55.68 370 20.12  51.71 399 20.16  55.26 373 20.12  51.67 400 20.16   0  0  2  0 97
 
C

craigdt

Explorer
Joined
Mar 10, 2014
Messages
74
The following smart status for each of the drives involved with the resilver process are:
Code:
root@mainnas] ~# smartctl -a /dev/ada0
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Green
Device Model:	 WDC WD20EARX-008FB0
Serial Number:	WD-WCAZAJ517340
LU WWN Device Id: 5 0014ee 20793559e
Firmware Version: 51.0AB51
User Capacity:	2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Sat Jun  3 18:57:45 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x85)	Offline data collection activity
					was aborted by an interrupting command from host.
					Auto Offline Data Collection: Enabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (30600) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   2) minutes.
Extended self-test routine
recommended polling time:	  ( 329) minutes.
Conveyance self-test routine
recommended polling time:	  (   5) minutes.
SCT capabilities:			(0x30b5)	SCT Status supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x002f   200   200   051	Pre-fail  Always	   -	   0
  3 Spin_Up_Time			0x0027   192   184   021	Pre-fail  Always	   -	   5383
  4 Start_Stop_Count		0x0032   100   100   000	Old_age   Always	   -	   141
  5 Reallocated_Sector_Ct   0x0033   200   200   140	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x002e   100   253   000	Old_age   Always	   -	   0
  9 Power_On_Hours		  0x0032   059   059   000	Old_age   Always	   -	   30357
10 Spin_Retry_Count		0x0032   100   100   000	Old_age   Always	   -	   0
11 Calibration_Retry_Count 0x0032   100   100   000	Old_age   Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   141
192 Power-Off_Retract_Count 0x0032   200   200   000	Old_age   Always	   -	   74
193 Load_Cycle_Count		0x0032   187   187   000	Old_age   Always	   -	   40849
194 Temperature_Celsius	 0x0022   130   106   000	Old_age   Always	   -	   20
196 Reallocated_Event_Count 0x0032   200   200   000	Old_age   Always	   -	   0
197 Current_Pending_Sector  0x0032   200   200   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0030   200   200   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x0032   200   200   000	Old_age   Always	   -	   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000	Old_age   Offline	  -	   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline	Completed without error	   00%	 30341		 -
# 2  Short offline	   Completed without error	   00%	 29934		 -
# 3  Extended offline	Completed without error	   00%	 29917		 -
# 4  Short offline	   Completed without error	   00%	 29599		 -
# 5  Extended offline	Completed without error	   00%	 29581		 -
# 6  Short offline	   Completed without error	   00%	 29219		 -
# 7  Extended offline	Completed without error	   00%	 29202		 -
# 8  Short offline	   Completed without error	   00%	 28883		 -
# 9  Extended offline	Completed without error	   00%	 28865		 -
#10  Extended offline	Completed without error	   00%	 28510		 -
#11  Short offline	   Completed without error	   00%	 28195		 -
#12  Extended offline	Completed without error	   00%	 28178		 -
#13  Short offline	   Completed without error	   00%	 27860		 -
#14  Extended offline	Completed without error	   00%	 27842		 -
#15  Short offline	   Completed without error	   00%	 27524		 -
#16  Extended offline	Completed without error	   00%	 27507		 -
#17  Short offline	   Completed without error	   00%	 27117		 -
#18  Extended offline	Completed without error	   00%	 27099		 -
#19  Short offline	   Completed without error	   00%	 26850		 -
#20  Extended offline	Completed without error	   00%	 26832		 -
#21  Short offline	   Completed without error	   00%	 26482		 -

Code:
[root@mainnas] ~# smartctl -a /dev/ada1
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:	 ST6000VN0041-2EL11C
Serial Number:	ZA1780PD
LU WWN Device Id: 5 000c50 0a2b63085
Firmware Version: SC61
User Capacity:	6,001,175,126,016 bytes [6.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Rotation Rate:	7200 rpm
Form Factor:	  3.5 inches
Device is:		Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Sat Jun  3 19:00:45 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)	Offline data collection activity
					was completed without error.
					Auto Offline Data Collection: Enabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (  575) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   1) minutes.
Extended self-test routine
recommended polling time:	  ( 593) minutes.
Conveyance self-test routine
recommended polling time:	  (   2) minutes.
SCT capabilities:			(0x50bd)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x000f   077   068   044	Pre-fail  Always	   -	   44721288
  3 Spin_Up_Time			0x0003   096   096   000	Pre-fail  Always	   -	   0
  4 Start_Stop_Count		0x0032   100   100   020	Old_age   Always	   -	   2
  5 Reallocated_Sector_Ct   0x0033   100   100   010	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x000f   061   060   045	Pre-fail  Always	   -	   1295693
  9 Power_On_Hours		  0x0032   100   100   000	Old_age   Always	   -	   7 (24 190 0)
10 Spin_Retry_Count		0x0013   100   100   097	Pre-fail  Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   020	Old_age   Always	   -	   2
184 End-to-End_Error		0x0032   100   100   099	Old_age   Always	   -	   0
187 Reported_Uncorrect	  0x0032   100   100   000	Old_age   Always	   -	   0
188 Command_Timeout		 0x0032   100   100   000	Old_age   Always	   -	   0
189 High_Fly_Writes		 0x003a   100   100   000	Old_age   Always	   -	   0
190 Airflow_Temperature_Cel 0x0022   074   071   040	Old_age   Always	   -	   26 (Min/Max 20/28)
191 G-Sense_Error_Rate	  0x0032   100   100   000	Old_age   Always	   -	   74
192 Power-Off_Retract_Count 0x0032   100   100   000	Old_age   Always	   -	   0
193 Load_Cycle_Count		0x0032   100   100   000	Old_age   Always	   -	   3
194 Temperature_Celsius	 0x0022   026   040   000	Old_age   Always	   -	   26 (0 17 0 0 0)
195 Hardware_ECC_Recovered  0x001a   077   031   000	Old_age   Always	   -	   44721288
197 Current_Pending_Sector  0x0012   100   100   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0010   100   100   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x003e   200   200   000	Old_age   Always	   -	   0
240 Head_Flying_Hours	   0x0000   100   253   000	Old_age   Offline	  -	   7 (81 111 0)
241 Total_LBAs_Written	  0x0000   100   253   000	Old_age   Offline	  -	   1069153028
242 Total_LBAs_Read		 0x0000   100   253   000	Old_age   Offline	  -	   28148

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

Code:
[root@mainnas] ~# smartctl -a /dev/ada2
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Green
Device Model:	 WDC WD20EARX-008FB0
Serial Number:	WD-WCAZAJ545700
LU WWN Device Id: 5 0014ee 25ce86a67
Firmware Version: 51.0AB51
User Capacity:	2,000,397,852,160 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Sat Jun  3 19:01:56 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x85)	Offline data collection activity
					was aborted by an interrupting command from host.
					Auto Offline Data Collection: Enabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (30480) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   2) minutes.
Extended self-test routine
recommended polling time:	  ( 328) minutes.
Conveyance self-test routine
recommended polling time:	  (   5) minutes.
SCT capabilities:			(0x30b5)	SCT Status supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x002f   200   200   051	Pre-fail  Always	   -	   1
  3 Spin_Up_Time			0x0027   189   183   021	Pre-fail  Always	   -	   5533
  4 Start_Stop_Count		0x0032   100   100   000	Old_age   Always	   -	   298
  5 Reallocated_Sector_Ct   0x0033   200   200   140	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x002e   100   253   000	Old_age   Always	   -	   0
  9 Power_On_Hours		  0x0032   077   077   000	Old_age   Always	   -	   16909
10 Spin_Retry_Count		0x0032   100   100   000	Old_age   Always	   -	   0
11 Calibration_Retry_Count 0x0032   100   100   000	Old_age   Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   290
192 Power-Off_Retract_Count 0x0032   200   200   000	Old_age   Always	   -	   98
193 Load_Cycle_Count		0x0032   200   200   000	Old_age   Always	   -	   323
194 Temperature_Celsius	 0x0022   130   109   000	Old_age   Always	   -	   20
196 Reallocated_Event_Count 0x0032   200   200   000	Old_age   Always	   -	   0
197 Current_Pending_Sector  0x0032   200   200   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0030   200   200   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x0032   200   200   000	Old_age   Always	   -	   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000	Old_age   Offline	  -	   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline	   Completed without error	   00%	 16892		 -
# 2  Extended offline	Completed without error	   00%	 16891		 -
# 3  Short offline	   Completed without error	   00%	 16487		 -
# 4  Extended offline	Completed without error	   00%	 16469		 -
# 5  Short offline	   Completed without error	   00%	 16151		 -
# 6  Extended offline	Completed without error	   00%	 16133		 -
# 7  Short offline	   Completed without error	   00%	 15771		 -
# 8  Extended offline	Completed without error	   00%	 15754		 -
# 9  Short offline	   Completed without error	   00%	 15436		 -
#10  Extended offline	Completed without error	   00%	 15417		 -
#11  Extended offline	Completed without error	   00%	 15062		 -
#12  Short offline	   Completed without error	   00%	 14748		 -
#13  Extended offline	Completed without error	   00%	 14730		 -
#14  Short offline	   Completed without error	   00%	 14412		 -
#15  Extended offline	Completed without error	   00%	 14394		 -
#16  Short offline	   Completed without error	   00%	 14076		 -
#17  Extended offline	Completed without error	   00%	 14059		 -
#18  Short offline	   Completed without error	   00%	 13669		 -
#19  Extended offline	Completed without error	   00%	 13651		 -
#20  Short offline	   Completed without error	   00%	 13402		 -
#21  Extended offline	Completed without error	   00%	 13384		 -
 
C

craigdt

Explorer
Joined
Mar 10, 2014
Messages
74
Continuation of smart results (hit character limit on original post)
Code:
[root@mainnas] ~# smartctl -a /dev/ada3
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Red
Device Model:	 WDC WD20EFRX-68EUZN0
Serial Number:	WD-WCC4M1RLS7JZ
LU WWN Device Id: 5 0014ee 20c760e56
Firmware Version: 82.00A82
User Capacity:	2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Rotation Rate:	5400 rpm
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Sat Jun  3 19:02:54 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (27600) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   2) minutes.
Extended self-test routine
recommended polling time:	  ( 279) minutes.
Conveyance self-test routine
recommended polling time:	  (   5) minutes.
SCT capabilities:			(0x703d)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x002f   200   200   051	Pre-fail  Always	   -	   0
  3 Spin_Up_Time			0x0027   174   171   021	Pre-fail  Always	   -	   4275
  4 Start_Stop_Count		0x0032   100   100   000	Old_age   Always	   -	   59
  5 Reallocated_Sector_Ct   0x0033   200   200   140	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x002e   100   253   000	Old_age   Always	   -	   0
  9 Power_On_Hours		  0x0032   082   082   000	Old_age   Always	   -	   13260
10 Spin_Retry_Count		0x0032   100   253   000	Old_age   Always	   -	   0
11 Calibration_Retry_Count 0x0032   100   253   000	Old_age   Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   59
192 Power-Off_Retract_Count 0x0032   200   200   000	Old_age   Always	   -	   18
193 Load_Cycle_Count		0x0032   200   200   000	Old_age   Always	   -	   198
194 Temperature_Celsius	 0x0022   124   111   000	Old_age   Always	   -	   23
196 Reallocated_Event_Count 0x0032   200   200   000	Old_age   Always	   -	   0
197 Current_Pending_Sector  0x0032   200   200   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0030   100   253   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x0032   200   200   000	Old_age   Always	   -	   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000	Old_age   Offline	  -	   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline	   Completed without error	   00%	 13243		 -
# 2  Extended offline	Completed without error	   00%	 13230		 -
# 3  Short offline	   Completed without error	   00%	 12839		 -
# 4  Extended offline	Completed without error	   00%	 12820		 -
# 5  Short offline	   Completed without error	   00%	 12503		 -
# 6  Extended offline	Completed without error	   00%	 12485		 -
# 7  Short offline	   Completed without error	   00%	 12123		 -
# 8  Extended offline	Completed without error	   00%	 12105		 -
# 9  Short offline	   Completed without error	   00%	 11788		 -
#10  Extended offline	Completed without error	   00%	 11768		 -
#11  Extended offline	Completed without error	   00%	 11414		 -
#12  Short offline	   Completed without error	   00%	 11100		 -
#13  Extended offline	Completed without error	   00%	 11081		 -
#14  Short offline	   Completed without error	   00%	 10764		 -
#15  Extended offline	Completed without error	   00%	 10746		 -
#16  Short offline	   Completed without error	   00%	 10428		 -
#17  Extended offline	Completed without error	   00%	 10410		 -
#18  Short offline	   Completed without error	   00%	 10021		 -
#19  Extended offline	Completed without error	   00%	 10002		 -
#20  Short offline	   Completed without error	   00%	  9754		 -
#21  Extended offline	Completed without error	   00%	  9736		 -

Code:
[root@mainnas] ~# smartctl -a /dev/da4
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Red
Device Model:	 WDC WD20EFRX-68AX9N0
Serial Number:	WD-WMC300422651
LU WWN Device Id: 5 0014ee 003701d7c
Firmware Version: 80.00A80
User Capacity:	2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:	Sat Jun  3 19:03:53 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (27600) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   2) minutes.
Extended self-test routine
recommended polling time:	  ( 279) minutes.
Conveyance self-test routine
recommended polling time:	  (   5) minutes.
SCT capabilities:			(0x70bd)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x002f   200   200   051	Pre-fail  Always	   -	   0
  3 Spin_Up_Time			0x0027   173   171   021	Pre-fail  Always	   -	   4316
  4 Start_Stop_Count		0x0032   100   100   000	Old_age   Always	   -	   126
  5 Reallocated_Sector_Ct   0x0033   200   200   140	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x002e   100   253   000	Old_age   Always	   -	   0
  9 Power_On_Hours		  0x0032   061   061   000	Old_age   Always	   -	   29138
10 Spin_Retry_Count		0x0032   100   100   000	Old_age   Always	   -	   0
11 Calibration_Retry_Count 0x0032   100   100   000	Old_age   Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   126
192 Power-Off_Retract_Count 0x0032   200   200   000	Old_age   Always	   -	   124
193 Load_Cycle_Count		0x0032   200   200   000	Old_age   Always	   -	   1
194 Temperature_Celsius	 0x0022   123   101   000	Old_age   Always	   -	   24
196 Reallocated_Event_Count 0x0032   200   200   000	Old_age   Always	   -	   0
197 Current_Pending_Sector  0x0032   200   200   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0030   100   253   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x0032   200   200   000	Old_age   Always	   -	   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000	Old_age   Offline	  -	   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline	   Completed without error	   00%	 29120		 -
# 2  Extended offline	Completed without error	   00%	 29115		 -
# 3  Short offline	   Completed without error	   00%	 28715		 -
# 4  Extended offline	Completed without error	   00%	 28697		 -
# 5  Short offline	   Completed without error	   00%	 28379		 -
# 6  Extended offline	Completed without error	   00%	 28361		 -
# 7  Short offline	   Completed without error	   00%	 28000		 -
# 8  Extended offline	Completed without error	   00%	 27981		 -
# 9  Short offline	   Completed without error	   00%	 27664		 -
#10  Extended offline	Completed without error	   00%	 27645		 -
#11  Extended offline	Completed without error	   00%	 27290		 -
#12  Short offline	   Completed without error	   00%	 26976		 -
#13  Extended offline	Completed without error	   00%	 26958		 -
#14  Short offline	   Completed without error	   00%	 26640		 -
#15  Extended offline	Completed without error	   00%	 26622		 -
#16  Short offline	   Completed without error	   00%	 26305		 -
#17  Extended offline	Completed without error	   00%	 26286		 -
#18  Short offline	   Completed without error	   00%	 25897		 -
#19  Extended offline	Completed without error	   00%	 25879		 -
#20  Short offline	   Completed without error	   00%	 25630		 -
#21  Extended offline	Completed without error	   00%	 25612		 -

Code:
[root@mainnas] ~# smartctl -a /dev/da5
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Green
Device Model:	 WDC WD20EZRX-00DC0B0
Serial Number:	WD-WMC1T2313721
LU WWN Device Id: 5 0014ee 6adc84e73
Firmware Version: 80.00A80
User Capacity:	2,000,397,852,160 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:	Sat Jun  3 19:04:21 2017 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x85)	Offline data collection activity
					was aborted by an interrupting command from host.
					Auto Offline Data Collection: Enabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (27720) seconds.
Offline data collection
capabilities:			  (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	  (   2) minutes.
Extended self-test routine
recommended polling time:	  ( 280) minutes.
Conveyance self-test routine
recommended polling time:	  (   5) minutes.
SCT capabilities:			(0x70b5)	SCT Status supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x002f   200   200   051	Pre-fail  Always	   -	   0
  3 Spin_Up_Time			0x0027   184   178   021	Pre-fail  Always	   -	   5766
  4 Start_Stop_Count		0x0032   100   100   000	Old_age   Always	   -	   245
  5 Reallocated_Sector_Ct   0x0033   200   200   140	Pre-fail  Always	   -	   0
  7 Seek_Error_Rate		 0x002e   100   253   000	Old_age   Always	   -	   0
  9 Power_On_Hours		  0x0032   078   078   000	Old_age   Always	   -	   16456
10 Spin_Retry_Count		0x0032   100   100   000	Old_age   Always	   -	   0
11 Calibration_Retry_Count 0x0032   100   100   000	Old_age   Always	   -	   0
12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   244
192 Power-Off_Retract_Count 0x0032   200   200   000	Old_age   Always	   -	   139
193 Load_Cycle_Count		0x0032   083   083   000	Old_age   Always	   -	   351054
194 Temperature_Celsius	 0x0022   128   110   000	Old_age   Always	   -	   22
196 Reallocated_Event_Count 0x0032   200   200   000	Old_age   Always	   -	   0
197 Current_Pending_Sector  0x0032   200   200   000	Old_age   Always	   -	   0
198 Offline_Uncorrectable   0x0030   200   200   000	Old_age   Offline	  -	   0
199 UDMA_CRC_Error_Count	0x0032   200   200   000	Old_age   Always	   -	   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000	Old_age   Offline	  -	   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline	   Completed without error	   00%	 16438		 -
# 2  Extended offline	Completed without error	   00%	 16424		 -
# 3  Short offline	   Completed without error	   00%	 16033		 -
# 4  Extended offline	Completed without error	   00%	 16014		 -
# 5  Short offline	   Completed without error	   00%	 15697		 -
# 6  Extended offline	Completed without error	   00%	 15678		 -
# 7  Short offline	   Completed without error	   00%	 15318		 -
# 8  Extended offline	Completed without error	   00%	 15299		 -
# 9  Short offline	   Completed without error	   00%	 14982		 -
#10  Extended offline	Completed without error	   00%	 14962		 -
#11  Extended offline	Completed without error	   00%	 14608		 -
#12  Short offline	   Completed without error	   00%	 14294		 -
#13  Extended offline	Completed without error	   00%	 14275		 -
#14  Short offline	   Completed without error	   00%	 13959		 -
#15  Extended offline	Completed without error	   00%	 13940		 -
#16  Short offline	   Completed without error	   00%	 13623		 -
#17  Extended offline	Completed without error	   00%	 13604		 -
#18  Short offline	   Completed without error	   00%	 13215		 -
#19  Extended offline	Completed without error	   00%	 13196		 -
#20  Short offline	   Completed without error	   00%	 12949		 -
#21  Extended offline	Completed without error	   00%	 12930		 -
 
SweetAndLow

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
How full is the pool? Great post with good information btw. One thing to keep in mind is signatures are not visible to people on Mobile platforms. Posting that info in the thread is best.

Resilver are slower as the pool is more full. The speeds it reports can also be ignored it's not really accurate.

Sent from my Nexus 5X using Tapatalk
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
It will also speed up as time goes on because there's a lot of metadata at the beginning. Fragmentation is also going to affect resilver speeds.

There is work underway that should improve scrub/resilver performance by not having to jump around as much.
 
C

craigdt

Explorer
Joined
Mar 10, 2014
Messages
74
SweetAndLow and Ericloewe, thanks for the information, it seems you are correct, the resilver is finished and the results are:
resilvered 1.62T in 17h57m with 0 errors on Sun Jun 4 05:47:38 2017

Which puts it over 90meg/sec

According the zpool list, the fragmentation is 22%

Once again, thanks for alleviating my fears that I may have had a hardware problem :)
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
craigdt said:
According the zpool list, the fragmentation is 22%
Click to expand...
That number isn't very meaningful in this case, it's just a measure of free space fragmentation. It's a shortcoming of the available tools, but there isn't a good way to quantify just how bad fragmentation is in the traditional way.
 
Status
Not open for further replies.

Similar threads

H
  • Locked
Aoc-sat2-mv8 slow?
Replies
5
Views
4K
hibble
H
C
  • Locked
10GbE Read/Write Performance and Replication Performance Advice
Replies
6
Views
1K
Allan Jude
Allan Jude
jafin
  • Locked
Replace drive, now showing multiple resilvering actions expected only 1 (v9.3.1)
Replies
4
Views
972
jafin
jafin
D
  • Locked
State degraded but Smart is okay, what to do?
Replies
5
Views
2K
Bidule0hm
Bidule0hm
Vincent Saelzler
iSCSI Performance: Par for the Course?
Replies
5
Views
3K
Vincent Saelzler
Vincent Saelzler
Top