Redoing my FreeNAS from scratch

[Fongaboo]

Sorry for the lack of Google-fu, but the terms 'rebuilding', 'reconfiguring', 'redoing', and 'factory reset', etc. have a lot of different connotations in the FreeNAS world that I'm having a hard time finding what I'm asking.

So I ran into some snafus that hosed up my FreeNAS for a few days. I tried some of the things that folks suggested, but it came back to life kind of unexpectedly in a way that couldn't conclusively be correlated to what things I tried.

I'm still having some issues, such as most of my Windows clients not being able to browse my SMB shares no matter what I do*. I had set my FreeNAS up to coordinate with an actual Windows Domain Controller on my network and things had been working well. I hadn't the sense to backup my config before things went haywire unfortunately.

When you run FreeNAS alongside a Windows DC, I feel there are too many variables on both sides to solve this mystery Samba issue in place without losing all my hair. So I'd rather just start from factory reset FreeNAS and work my way again through the Active Directory tutorials I had used.

However I am wondering what happens to my zpool in that scenario. It's a 4x2TB RAID-Z1 pool. Can I import it within my fresh FreeNAS configuration and expect it to reconstitute the data in place? If so, am I risking just re-importing the problems that lead me here?

Is it better that I wipe it and start from scratch? Luckily I had the sense to backup all my data onto external drives as I went along. Of course I'd prefer not to because it took days to migrate it over the first time.

My pool currently displays as 'healthy', but I'm wondering about what I'm seeing under Volumes -> zroot -> Volume Status -> Resilver:

What does the 5 under the fourth drive's checksum mean? I wasn't really clear what was happening when boot was held up, as described in my previous post. Does this imply it was just a long resilver that I had to hang in for?

TIA for any advice...

*My actual domain controller is the only machine that can see my FreeNAS's SMB shares, but only if I enable Guest Mode on all of them.

 

[Nick2253]

You never need to completely start from scratch. At worst, you can simply wipe your boot device, and start from a fresh config. Your pool (and therefore your data) should reimport seamlessly.

So I'd rather just start from factory reset FreeNAS and work my way again through the Active Directory tutorials I had used.

PLEASE please please do not follow a random Active Directory guide. All you need to integrate FreeNAS with Active Directory is well documented in the official manual. Many of these guides are outdated at best, or misleading or completely inaccurate at worst.

What does the 5 under the fourth drive's checksum mean?

It means that ZFS corrected 5 checksum errors on this drive. That is not necessarily something to be concerned about, but you want to keep an eye on it. It could have been that you had a bad cluster of sectors, and the low-level checksumming in the HDD was unable to completely repair the data. A great reason to have ZFS, because the ZFS checksumming caught and repaired it! However, if that number keeps growing, plan on replacing the drive (or at least something associated with that drive; it could be a cable or controller issues as well).

 

[Fongaboo]

OK thank you. All really solid advice. I'm gonna give it a try tomorrow when I can dedicate some time.

That drive had been posting some errors about bad sectors to console, but I wasn't sure how to gauge it beyond that. I will definitely keep an eye on it now that I know where to look.

I just wanted to clarify, the FreeNAS manual... its section on Active Directory... does it instruct you how to have it act as a domain controller? or how to get it to play nice with a bona fide free-standing domain controller? I'm trying to do the latter, but I did see that 'Domain Controller' is one of the services that FreeNAS itself can provide.

 

[DrKK]

we should really see the result of:

Code:

smartctl -qnoserial -x /dev/ada3

Please post the output of that in code tags (or link us to a pastebin), so we can see what the deal is with ada3. Make sure you issue the command that I suggest, with the -x.

 

[Fongaboo]

yeah I had tried some variance of smartctl a little while ago, but I didn't know these flags. Thanks!

Code:

smartctl -qnoserial -x /dev/ada3
smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:	 Western Digital Green
Device Model:	 WDC WD20EZRX-00D8PB0
Firmware Version: 80.00A80
User Capacity:	2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:	 512 bytes logical, 4096 bytes physical
Rotation Rate:	5400 rpm
Device is:		In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Thu Feb  8 21:10:39 2018 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
										was completed without error.
										Auto Offline Data Collection: Enabled.
Self-test execution status:	  (   0) The previous self-test routine completed
										without error or no self-test has ever
										been run.
Total time to complete Offline
data collection:				(26160) seconds.
Offline data collection
capabilities:					(0x7b) SMART execute Offline immediate.
										Auto Offline data collection on/off support.
										Suspend Offline collection upon new
										command.
										Offline surface scan supported.
										Self-test supported.
										Conveyance Self-test supported.
										Selective Self-test supported.
SMART capabilities:			(0x0003) Saves SMART data before entering
										power-saving mode.
										Supports SMART auto save timer.
Error logging capability:		(0x01) Error logging supported.
										General Purpose Logging supported.
Short self-test routine
recommended polling time:		(   2) minutes.
Extended self-test routine
recommended polling time:		( 265) minutes.
Conveyance self-test routine
recommended polling time:		(   5) minutes.
SCT capabilities:			  (0x7035) SCT Status supported.
										SCT Feature Control supported.
										SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAGS	VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate	 POSR-K   200   200   051	-	376
  3 Spin_Up_Time			POS--K   174   172   021	-	4266
  4 Start_Stop_Count		-O--CK   070   070   000	-	30123
  5 Reallocated_Sector_Ct   PO--CK   200   200   140	-	0
  7 Seek_Error_Rate		 -OSR-K   200   200   000	-	0
  9 Power_On_Hours		  -O--CK   087   087   000	-	10167
 10 Spin_Retry_Count		-O--CK   100   100   000	-	0
 11 Calibration_Retry_Count -O--CK   100   253   000	-	0
 12 Power_Cycle_Count	   -O--CK   100   100   000	-	62
192 Power-Off_Retract_Count -O--CK   200   200   000	-	18
193 Load_Cycle_Count		-O--CK   176   176   000	-	74978
194 Temperature_Celsius	 -O---K   115   103   000	-	32
196 Reallocated_Event_Count -O--CK   200   200   000	-	0
197 Current_Pending_Sector  -O--CK   200   200   000	-	0
198 Offline_Uncorrectable   ----CK   200   200   000	-	0
199 UDMA_CRC_Error_Count	-O--CK   200   200   000	-	0
200 Multi_Zone_Error_Rate   ---R--   200   200   000	-	1
							||||||_ K auto-keep
							|||||__ C event count
							||||___ R error rate
							|||____ S speed/performance
							||_____ O updated online
							|______ P prefailure warning

ATA_READ_LOG_EXT (addr=0x00:0x00, page=0, n=1) failed: 48-bit ATA commands not implemented for legacy controllers
Read GP Log Directory failed

SMART Log Directory Version 1 [multi-sector log support]
Address	Access  R/W   Size  Description
0x00		   SL  R/O	  1  Log Directory
0x01		   SL  R/O	  1  Summary SMART error log
0x02		   SL  R/O	  5  Comprehensive SMART error log
0x06		   SL  R/O	  1  SMART self-test log
0x09		   SL  R/W	  1  Selective self-test log
0x80-0x9f	  SL  R/W	 16  Host vendor specific log
0xa0-0xa7	  SL  VS	  16  Device vendor specific log
0xa8-0xb7	  SL  VS	   1  Device vendor specific log
0xbd		   SL  VS	   1  Device vendor specific log
0xc0		   SL  VS	   1  Device vendor specific log
0xe0		   SL  R/W	  1  SCT Command/Status
0xe1		   SL  R/W	  1  SCT Data Transfer

SMART Extended Comprehensive Error Log (GP Log 0x03) not supported

SMART Error Log Version: 1
No Errors Logged

SMART Extended Self-test Log (GP Log 0x07) not supported

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline	Completed without error	   00%	  9493		 -
# 2  Extended offline	Completed without error	   00%	  9445		 -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
	1		0		0  Not_testing
	2		0		0  Not_testing
	3		0		0  Not_testing
	4		0		0  Not_testing
	5		0		0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:				  3
SCT Version (vendor specific):	   258 (0x0102)
SCT Support Level:				   1
Device State:						Active (0)
Current Temperature:					32 Celsius
Power Cycle Min/Max Temperature:	 31/37 Celsius
Lifetime	Min/Max Temperature:	 16/44 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:	 2
Temperature Sampling Period:		 1 minute
Temperature Logging Interval:		1 minute
Min/Max recommended Temperature:	  0/60 Celsius
Min/Max Temperature Limit:		   -41/85 Celsius
Temperature History Size (Index):	478 (383)

Index	Estimated Time   Temperature Celsius
 384	2018-02-08 13:13	32  *************
 ...	..(476 skipped).	..  *************
 383	2018-02-08 21:10	32  *************

SCT Error Recovery Control command not supported

Device Statistics (GP/SMART Log 0x04) not supported

ATA_READ_LOG_EXT (addr=0x11:0x00, page=0, n=1) failed: 48-bit ATA commands not implemented for legacy controllers
Read SATA Phy Event Counters failed

 

[DrKK]

Alright, so I'll tell you what I see.

You've got a WD green. This has loading/unloading behavior that is highly suboptimal for a NAS. Sure enough, #193 shows 75000 loading cycles, which is already pretty bad, even though the drive is only 10000 hours old. You've got 30000 starts/stops on the drive. I have no idea how you managed that, that's a start/stop every 20 minutes of service.

I don't care for attribute #200 being non-zero, but that's not the end of the world.

Temperature history looks good.
Clearly, there's not a good enough SMART testing regimen here on the drive, looks like the drive has never had regular SMART short and long tests. You'll definitely want to set that up.

Conclusions:

If this drive were a person, it'd be a 28 year old that is generally in OK shape, but partied way way too hard in college, and will have to watch herself in middle age, and looks older than she actually is already.

Actions:

Set some SMART tests, turn off all power saving features if you are going to insist on using the WD Green---you are wearing them out by using them in a NAS.

 

[Fongaboo]

very insightful... i love analogies! you've given me good stuff to go research.

i'll probably eventually replace the greens, but was just getting a FreeNAS up and running using leftover drives. thanks!

 

[Fongaboo]

Ain't this some shit....

Normally for a desktop you'll do a lot of reading for a short period of time, followed by significant idle time. So getting to a large number of cycles for a desktop is quite difficult. But, on a server things are a bit different. If you start streaming a movie ZFS will read-ahead on the movie. It will probably be more than 8 seconds between reads. This means that the whole time you are watching a movie the drive will be parking and unparking itself. Some people have recorded more than 250k cycles in less than a year. Western Digital knows about all of this stuff, and there are stories of WD denying warranties on drives with excessive cycles because it's an indicator that you used the drive outside its "designed" criteria. Because of this claim they do not have to honor the warranty. Good luck trying to fight them on it too.

So that's how I managed that...

 

[Chris Moore]

There is a utility that I can't recall the name of, I think it is something like, "wdidle2".
It changes the idle timeout so the drive is not constantly parking.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Fongaboo]

There is a utility that I can't recall the name of, I think it is something like, "wdidle2".
It changes the idle timeout so the drive is not constantly parking.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Yup they mentioned that in the article/post I linked. Unfortunately it seems to be for DOS only though. That's ok cuz I should be able to run a SATA to USB adapter to a nearby bare metal Windows machine.

 

[DrKK]

Do not. Ever. Run. A SATA to USB drive in FreeNAS/ZFS. Ever.

 

[wblock]

I think he meant temporarily, to run wdidle on the drive. But that might not work, either, some drive utilities won't work unless the drive is directly connected.

 

[Fongaboo]

Do not. Ever. Run. A SATA to USB drive in FreeNAS/ZFS. Ever.

No I meant I would temporarily disconnect my green drives from SATA on my FreeNAS and connect them to another Windows machine so I could run widdle.exe on them. The USB to SATA adapter will never get connected to the FreeNAS itself.

 

[DrKK]

No I meant I would temporarily disconnect my green drives from SATA on my FreeNAS and connect them to another Windows machine so I could run widdle.exe on them. The USB to SATA adapter will never get connected to the FreeNAS itself.

I wonder if that's even possible. In order to run wdidle, you have to have DOS and very low level access. The very act of connecting a SATA drive through the southbridge probably vitiates that.

 

[Chris Moore]

No I meant I would temporarily disconnect my green drives from SATA on my FreeNAS and connect them to another Windows machine so I could run widdle.exe on them. The USB to SATA adapter will never get connected to the FreeNAS itself.

I agree with what @DrKK is saying here. I have never been able to tinker with (upgrade) the firmware on a SATA drive unless it was directly connected to a real hardware SATA port. I made a point of buying a laptop with an eSATA connector for that purpose.

 

[Adrian]

Some recovery CDs contain wdidle3, such as http://www.ultimatebootcd.com/index.html
This should allow you to operate on your disks in-situ.

There is a 17 page long thread on green (and red) drives at https://forums.freenas.org/index.php?threads/hacking-wd-greens-and-reds-with-wdidle3-exe.18171/

Another approach is to have something running which accesses your drives every 5 seconds. I used to have a python script I'd found somewhere running on FreeBSD ZFS server which did this, but can not find it online, and the server has been powered off for a few years.

 

[DrKK]

If his intent is to replace the greens, which is what he stated, then if he doesn't mind prematurely aging the drives, he is not likely to have problems over the next year or so. A big bunch of hassle, at this point, to try to wdidle the drives is probably not optimal deployment of his temporal cycles.

 

[Fongaboo]

I wonder if that's even possible. In order to run wdidle, you have to have DOS and very low level access. The very act of connecting a SATA drive through the southbridge probably vitiates that.

Oh I see what you mean now.

 

[fracai]

I've used idle3-tools under Debian (actually Raspbian on my Pi) to correct the idle timer.
The trick here is that not all USB-SATA adapters are supported. Or maybe it's just SMART that's not supported by the adapters? I forget exactly, but it is indeed possible. I can look up the adapter that's been successful if anyone is interested. I think it was from Monoprice.

 

[Chris Moore]

I can look up the adapter that's been successful if anyone is interested. I think it was from Monoprice.

I would like to know about a USB adapter that will work with the drive SMART controls. I have been disappointed by several already.