Boot Disk

[capa]

Hi all,

I have been building my first FreeNAS following the general recommendations found on this site and in the forum. All went actually pretty well. There's just one thing that still gives me some headaches. I am still to find the correct configuration for the boot disk(s). I started with two SANDisk SSD Plus 2.5’’ and tried to set up a mirrored boot configuration from the beginning. It never worked. With one exception I only managed to install FreeNAS once on the SANDisk SSD Plus 2.5''! Frustrated I turned to an 10 year old Samsung disk with 250 GB. With it, it always worked and I never experienced any errors (so far).
I returned both my SANDisk SSD Plus 2.5'' an ordered one WD Green 3D Nand. With this the installation was unproblematic. However, after a week the first scrub run on the boot device detected a couple of corrupted files. I managed to reinstall FreeNAS, only this time it also took me two attempts. Then, yesterday, I ran a verify installation on the bootdisk only to find out, that there are corrupted files... Presently, I have put back my old SAMSUNG disk and everything is fine again. But I am still puzzled as to what boot disk I should use. A lot of users seem to go for the usb thumb drives, an option I discarded because using SSD was the preferred option according to the hardware guide. So, which SSD should I use, and why did I experience such problems with the ones I tried. Thank you very much for your help.

Referenced Parts:
SANDisk SSD Plus 2.5''
WD Green 3D Nand
2.5'' SAMSUNG SpinPoint M5S 250 GB (sold out)

 

[BigDave]

So, which SSD should I use, and why did I experience such problems with the ones I tried. Thank you very much for your help.

Since you have left out a few details, please look over the Forum Rules (linked for your convenience),
and follow the suggestions in that post. I would like to help you, but you have to give me more info
so that I can "see" what you have done AND also what you have not done.

1) Is your boot drive connected directly to the first SATA port (SATA0) on your motherboard?
2) Have you tested the SSD drives at any time before attempting to install FreeNAS?
3) Have you read the User Guide and followed the steps for installing FreeNAS?
4) What are you trying to accomplish by the mirroring the SSD boot drive?

 

[capa]

Hi BigDave, thank you very much for your reply. Actually, yes, I have read the Forum Rules and I thought that I had provided all the information needed (my hardware configuration is in the signature). Just to avoid any misunderstandings, I managed to install FreeNAS successfully, setup a RAIDZ2, a few shares and a PlexMedia Server. This is all fine. But I keep getting corrupted files on the boot drive, either from the beginning, i.e. I can successfully install FreeNAS but on the first boot I get already corrupted files (see attached fotos), or I can successfully install it and boot it but then later (after a week) I again get corrupted files. This happened with three different SSDs. I'm rather confident that the problem lies with the SSD since I have not experienced any such problems with my 10 year old SAMSUNG HDD. Sorry, for repeating myself somehow. Now, back to your questions:

1. Yes, the boot drive is directly connected to the first SATA (see phote SATA_Connections).
2. No, I have not tested the SSD before installing the FreeNAS. Then again, this would only have shown that the SSD have some problem, and I should not use them. Right? This would still leave me to wonder which SSD I should use instead. Or am I missing something?
3. Yes, I did.
4. I'm just following a recommendation I read in one of the guides and/or in the forums. My understanding is that once you have setup successfully a mirrored boot device the boot device could self-heal itself. At the moment, when corrupted files are detected, the only option I have is to reinstall FreeNAS. Looking through posts for the problem of corrupted files on the Boot Device this seems to be the general way to treat such problems.

Thanks again for your help.

 

[BigDave]

No, I have not tested the SSD before installing the FreeNAS. Then again, this would only have shown that the SSD have some problem, and I should not use them. Right? This would still leave me to wonder which SSD I should use instead. Or am I missing something?

I can see your thinking on this, but what if the testing of the SSD (before) use reveals that the drive is 100% good and fully functional. If this happens to be the case, then you have to look at other possible causes of file corruption. In my opinion, the something you are missing... is you seem to be hung up on the assumption that you've had three "bad" SSDs and I can say (with a great deal of confidence) that the odds of that occuring are very low. If you still had the SSDs, my recommendation would be to test them, but if they have been returned, then you obviously can't do that. I have no explanation as to why your 10 year old hard drive is error free, but users will tend to recommend what has worked for them. I run old used Intel X25e SSDs in 32GB and 64GB sizes (wothout a single failure) but I have no idea if you will have access to these in Switzerland.
Good Luck.

 

[Redcoat]

I also cannot tell you why the SSDs you tried did not work out. But I do have some comments.

I and others here (notably @Chris Moore who has written about it may times) have used, or are using, "old" laptop HDDs as boot drives. I keep a couple in reserve in the case that I need them, as I have moved from the used HDD to an SSD. I first used a laptop HDD from my stock as a boot drive when I experienced unexpectedly short life of USB sticks as I commissioned my backup/test machine. Only possible downside of the HDD over the SSD that I saw was that boot was slightly slower - but this is a server application, so I did not care. If you are happy with an HDD as boot drive you might just consider keeping it in place.

Many of us using SSDs or SATADOMs do not mirror those drives - the reliability proves to be so high that the mirror is not really necessary given that it is so easy to reload FreeNAS and one's saved configuration.

You seem to be under a misapprehension about the mirroring of the boot drive - it is not a "failover" configuration, nor will it "self-heal". If you do have a mirrored boot and the active drive becomes corrupted that corruption will be reported (as you have apparently seen), the bad one must be identified and inactivated (for removal/replacement) and the good one configured to be the active boot.

 

[Chris Moore]

It is a "best practice" to include the hardware details in the post because we can't see the signature information on Tapatalk. Also, it preserves the system as it was when the question is asked because the signature will continue to change in the future as you update your account.

I have seen a couple of other issues with SSD drives not being recognized. It makes me think that there might be a driver issue that is only a problem with the newest hardware.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Stux]

The strange thing is that modern new SSDs should not be corrupting like this.

OP has ECC, so that’s weird :-/

 

[capa]

I ordered another SSD Disk (ADATA SP920 128 GB) and I was able to install FreeNAS without any problem. I got this far also with the WD Green 3D Nand, but never (with one exception) with the SANDisk SSD Plus 2.5''. So, this let's me believe that there might be some differences in the SSDs. Anyway, the only thing I changed this time round was that I set the SSD in the BIOS to "SSD", instead of "HDD". I haven't investigated yet whether this is something one should do, and what the impact is of not doing it, but it just seemed sensible, so I did it. So, from an installation point of view the SDD looks alright. I have also performed some SMART-tests, and those confirm that the disk are OK. So, we can establish with quite some degree of confidence that this SSD is good (so far).

Code:

smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:	 ADATA SP920SS
Serial Number:	8H1320013945
LU WWN Device Id: 5 707c18 700287204
Firmware Version: 1.14
User Capacity:	128,035,675,648 bytes [128 GB]
Sector Size:	  512 bytes logical/physical
Rotation Rate:	Solid State Device
Device is:		Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:	Fri Jun  1 19:06:31 2018 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x02)	Offline data collection activity
					was completed without error.
					Auto Offline Data Collection: Disabled.
Self-test execution status:	  (   0)	The previous self-test routine completed
					without error or no self-test has ever
					been run.
Total time to complete Offline
data collection:		 (	0) seconds.
Offline data collection
capabilities:			 (0x15) SMART execute Offline immediate.
					No Auto Offline data collection support.
					Abort Offline collection upon new
					command.
					No Offline surface scan supported.
					Self-test supported.
					No Conveyance Self-test supported.
					No Selective Self-test supported.
SMART capabilities:			(0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:		(0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine
recommended polling time:	 (   2) minutes.
Extended self-test routine
recommended polling time:	 (  10) minutes.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME		  FLAG	 VALUE WORST THRESH TYPE	  UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate	 0x000f   100   100   000	Pre-fail  Always	   -	   804
  5 Reallocated_Sector_Ct   0x0033   100   100   003	Pre-fail  Always	   -	   0
  9 Power_On_Hours		  0x0032   100   100   000	Old_age   Always	   -	   49
 12 Power_Cycle_Count	   0x0032   100   100   000	Old_age   Always	   -	   3
174 Unknown_Attribute	   0x0030   100   100   000	Old_age   Offline	  -	   2
177 Wear_Leveling_Count	 0x0000   100   100   000	Old_age   Offline	  -	   0
187 Reported_Uncorrect	  0x0032   100   100   000	Old_age   Always	   -	   0
194 Temperature_Celsius	 0x0022   036   038   000	Old_age   Always	   -	   36 (Min/Max 0/38)
195 Hardware_ECC_Recovered  0x001c   100   100   000	Old_age   Offline	  -	   0
231 Temperature_Celsius	 0x0013   100   100   010	Pre-fail  Always	   -	   0
233 Media_Wearout_Indicator 0x0000   000   000   000	Old_age   Offline	  -	   1278
234 Unknown_Attribute	   0x0032   000   000   000	Old_age   Always	   -	   543
241 Total_LBAs_Written	  0x0032   000   000   000	Old_age   Always	   -	   1239
242 Total_LBAs_Read		 0x0032   000   000   000	Old_age   Always	   -	   405

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description	Status				  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline	   Completed without error	   00%		 0		 -
# 2  Short offline	   Completed without error	   00%		 0		 -
# 3  Short offline	   Completed without error	   00%		 1		 -

Selective Self-tests/Logging not supported

Based on previous experiences I would expect some files to get corrupted in a week or so. I will keep you posted and hope to get some support if it happens again, since I honestly would not know where to look for the problem. Chris Moore mentioned that it could be a driver problem. How would I find out.

Configuration:
FreeNAS-11.1-RELEASE
CPU Intel(R) Xeon(R) CPU E3-1220 v5 @ 3.00GHz
RAM 2 x Crucial 16GB Single 2133MT/s DDR4 PC4-17000 Dual Ranked x8 ECC DIMM
HDD 4 x WD Red 2 TB 3.5’’
Power Supply Seasonic Platinum 400
Motherboard Supermicro X11SSM-F

 

[anmnz]

I reported file corruption issues when trying to use a SanDisk SSD Plus 2.5" as a boot disk in this thread:

https://forums.freenas.org/index.php?threads/freenas-installer-sandisk-ssd-checksum-errors.64049/

Just thought I'd mention it here in case it's becoming a pattern.

 

[Chris Moore]

Anyway, the only thing I changed this time round was that I set the SSD in the BIOS to "SSD", instead of "HDD".

That is not a setting I have seen in a BIOS before, but I would guess it makes a difference or they wouldn't have a setting for it.

Interesting.

 

[capa]

I just did a quick search on the internet to find some details about the BIOS setting called "SATA Device Type" that I changed from "HDD" to "SSD". Here's the answer from the supermicro site.

 

[capa]

This time around things look considerably better. After one week of operation and two scrubs my SSD (ADATA SP920 128 GB) has no corrupted files. My conclusion is that the problem lies with the Silicon Motion Controller SM2246 or SM2256 or some variant of them. Both the SSD that showed corrupted data (SANDisk SSD Plus 2.5'', WD Green 3D Nand) have this controller (as far as I was able to research in the internet), whereas the ADATA SP920 128 GB has a Marvell controller.

Obviously, I'm not saying that these controller are the reason for the problem, but rather that the combination of FreeNAS 11.0 and this controller does not to work as expected.

As a reminder anmnz reported the same problem with his SANDisk SSD Plus 2.5'' (see his thread), and he moved to a Kingston. It's not entirely clear what controller they use ('latestet generation controller", 2HC, Phison S11?). In any case, it is not one from Silicon Motion.

Curious to hear your comments?

 

[Chris Moore]

Curious to hear your comments?

I have only used Intel and HGST branded SSD drives in my FreeNAS and had no trouble with either of them. It is curious.

about the BIOS setting called "SATA Device Type" that I changed from "HDD" to "SSD". Here's the answer from the supermicro site.

I don't think the BIOS setting you mentioned had anything to do with this.

Were you using the boot drive to host your system dataset?

 

[capa]

Were you using the boot drive to host your system dataset?

I'm not sure I understand. I have not changed any setting in this regards. So, I guess I'm on the default setting. Are we talking about the following setting? Then it is not on the boot drive, and I don't recall having changed anything around this.

 

[Chris Moore]

I'm not sure I understand. I have not changed any setting in this regards. So, I guess I'm on the default setting. Are we talking about the following setting? Then it is not on the boot drive, and I don't recall having changed anything around this.

When FreeNAS is installed, it is supposed to move the system dataset to the first pool created so that it is not on the boot device. This was not always true and there are instances where it might not work out the way it is supposed to. It is best to be sure.
In my situation, using hard drives for the boot pool, I move my system dataset , the syslog and reporting database all to the boot pool because I am not concerned about wearing out those hard drives. If you are using a USB memory stick, you would just want to ensure that the opposite is true. You would want those items on a disk based pool so that the constant write does not wear down the capability of any lower write rated device.

 

[capa]

Chris, could you please post the controller that are used in the SSD that you are using.

 

[Chris Moore]

Chris, could you please post the controller that are used in the SSD that you are using.

I don't use SSD for boot. I use plain old hard disk drives that have a spinning disk inside.

 

[MrToddsFriends]

This time around things look considerably better. After one week of operation and two scrubs my SSD (ADATA SP920 128 GB) has no corrupted files. My conclusion is that the problem lies with the Silicon Motion Controller SM2246 or SM2256 or some variant of them. Both the SSD that showed corrupted data (SANDisk SSD Plus 2.5'', WD Green 3D Nand) have this controller (as far as I was able to research in the internet), whereas the ADATA SP920 128 GB has a Marvell controller.

Thanks for the detailed information. So far I didn't use anything else than the following SSDs in FreeNAS, with no apparent problems.

Code:

Model			   | Device No	   | Controller	   | Firmware Version
--------------------|-----------------|------------------|-----------------
SanDisk X300s  64GB | SD7SB3Q064G1122 | Marvell 88SS9188 | X2170000
SanDisk X400  128GB | SD8TB8U128G1122 | Marvell 88SS1074 | X4163100

Controller hardware as stated in
https://www.anandtech.com/show/8398/sandisk-x300s-512gb-review
https://www.anandtech.com/show/10296/the-sandisk-x400-1tb-ssd-review

 

[capa]

Code:

Model			   | Device No	   | Controller	   | Firmware Version
--------------------|-----------------|------------------|-----------------
SanDisk X300s  64GB | SD7SB3Q064G1122 | Marvell 88SS9188 | X2170000
SanDisk X400  128GB | SD8TB8U128G1122 | Marvell 88SS1074 | X4163100

Thank you so much for sharing. This confirms, or at least does not refute my hypothesis, that the problem lies with some brands of Silicon Motion controllers (SM2246 or SM2256).

To everyone: If you have a FreeNAS configuration with an SSD, could you please post the details for the SSD taking MrToddsFriends post above as a guiding example.

 

[capa]

I have created a bug report.