Freenas wouldn't start from USBdrive, reinstall fails

[Oelc]

System
MB: ASRock C2550D4I
HDD: WD Red WD40EFRX, 4TB (9 HDDs)
RAM: Crucial CT102464BA160B 8GB (4 pieces) 32GB total 1.600MHz
PSU: Corsair RM550x
USB: Sandisk Cruzer Ultra Fit 32GB

Before startup problem
I switched two older RAM modules 8GB 1.333MHz to the current onces mentioned above. (before i had 2 1.333MHz and 2 1.600MHz 8GB modules installed) And started the system, it failed to start up. And after some searching, I concluded there must be a problem with the Sandisk cruzer Ultra Fit, it gets very hot! I tried it with different USBdrives. With different results but all failed. These 4 drives I tried:
- Sandisk Cruzer Ultra Fit 32GB (two different USB drives I've tried)
- Kingston datatraveler 100 G3 16 GB
- Kingston DTM30 32GB

Here are two screenshots of 2 failed install attempts I documented. I tried lots of combinations with my USBdrives. With on one USB the "FreeNAS-9.3-STABLE-201602031011.iso" and the other clean formatted.

What am i doing wrong.

 

[joeschmuck]

I can't read your screen shots, they are too small.

First, have you run MemTest86 on your system to see if the RAM even works? Maybe you need to re-seat the modules as well.
Second, reinstall your original RAM and see if that fixes it.

 

[clarknova]

1. What are you using as install media (the physical device)?
2. How did you prepare the install media (ie, how did you get the iso onto the install device)?
3. What device are you trying to install to?

 

[Oelc]

@joeschmuck
The two screenshots:
http://s30.postimg.org/4tgumclb5/freenas_fail.jpg
http://s30.postimg.org/8bsuiqm75/freenas_fail2.jpg

I'll run a Memtest, but i dont think i every heard a story of failed RAM (still there must be a first time). After this i'll maybe switch to the old RAM config.


@clarknova
1. A USB thumb drive, so at the start of the install, i have 2 USB drives in the system, one with the Freenas install otherone formatted (fat32/ntfs doesn't matter, right?)
2. In windows 10 i use Win32DiskImager to write the latest stable freenas iso to one of the USB thumb drives.
3. I have 4 available USB drive to install on/to. Numbered A to D:
A - Sandisk Cruzer Ultra Fit 32GB
B - Sandisk Cruzer Ultra Fit 32GB
C - Kingston datatraveler 100 G3 16 GB
D - Kingston DTM30 32GB
So for example I've tried to install from A (with freenas iso loaded) to B, B to A, C to A, C to B etc.
I wanted to use the Sandisk Ultra Fit USB drives because of the low profile/there really small


Now i'll start with a MEMtest

 

[clarknova]

@joeschmuck]I'll run a Memtest, but i dont think i every heard a story of failed RAM

RAM can and does fail. memtest will help with that determination. It's also possible that the board and RAM simply don't work well together. memtest can also be helpful there.

1. A USB thumb drive, so at the start of the install, i have 2 USB drives in the system, one with the Freenas install otherone formatted (fat32/ntfs doesn't matter, right?)

Right. I don't see any problem with your install method.

No i'll start with a MEMtest

Did you verify your iso download against the hash? If the iso you're using is good, then it sounds like a hardware problem to me, or a compatibility issue. You could try tweaking your BIOS/firmware settings, but memtest is a good place to start.

 

[Oelc]

I used the same Iso as before the problems started. It worked great for 2 weeks.

update: the SHA-256 match

 

[clarknova]

Ok, I initially thought you were trying to do an install and the installer was failing. Now it appears that you had a working install that is now failing to boot.

If that is correct then I definitely suspect failing hardware. The hot USB stick and the failure of multiple USB sticks in the same system is consistent with the failing hardware hypothesis. If this were my system I would disconnect my data drives immediately and continue troubleshooting with just the boot drive.

I suspect a problem with your board. If memtest fails then try other RAM, but a failing board can cause errors in memtest as well. You may want to verify your power supply as well, as an out-of-spec power supply can kill a board as well as anything else connected to it or the board.

 

[Oelc]

If this were my system I would disconnect my data drives immediately and continue troubleshooting with just the boot drive.

That's the first think I did before trying to reinstall.

an out-of-spec power supply can kill a board as well

I have an Corsair RM550x 550W PSU. Now only connected to the motherboard.

 

[clarknova]

I have an Corsair RM550x 550W PSU. Now only connected to the motherboard.

Corsair PSUs are generally good quality, and obviously 550W should be ample for most typical NAS systems (unless you had scads of drives overloading the 5V rail). Still, the symptoms suggest a failing board or PSU to my mind.

 

[joeschmuck]

Is it only me or are the images still too small to see? The image file states it is 180x100 pixels, not very large at all.

Regardless...

Let me ask a few questions...
1) Did the failure only occur after you replaced the RAM and powered on the system? If yes, it's the RAM in some manner.
2) If you had stable running system after you replaced the RAM then it could be something else. Stable means longer than 24 operating hours in my mind, and sometimes longer depending on the system.

Lets explain a little about the RAM... You stated that you had originally a mix of two different RAM types, half of 1.333GHz RAM and half of 1.6GHz RAM. What happens in your system is all your RAM will run at the slowest speed so that would be 1.333GHz. When you replaced your slow RAM with all 1.6GHz RAM then your system "should" have automatically changed the RAM clock to match. It happens often that even though RAM may be rated for a specific speed, that it really doesn't run at full speed without some tweaks like maybe a minor voltage increase (don't do this unless you know what you are doing and can afford to burn up your RAM, CPU, and motherboard). If you find out your RAM is running at 1.6GHz and MemTest86 fails (run at least 3 passes, a few days is preferred) then go into your BIOS and drop your RAM clock to 1333MHz and retest. I suspect this is your problem.

As for power supplies, hey, they can work fine for years and then just because you cycle power on them they fail. If the RAM thing doesn't solve this issue, swap out the power supply and only the motherboard needs to be connected, no drives since you are testing out your hardware.

 

[joeschmuck]

Corsair PSUs are generally good quality, and obviously 550W should be ample for most typical NAS systems (unless you had scads of drives overloading the 5V rail). Still, the symptoms suggest a failing board or PSU to my mind.

I currently do not agree, not until some more information has been passed along like RAM clock speed and if MemTest passes. True, the PSU can result in the same results but the fact that the RAM was replaced and then nothing worked is what I'm working from. The most obvious component is the RAM and I suspect it's running at 1.6GHz.

 

[Oelc]

Except for the drives and the 'old' 1.333MHz RAM all hardware is brand new, purchased at beginning of march.

The system had been power on from march 10 till beginning of this week, with like 5 systems reboots between. Running really well.

And after the failure to boot into freenas, I saw in the bios all the RAM at the right speed. (If i remember correctly)
The failure happend after i switched the system off for the RAM changing.

Memtest86 v4.3.7 is currently at second pass 14% with no errors found.

After memtest i'll look if it's running the most up to date bios version, maybe that will solve this troubles.

 

[joeschmuck]

First I'd like to say, good on you to think about a possible BIOS update, that could make all the difference in the world if there is one which addresses your issue.

The initial reason I'm picking on the RAM is because you made that change and all hell broke loose.

Also report your RAM Speed both reported by MemTest and then in the BIOS (I would expect them to be the same but who knows).

Glad you made one of the screen shots larger, and it does look like an installation failure up front but if you notice the error message says "Bus error" and then gives you a general message of an installation failure. The bus error message specifically deals with a memory problem (do a google search for "FreeBSD bus error") and you will see what I came up with. You should try the old fashioned way of doing an installation just to rule out your method isn't the cause (I doubt it but this is what troubleshooting is about, ruling out possible causes until you fine the problem), that being burning the ISO to a CD-R and then booting up the CD and building the flash drive. This can be done in any computer, just make sure you pick the correct drive to make your boot device, selecting your Windows system disk would be a bad move. You can even boot that USB Flash device on the same computer just to ensure it works, it will not write on any drives you have installed, well unless you start configuring it, just see if it boots and then you will know you have a good boot device for your FreeNAS system.

As for those Sandisk FIT USB 3.0 devices, I never recommend those even though others will. They are small which is great so they do not protrude from the case but they build up a lot of heat, just about every USB 3.0 device does these days so dissipating heat it difficult to do. SanDisk does make a FIT USB 2.0 version which should not get so hot, assuming you still want to go down the USB path. I try to steer folks to the SSD boot device if you have an available SATA port. But if you do want a USB 3.0 boot flash, I'd recommend something with a metal case and is at least 2" long to get rid of the heat. That doesn't make it pretty, but it should make it last a bit longer.

I'm curious how the RAM test will turn out. Even if it passes and there are no BIOS updates, my advice is to ensure the RAM is running at 1333MHz and try again. I've given you a few options to try and you can do that in any way you like.

 

[Oelc]

MemTest86 couldn't find any errors after 6 passes: http://s15.postimg.org/q58sf4i0b/freenas_fail_memtest86_final.jpg

I checked the bios and all the versions where up to date BIOS and bmc (latest version sept 2014).

Finally i disassembled all the hardware and found a difference in RAM modules. All same size, speed and voltage, except one of the four has an CL11, while the others a CL13... Lucky me i could swap the odd one out and now the system seems to be working again. (Knock on wood). I curious why memtest couldn't find an error/noticed it.

I would like to thank you for the support & help. I'm hoping the system will keep running without any problems now!

 

[joeschmuck]

Yes, the CL value can make a difference depending on how your motherboard handled configuring the RAM. Your screen shot is yet again too small to read, you should just upload the image files instead.

As for MemTest86 V4.xxx, well it still had some problems but maybe version 6 would have found the issue but you need to boot from UEFI to get that to work, and I'd enable all CPU cores to run in parallel.

Glad you got it working and hope it stays that way.

 

[Oelc]

http://s15.postimg.org/q58sf4i0b/freenas_fail_memtest86_final.jpg

I dont know why these images are sometimes smaller for you, I upload them all with same method. Screenshot > save > postimg. Maybe you use the zoom function in your webbrowser? I can see them all in original size; 700+ by 400+ pixels.

 

[joeschmuck]

http://s15.postimg.org/q58sf4i0b/freenas_fail_memtest86_final.jpg

I dont know why these images are sometimes smaller for you, I upload them all with same method. Screenshot > save > postimg. Maybe you use the zoom function in your webbrowser? I can see them all in original size; 700+ by 400+ pixels.

for the image link above, here is what I get from my FireFox browser with respect to the image file details...

Maybe you could "Upload a File" and then select "Full Image". I also tried IE and it states the same image size. Not sure how you are viewing them.

 

[Oelc]

In firefox i see a size of 55KB. Maybe it is because of the different countries we live in, some sort of restrictions... You in usa, I in nl.

 

[joeschmuck]

I honestly don't see how there could be a difference since it's on the internet. What you are proposing is the file shrinks just because I'm out of the country. I'd be more apt to believe you are looking at the original file and not the file referenced in the link. Maybe if you could open up your link and right click on it, go into the properties and then take a screen shot and "Upload" the file here as full size so I can see if there is any distinct URL differences. I know this has nothing to do with the system problem but if folks cannot read your postings, it's just going to hamper supporting you. Also, I haven't heard anyone else speak up that they cannot open all the images and read them, it would be nice to know if I'm the only one.

 

[BigDave]

All the images in this post (links and posted to forum) appear full size to me Mark, I think it's on your end.