SOLVED Can't install FreeNAS on Dell Poweredge 2950

[Hexland]

I picked up a refurbished rackmount server with the express intention of using FreeNAS -

Dell Poweredge 2950 Gen III
2 x Quad Core Xeon E5450 @ 3.0Ghz
32Gb ECC Ram
Dell PERC 5 SAS Controller
IBM m1015 SAS Controller (IT Mode)
6 x WD 3Tb drives (Connected to the m1015)
1 x Intel 80Gb SSD (Connected to the motherboard internal SATA)

I can run ESXi on it, but my performance is awful (20mb/s through a virtualized ZFS-RAIDZ2 pool) -- I had intended on running with directpath I/O, but it would appear that my system doesn't support it, so I thought I'd install FreeNAS directly with no virtualization.

Booting FreeNAS from CD works fine, and goes through the install process seemingly OK.

Attempting to boot the installed FreeNAS from either USB stick or internal SSD (I tried installing there too, incase my USB stick was corrupt) - only gets so far, before hitting

GEOM: da0: the primary GPT table is corrupt or invalid
GEOM: da0: using the secondary instead -- recovery strongly advised

It then does some sort of stack track, and dumps me at a debugger command prompt (not even a shell)

mountboot>


This happens on 3 different USB sticks, and an internal SSD... so I'm pretty sure nothing is borked or corrupt on the hardware side... and Centos, Windows and ESXi all run fine on this hardware...


Any ideas whats going on?

Thanks
Dave

 

[Hexland]

Solved.

Turns out I had to zero not only the USB flash drive, but also all 6 of the data drives before it would boot.

Boot with the FreeNAS install CD, and "dd if=/dev/zero of=/dev/da4 bs=2048k" to the USB Drive, then install FreeNAS.... and do the same thing for the 6 x 3Tb WD drives.

 

[poldas]

Does IBM M1015 fit to DELL 2950 cover? How do you plug power to disks?

 

[Ericloewe]

Does IBM M1015 fit to DELL 2950 cover? How do you plug power to disks?

It's a normal half-height PCIe card, so it will almost certainly fit, unless Dell uses some stupid proprietary format.

As for power, the HBA has absolutely nothing to do with powering the disks. You connect them to the PSU, like all HDDs.

 

[Hexland]

The 1015 fits into the vertical PCI riser and runs horizontally -- fits no problem.
There is a 4-pin molex power around where the Perc is attached... I run additional power from that plug.

With the Perc removed, there's enough space between the lid and the chasis to put some SSD's

Here's an older picture of my rig... You can see I've run new SATA cables from the 1015 to the drive chasis (the thick shiny blue ones) and I'm running a couple of additional SSD's from the internal motherboard SATA ports for a total of 8 disks.

The CPU fans have been modified with some in-series resistors to lower their spin speed, and I've reflashed the baseboard bios to lower the alarm thresholds for the CPU fans speeds - so the resistors don't cause alarms. The machine is now as quiet as a desktop.

 

[poldas]

Thank you for picture :)
I want to buy used controller. Now I can see that cable I will have to buy separately

 

[Hexland]

http://www.monoprice.com/Product?c_id=102&cp_id=10254&cs_id=1025408&p_id=8191&seq=1&format=2

$12 each - you'll need 2.

 

[Mbohde]

The CPU fans have been modified with some in-series resistors to lower their spin speed, and I've reflashed the baseboard bios to lower the alarm thresholds for the CPU fans speeds - so the resistors don't cause alarms. The machine is now as quiet as a desktop.

I've already done the 47 ohm resistor modification to my system as well. If you wouldn't mind help with the baseboard bios flashing would be very appreciated.

 

[Hexland]

I don't remember much about it... I did this back in January (I really should write this stuff down when I do it)

I followed the guide here -- http://projects.nuschkys.net/2011/11/15/how-to-adjust-the-fan-thresholds-of-a-dell-poweredge/

I do remember having a bugger of a time actually flashing to BIOS... I think I booted with an Ubuntu live CD, and downloaded and flashed from there.
I may even have had to download, manually unpack and do some other shenanigans to get it flashed.

I'm sorry I can't be of more help, it was a while ago.

I've since gotten rid of the hardware in favor of a smaller Xeon i3 solution that could handle PCI passthrough for the HDD controller... the lack of VT-d on the Poweredge was very disappointing.

 

[Mbohde]

Is there any chance you saved your modified bmcflsh.dat?

 

[Hexland]

Not a chance, sorry... I've gone through so many re-installs, hardware changes, etc.

 

[Hexland]

This blog post also helped I think... it looks familiar.
http://packetinspection.blogspot.co...howComment=1376765788960#c3773544318023773436

 

[Mbohde]

That's okay I think I can figure it out. * Edit * That blog post looks very helpful Thanks.
By the way were you using interposers on your backplane?

 

[Hexland]

I don't know what an interposer is... is that the little adapter thingy to convert from SAS to SATA or something?
If so, no... when I purchased the server, it came with 6 x SATA HDD caddies - no converters were necessary.

 

[petr]

Dell Poweredge 2950 Gen III
2 x Quad Core Xeon E5450 @ 3.0Ghz
32Gb ECC Ram
Dell PERC 5 SAS Controller
IBM m1015 SAS Controller (IT Mode)
6 x WD 3Tb drives (Connected to the m1015)
1 x Intel 80Gb SSD (Connected to the motherboard internal SATA)

Hello! I am awaiting delivery of almost identical setup to yours, including the controller and I am doing my prep research.

I intend to migrate my current install onto the server (direct SATA connections to MOBO), could you elaborate what was the reason for zero-ing out the drives? Were they brand-new or with old data?

I intend to hook them up to the M1015 and just move over the current bootable USB. I do not intend to use the PERC - did you remove it altogether?

 

[Hexland]

I had to zero out the drives because there was an issue when booting a freshly dd'd copy of the FreeNAS image - it would get only so far, and hang. I found the info for zero'ing out the drive on another thread, and it seemed to work. Something to do with crap left on unused blocks (which doesn't really make much sense to me unless FreeNAS is using a raw block device during boot that it assumes is zero or something).

I removed the PERC just so I could sit another couple of SSD's in that spot. I didn't have to remove it, but I also didn't want to have to deal with potential hardware issues if I left it in there. Same with the remote control board thingy.

For the sake of full disclosure - I eventually got rid of this system, and built a smaller, quieter custom one with a Xeon i5 CPU. I moved house, and didn't have anywhere to put the Dell that I couldn't hear (even with the modified baseband firmware and the in-series resistors on the chasis fans). I also eventually dropped FreeNAS in favor of Napp-It -- I was having nothing but trouble with Windows ACL's on the CIFS shares, and just couldn't get FreeNAS to cooperate... rather than wait on the infamous ACL help document (almost a year and counting now!), I just went with Napp-it and bought the ACL extension -- I had the server up and running within an hour, with no 'access denied' problems.

 

[petr]

Ok, so it was not the boot disk, sounds odd but will keep it in mind just in case!

Thank you for the further detail - the number of windows clients equals to 0, and I have been generally quite happy with FreeNAS.

How was the machine performing while you still had it?

 

[messy]

http://www.monoprice.com/Product?c_id=102&cp_id=10254&cs_id=1025408&p_id=8191&seq=1&format=2

$12 each - you'll need 2.

Is there anyway you could post another link or part # and maker of this cable? Link seems to be dead.