FreeNAS on Bare metal Dell R720XD (H310 MINI) or ESXi VM?

[gallyjh]

Hi, I've inherited a Dell R720XD with an H310 MINI Raid Controller. I've read a couple posts that it can be troublesome to load FreeNAS on it. In fact I've tried and so far I keep getting these long boots and errors that say something along "Error on compression algorithm 96...". I did some research there, and some people point to that problem being GRUB's, with no real solution to follow. Some say use the FreeNAS nightly that moves away from GRUB, but I'm looking for something more stable.

Anyway, my system has 2x 8TB drives in RAID0, 128GB RAM and I'm hoping to have maximize my performance. I'd like my server to have these abilities:

# Run Plex Media Server
# Run Sonarr
# Run Couchpotato
# Run VM's for development
# Provide storage to other workstations

I originally thought FreeNAS would be great for these purposes on bare metal, but it's proving to be difficult. Am I better off just throwing ESXi on there and making a VM for FreeNAS?

Edit: I should mention when I installed FreeNAS, I attempted to install it on my internal USB Samsung stick.

Edit: Changed R270XD to R720XD . Thanks @Chris Moore

 

[Chris Moore]

Dell R270XD

Do you mean a 720?

H310 MINI Raid Controller.

NOTE: this answer has changed as of 24 Apr 2019: Updated here for the sake of others searching for this answer.

You might be better off selling that one on eBay and buying one that is already flashed to IT mode. You shouldn't use it with FreeNAS as it is. The easy fix is to get one already flashed because you can't follow the guides as they are written for any of the regular H310 cards (it will brick your card) but you can flash the card.
Here is a link to a video that discusses it:

Dell H310 mini with LSI IT mode firmware
https://www.youtube.com/watch?v=Y1Xi5NZRlXM

in RAID0,

Not until you install FreeNAS. Forget hardware RAID. It doesn't exist any more. ZFS mirrors the drives at a filesystem level.

Edit: I should mention when I installed FreeNAS, I attempted to install it on my internal USB Samsung stick.

Lets find out what hardware you have first, because that is really the first thing to find out and what you listed will need some modifications to make it work.

You also need to do some reading to get you indoctrinated.

Updated Forum Rules 4/11/17
https://forums.freenas.org/index.php?threads/updated-forum-rules-4-11-17.45124/

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

 

[gallyjh]

Do you mean a 720?

May as well throw that on eBay, see if you can sell it. You can't use it with FreeNAS.

Not until you install FreeNAS. Forget hardware RAID. It doesn't exist any more. ZFS mirrors the drives at a filesystem level.

Lets find out what hardware you have first, because that is really the first thing to find out and what you listed will need some modifications to make it work.

You also need to do some reading to get you indoctrinated.

Updated Forum Rules 4/11/17
https://forums.freenas.org/index.php?threads/updated-forum-rules-4-11-17.45124/

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

Thanks Chris, this helps. I've also been cruising other threads and almost feel I'm regretting purchasing this thing.

It sounds like MINI is the problem, is there are controller you recommend for this?

Here is a list of my hardware:

Code:

PER720xd 12x LFFE5-2690V2NO
DUAL INTEL XEON PROCESSOR E5-2690 V2 TEN CORE 25M CACHE 3.0GHz
128 DDR3 ECC RAM
2X Seagate IronWolf 8TB NAS Hard Drive 7200 RPM 256MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive ST8000VN0022
Dell Perc H310 SAS/SATA Raid Controller
Broadcom 5720 Quad Port Gigabit Daughter Card
Samsung 32GB USB 3.0 Flash Drive Fit (MUF-32BB/AM)

Thanks @Chris Moore

 

[Chris Moore]

It sounds like MINI is the problem, is there are controller you recommend for this?

NOTE: this answer has changed as of 24 Apr 2019: Updated here for the sake of others searching for this answer.

The H310 MINI card has some important (Dell specific) identification bits set in the programmable portion of memory that is cleared when you follow the standard instructions for flashing a LSI SAS HBA, so if you clear the program space and reboot the system, it bricks the card because the system will not reboot as it does not recognize the card. So, it needs special instructions that have not exactly been documented, but you might figure it out if you take a look at this video:

Dell H310 mini with LSI IT mode firmware
https://www.youtube.com/watch?v=Y1Xi5NZRlXM

What you could use is one of these:
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601

It is same hardware in a standard form factor and it has already been flashed with the IT firmware. Put it in one of the PCI slots, connect the cables.

This seller also has plans to put some of the pre-flashed H310 Mini cards on their eBay store, so you might want to check on that:
https://www.ebay.com/str/theartofserver/HBA-IT-mode-controllers/_i.html?_storecat=23844485012

I can't say why you are having trouble getting FreeNAS to install. I know that other people have used the R710 server. I can't imagine that there would be anything in the 720 that would present a real problem.

You can do the install of FreeNAS onto the boot media using any computer. Then put the boot drive in the FreeNAS and turn it on. FreeNAS detects hardware at boot. The only configurations that are saved boot to boot are your settings that are configured through the GUI and saved in the configuration database.

 

[gallyjh]

What you could use is one of these:
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601

Thanks, I've placed an order for it, along with two cables. Hopefully this will get me up and running.

I can't say why you are having trouble getting FreeNAS to install. I know that other people have used the R710 server. I can't imagine that there would be anything in the 720 that would present a real problem.

I'm going to follow your directions and try installing it to the USB drive on a different machine and then pop that USB drive back into the R720XD and see what happens. I'll follow up shortly...

 

[gallyjh]

Hi @Chris Moore, thanks again for your feedback.

I got the LSI card in and swapped out the H310 MINI. I lost some Dell functions like not being able to green status lights on the drive caddy anymore, and some of my drives don't seem compatible with new controller and don't show. Otherwise the disks that do work get listed. I was able to get FreeNAS installed on the internal USB finally. It seems the R270XD internal USB drive only supports 2.0 (you'd think that 3.0 drive would still work). Buying a 2.0 flash drive did the trick there.

After playing with it over the last few days, I decided loading ESXi on bare metal with LSI passthrough on the controller then making a FreeNAS VM is the way to go. It seems to be the way many youtube video's seem to suggest. Haven't gotten there yet, because now I'm waiting on my dual SD card module to come in so I can load FreeNAS on that, because ESXi can't see the drives I have if I have LSI on passthrough.

Hopefully this gets me moving. Thanks again!

 

[Chris Moore]

It seems to be the way many YouTube video's seem to suggest.

Be careful of the YouTube videos. Many of those people don't really know what they are talking about. They just learn enough to make a video so they can get the views to make the money and don't spend the time to really learn enough about how it works.

This is a thread from one of the folks here on the forum that has done the virtual FreeNAS on ESXi. You might want to read what @Stux has to say because it is all good:

Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://forums.freenas.org/index.ph...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

Testing the benefits of SLOG
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

 

[Chris Moore]

It seems the R270XD internal USB drive only supports 2.0 (you'd think that 3.0 drive would still work). Buying a 2.0 flash drive did the trick there.

With the USB 3.0 issue, I have actually run into that myself with certain BIOS / UEFI firmware. I don't get why it isn't compatible either.
If you run into any problems like that, please post, someone here will likely have run into it and might even remember if it was recently enough.

 

[neb50]

Hi @Chris Moore, thanks again for your feedback.

I got the LSI card in and swapped out the H310 MINI. I lost some Dell functions like not being able to green status lights on the drive caddy any more, and some of my drives don't seem compatible with new controller and don't show. Otherwise the disks that do work get listed. I was able to get FreeNAS installed on the internal USB finally. It seems the R270XD internal USB drive only supports 2.0 (you'd think that 3.0 drive would still work). Buying a 2.0 flash drive did the trick there.

After playing with it over the last few days, I decided loading ESXi on bare metal with LSI passthrough on the controller then making a FreeNAS VM is the way to go. It seems to be the way many youtube video's seem to suggest. Haven't gotten there yet, because now I'm waiting on my dual SD card module to come in so I can load FreeNAS on that, because ESXi can't see the drives I have if I have LSI on passthrough.

Hopefully this gets me moving. Thanks again!

I'm not sure that ESXi will see the SD cards as a drive that you can use.

I just got a R720xd with the dual SD cards and loaded ESXi on them. I then got a cheap PCIe x4 to NVMe adaptor and Intel Optane 16Gb card and loaded FreeNAS on it as a VM within ESXi. I was hoping to use the built in SATA ports on the MB, but there is something on the xd that disables the SATA controller on the MB. If you put the same MB into a r720 or Compellent sc8000 case, they will work.

I am waiting on my HBA and cables to get the drives in and should have them tonight to get the rest of the setup running.

I am using Spearfoot's ESXi scripts on my old setup to host other ESXi VM's using the FreeNAS storage and will use them on this server as well.

I would also note that when I tried to run Plex within FreeNAS as a plugin with ESXi, I would have issues and moved Plex to it's own VM in ESXi to fix it.

 

[gallyjh]

Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://forums.freenas.org/index.ph...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

Testing the benefits of SLOG
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

Thanks for the links, will definitely look into these!

 

[gallyjh]

Sounds like you are doing exactly what I'm trying to do.

I'm not sure that ESXi will see the SD cards as a drive that you can use.

I just got a R720xd with the dual SD cards and loaded ESXi on them. I then got a cheap PCIe x4 to NVMe adaptor and Intel Optane 16Gb card and loaded FreeNAS on it as a VM within ESXi. I was hoping to use the built in SATA ports on the MB, but there is something on the xd that disables the SATA controller on the MB. If you put the same MB into a r720 or Compellent sc8000 case, they will work.

It's funny you mention this. I was hoping to use those back ports to host the FreeNAS VM, so I thought I'd shove a couple 2.5 drives on these sata ports on the back of the R720XD. Originally I thought they were embedded on the motherboard and the BIOS would pick it up separately. Nope, they are actually being driven by the new LSI controller too. So no putting FreeNAS there.

I see that you got the dual SD cards, and because it's not hosted by the LSI controller I would hope ESXi would pick those up as possible datastorages, and in turn can run FreeNAS on that in mirror mode. Right now I have ESXi running off a USB flash drive. I won't know until I get that dual SD card and flash cards tonight.

I was also looking at a similar backup plan and thinking of going with a cheap PCIe NVMe card as well. We'll see after tonight's experiment first.

I would also note that when I tried to run Plex within FreeNAS as a plugin with ESXi, I would have issues and moved Plex to it's own VM in ESXi to fix it.

I was thinking this may be an issue for encoding performance, so yeah, I was going to just throw Plex in it's own VM, and have Couchpotato/Sonarr/Etc running via PhotonOS / Docker.

 

[dir_d]

Sounds like you are doing exactly what I'm trying to do.



It's funny you mention this. I was hoping to use those back ports to host the FreeNAS VM, so I thought I'd shove a couple 2.5 drives on these sata ports on the back of the R720XD. Originally I thought they were embedded on the motherboard and the BIOS would pick it up separately. Nope, they are actually being driven by the new LSI controller too. So no putting FreeNAS there.

I see that you got the dual SD cards, and because it's not hosted by the LSI controller I would hope ESXi would pick those up as possible datastorages, and in turn can run FreeNAS on that in mirror mode. Right now I have ESXi running off a USB flash drive. I won't know until I get that dual SD card and flash cards tonight.

I was also looking at a similar backup plan and thinking of going with a cheap PCIe NVMe card as well. We'll see after tonight's experiment first.



I was thinking this may be an issue for encoding performance, so yeah, I was going to just throw Plex in it's own VM, and have Couchpotato/Sonarr/Etc running via PhotonOS / Docker.

I have a R720xd and i had ESXi running on the dual SDcard slots but i erased that config and ran Freenas bare metal on the R720xd on 2 mirrored USB drives. If i were to do it all over again i would put the ESXI on the dual internal and use the dual USB as Freenas. I was bummed about the 8 core lock in free ESXI, thats why i went bare metal and run VM's in bhyve. I have SSDs in the back 2 slots that i havent figured out what to do with them yet.

 

[gallyjh]

@dir_d you are correct that ESXi won't see the Dual SD as drives. But I don't think you can see any SD drives or USB drives unless you go through these instructions dealing with the usbarbitrator. In any case I ordered a PCIe and NVMe drive in order to get FreeNAS installed. I feel like I'm just tossing more and more money into this pile, LOL.

I was curious since you have a similar setup, are you no longer showing green lights for hard drive status on your caddies since switching to LSI? Is the status light near the power button also showing a yellow heartbeat/hdd icon?

 

[dir_d]

@dir_d you are correct that ESXi won't see the Dual SD as drives. But I don't think you can see any SD drives or USB drives unless you go through these instructions dealing with the usbarbitrator. In any case I ordered a PCIe and NVMe drive in order to get FreeNAS installed. I feel like I'm just tossing more and more money into this pile, LOL.

I was curious since you have a similar setup, are you no longer showing green lights for hard drive status on your caddies since switching to LSI? Is the status light near the power button also showing a yellow heartbeat/hdd icon?

The Hard Drive status lights do not work anymore since the switch to the LSI card. They still light up when data is written to them but that's about it. Sas2ircu doesn't work either but it does list the drives with serials in their order so i can use that vs FreeNas to tell which drives, in which location are having problems. My Status light is blue, i also have the fully unlocked iDRAC ($50 on ebay), so i monitor as much as i can in there.

 

[sfcredfox]

Sas2ircu doesn't work either

There's another utility to do this, but I can't find my notes about it. Sorry. I was not able to use SAS2IRCU on my enclosure, but the other one worked. Until I find those notes, at least you know there still might be a way.

 

[gallyjh]

The Hard Drive status lights do not work any more since the switch to the LSI card. They still light up when data is written to them but that's about it. Sas2ircu doesn't work either but it does list the drives with serials in their order so i can use that vs FreeNas to tell which drives, in which location are having problems. My Status light is blue, i also have the fully unlocked iDRAC ($50 on ebay), so i monitor as much as i can in there.

All my drives are brand new, but may not be on the Dell HCL, so that could be why I'm seeing a yellow indicator. Mine did come with iDRAC, and I can log in but it won't show the disks:

I'm guessing the LSI controller is not out-of-band capable? What is this fully unlocked iDRAC that you have? Can't seem to find anything really related on google or ebay, unless it's called something else?

 

[neb50]

@dir_d you are correct that ESXi won't see the Dual SD as drives. But I don't think you can see any SD drives or USB drives unless you go through these instructions dealing with the usbarbitrator. In any case I ordered a PCIe and NVMe drive in order to get FreeNAS installed. I feel like I'm just tossing more and more money into this pile, LOL.

I was curious since you have a similar setup, are you no longer showing green lights for hard drive status on your caddies since switching to LSI? Is the status light near the power button also showing a yellow heartbeat/hdd icon?

I got the card and cables and got everything working. The server status is good (shows blue) with my setup. The drive indicator lights only show when a drive is active (read or write), but does not show which drives are populated.

I also have the enterprise version of iDRAC and agree that it makes setting up the server much more convenient.

 

[neb50]

All my drives are brand new, but may not be on the Dell HCL, so that could be why I'm seeing a yellow indicator. Mine did come with iDRAC, and I can log in but it won't show the disks:

I'm guessing the LSI controller is not out-of-band capable? What is this fully unlocked iDRAC that you have? Can't seem to find anything really related on google or ebay, unless it's called something else?

Are you using the LSI controller in a PCI slot or the internal raid controller?

When I was using the internal raid controller (H710p), some drives would cause the server status indicator to go amber and the drive indicator would also flash amber. Now that I have the LSI card, I removed the H710 and do not have any drive indicators and the server status is good.

 

[gallyjh]

Are you using the LSI controller in a PCI slot or the internal raid controller?

When I was using the internal raid controller (H710p), some drives would cause the server status indicator to go amber and the drive indicator would also flash amber. Now that I have the LSI card, I removed the H710 and do not have any drive indicators and the server status is good.

OK, I just decided (out of curiosity) to remove all drives and see if it turns blue, and it did turn blue. Then I put in one drive at a time to see if it would remain blue, AND.IT.DID! Got more curious, and tried an old 1TB drive that initially wasn't working when i seated the PCIe LSI card, and I was even able to get a 1TB green drive to show. No friggin clue what changed, but I'm happy it's all blue and working.

Went back into iDRAC (I also have the enterprise edition, I believe, as it says on the top of the web UI), still won't show my drives there. Which LSI card did you specifically buy? I purchased the one Chris mentioned in post #4.

 

[neb50]

I got a 9207-8i. The storage portion of iDRAC doesn't show anything for me and it doesn't bother me since I am not using the hardware raid.

I noticed that sometimes you need to remove power and restart the server to get the server status to clear and other times a reboot will fix it. I am on the latest iDRAC and bios versions if that matters.