Powervault MD3820i as JBOD device for FreeNAS

[viniciusferrao]

Hello guys,

You've read correctly, I want to use this expensive DELL storage as a JBOD device using the external SAS interface to a single 1U machine loaded with FreeNAS.

It seems a bit insane from a point of view, but here's the situation: the Powervault device was already bought and it's on production. But this storage is getting us on nerves.

I was called to solve some issues on the network of this company, but when I arrived I've saw a lot of issues on other services, and the storage subsystem was one of them. I will not even mention that DELL sold this storage with only four disks, and recommended RAID5, to run more than 40 VMware VM's, including a 120 user Exchange Server. But that's another problem.

In 4 months we had three database crashes in the Exchange 2013 Server. Three times the database went to an unrepairable state and backups are rolled back with lost messages and full-day outages.

After much stress with DELL blaming everyone else, even VMware was blamed, they finally talked about this Firmware Update: http://www.dell.com/support/home/us/en/04/Drivers/DriversDetails?driverId=MDX0G

With the following fix: Addressed a rare condition which has the potential of causing a processor fault that could result in a data integrity issue.

To sum up everything the company is really pissed of with DELL and they do not trust anymore on the PowerVault device and they want a better solution.

To fix things, I was thinking in using those guys as JBOD devices, buy an enterprise grade machine with lots of RAM, two SSDs for L2ARC, Dual 10GB NICs and an External SAS controller to be FreeNAS appliance. The machine I've in mind is the Lenovo RD540 loaded with at least 64GB: http://shop.lenovo.com/us/en/servers/thinkserver/racks/rd540/

Here in Brazil, those machies are really cheap now (for their grade) since they become EOL and they come completely redundant with two 300GB SAS disk, IT controller, redundant PSU and etc.

And since the company have some disks across the three (yes, three) Powervault MD3820i devices, the idea is to put 11x 10k discs (one hotspare) on the main FreeNAS storage and use cheap 7200RPM disk on other MD3820i devices. So the plain is to built three FreeNAS machines with thoses MD3820i attached as JBOD devices.

So I was opening this thread just for sharing my frustration with DELL and to hear some opinions from our awesome community.

Thanks in advance,
Vinícius.

 

[cyberjock]

Well, normally I have a small stroke when I see someone using Dell hardware on FreeNAS. It seems that many people have compatibility issues with them. But... the hardware is paid for and it sounds like you have a firm grasp of the fundamentals and are going this route with the expectation that if it just won't work you'll find a more appropriate solution (yay). What really sucks is the people that try to force something to work when it really shouldn't be used.

All that being said, sounds like you're tired of Dell (no surprise there.. lots of iXsystems customers are tired of Dell too, hence they are switching to TrueNAS). Exchange really is a resource and iops hog. So if I were in your shoes I'd consider making a second smaller zpool that is all SSD and put the exchange server on that by itself. It really makes a very very big difference in the performance of the other VMs if Exchange is heavily used. Databases like SQL, Exchange, etc are the worst of the worst for ZFS. I've worked with a few companies that went all-SSD for their databases when their databases were already slowing down the entire storage system and they weren't really wanting to spend the money, but after it was spent they were totally thrilled with the performance and were glad I recommended it to them.

 

[viniciusferrao]

Thanks for sharing your experiences cyberjock.

About the idea of using SSDs for the databases, it's really a good one, I just need to calculate the budget to fix the entire storage subsystem of the company. The good part of using SSDs is that I can attach them directly to the FreeNAS machine.

But since we're talking about SSDs now, the same considerations of disk drives should apply (stripe of mirrors)? Or with the increased IOPS of those beasts a RAID-Z(2) would be viable? Because it can reduce the costs.

Another question: which SSD devices are you recommending? The Intels S3500/S3700 still a good option?

Thanks,
Vinícius

PS:

Which drives me mad about DELL is their fair play. They simply don't have it. Just assume that the PowerVault MD3800 series isn't a really good product, we'll be ok with those answer, but they don't. After a lot of questions of how this MD3820i works they always fails to answer with a good technical point, the common answer was: the storage is very good, it has advanced algorithm that handles everything.

After much bashing and flamed discussions by the phone, including one that the "expert" said that I don't know how a disc subsystem works, since data corruption doesn't happen on SAN storage because they export blocks, I went in a rage and talked much more seriously and the IT manager of the company even talked about suing DELL.

Some emails later an "Level 2" specialist answered some of my questions, little vague but it was exactly what I expected to hear: all data security features are for premium licensed features (even CoW, is a premium). Their scrubbing algorithm only works on disks, it's disabled on SSDs. Which is a shame for this overpriced storage.

I really want to know how those MD3800 series works. I think it was just a plain LSI 106X series controller, dual stacked, with a lot of memory cache and iSCSI 10GbE interfaces for marketing it at 10GbE.

 

[viniciusferrao]

Hello guys, this project was rebooted and the ideia was accepted.

At this moment we will be buying a DELL R730 machine loaded with one CPU, 4x 16GB of DDR4 Memory and 8x 2TB NL-SAS discs for local storage. The external JBOD devices isn't discarded yet, and it should be revived, that's why we are buying 16GB sticks at this moment looking for expansion in the future.

What is really concerning me at this moment, is the offered HBA. The LSI/Avago 9300 SAS and/or DELL PERC H330. I've seen that the mpr module loads those cards just fine in FreeBSD 9.3, but I would like to know if someone in the community already uses this SAS 3008 chipset.

Another question is about the NIC's. The machine will arrive with four Broadcom's 5720 NICs, and it appears to be unsupported/untested with FreeNAS. Any advices on this? I know that Intel's are the way to go, but is it really necessary?

Thanks in advance,
Vinícius.

 

[Darren Myers]

Another question is about the NIC's. The machine will arrive with four Broadcom's 5720 NICs, and it appears to be unsupported/untested with FreeNAS. Any advices on this? I know that Intel's are the way to go, but is it really necessary?

Buy an Intel NIC, or a quad NIC if you just want more ports..

but I would like to know if someone in the community already uses this SAS 3008 chipset.

From a recent thread:

Even with the great LSI 2008 products, I don't recommend the 3008 just yet. Not because I don't trust LSI, but there's relatively little data and prior to 9.3.1 virtually everyone had major problems including corrupted pools that ended up going bad before they existed long enough to even store data. (I'm just very conservative and don't like to endorse something until I feel it is safe enough that I would use it myself.)

You can also have a look here and here

 

[depasseg]

My supermicro box has a 3008 based LSI HBA. It's been working fine with the SAS3 backplane, but there are (or at least 'were') issues trying to connect an external SAS2 JBOD enclosure, so I'm using an M1015 for that.

Why not order your Dell R730 with an Intel quad port 1GbE Network Daughter Card instead of the Broadcom.

 

[viniciusferrao]

My supermicro box has a 3008 based LSI HBA. It's been working fine with the SAS3 backplane, but there are (or at least 'were') issues trying to connect an external SAS2 JBOD enclosure, so I'm using an M1015 for that.

Why not order your Dell R730 with an Intel quad port 1GbE Network Daughter Card instead of the Broadcom.

Already talked to DELL and they will change the Broadcom cards for the Intel ones.

At this moment I'm worried about the SAS Controller and the possibility of using the MD3820i as JBOD on the future.


Sent from my iPhone using Tapatalk

 

[depasseg]

I wouldn't sweat the HBA. Like me, you can always add one later if there is an issue.

 

[viniciusferrao]

I wouldn't sweat the HBA. Like me, you can always add one later if there is an issue.

Well, I ordered the machine with a PERC H330.

The major issue now is the giant elephant named PowerVault MD3820i. I don't know what to do with this guy...


Sent from my iPhone using Tapatalk

 

[depasseg]

Isn't that an iscsi device? not DAS? Return it and get something like an MD3460.

 

[viniciusferrao]

Isn't that an iscsi device? not DAS? Return it and get something like an MD3460. :)

Yep, that's the problem. I've asked to DELL and they said that this storage can export a RBOD (I dunno WTF is RBOD) via the external SAS connector. The dual controllers on this guy uses iSCSI and SAS ports.

Unfortunately I don't know if we can return this MD3820i. I really would like to, but the return policy isn't available here in Brazil. If we can use this MD3820i as a JBOD would be the excellent solution. If not we will try under the juridical department to get a replacement on this box (since it isn't behaving as good as it was marketed).