Has Anyone Tried using LSI Warpdrive/SUN F40 Type Cards for ZIl/Cache?

[Matt Mabis]

I have been interested in trying out one of these cards, i have seen a lot of the reports on them, and b/c they are PCIe i would assume some of the overheads like SAS might even make it lower latency than some of the options out there...

Since the Card also makes 4 separate Drives (4x100GB) i was wondering if i could use the same card for ZIL/Mirrored Cache or would that be a bad idea? Picked on up on Ebay for 80$

SUN / Oracle 7070787 / 7026993 Flash Accelerator F40 400GB Solid State Memory
Ebay Link

SUN Data Sheet on the F40
Sun Link

KEY FEATURES
--400 GB capacity
--Up to 149K IOPS (8K) performance
--Over 2.0 GB/s throughput
--95 microsecond write latency
--Embedded Flash controllers for greater performance , compatibility and low CPU burden
--Advanced write endurance
--Proactive monitoring features
--Low-profile PCIe form factor
--Enterprise reliability

STH Forum Talk about the Card
STH Linkhttps://forums.servethehome.com/ind...accellerator-4x-100gb-ssd-85-usd.15342/page-2

But based on the specs i don't see why it wouldn't be a bad idea, just wanted to get some feedback of those that might have tried? There were some discussions but never really got rooted out so i wanted to see if anyone has had any of these running/working especially on a FreeNAS implementation.

My FreeNAS Setup is 11.0 - with 8x8TB (Raid-Z2 - Primary) and 8x4TB (Raid-Z2 - Backups) configurations... i would only have the device running on the 8x8TB Config

 

[_bolek_]

Was this drive showed on FreeNAS? I mean did you put them inside FreeNAS and check, is there drivers or did you check if there are not dedicated to specific raid controller?
For what I know, it should be possible using this drive as cache but ... if is used i would advise not using this drive (first check TBW).

 

[Matt Mabis]

@_bolek_ I just got the card yesterday, from what i can tell within the LSI firmware they register as an HBA 4x100GB Hard Disk setup. When I booted to FreeNAS 11.0-U1 the drives did show and populate so it does find a driver within FreeNAS for this unit to function.

From an Endurance stance there really isn't much information on it, the one thing I can mention is that NetApp use to use these as cache cards for their storage arrays which to me would sound like they should have some good endurance to them.

I'll run some more tests to see what kind of throughput I can get out of the default firmware/driver that it runs off of...

 

[svtkobra7]

There isn't much info on this forum regarding the Sun F40 / F80, and I just purchased/installed a couple, so I figured that I would post some feedback for the possible benefit of others. I originally intended on using for the reason the OP posted, but ended up taking a different path.

High Level Specs

• As per prior comment, they are 4 x 100 GB flash modules (32 nm eMLC NAND).
  
• PCI-E 2.0 x8 host interface
• LSI 2008 SAS/SATA 2 x4 6 Gbps (bootable via OPROM)
• Data sheet

Value Proposition

• I purchased two used drives on ebay for $60 each less $15 coupon.
• I figure that 800 GB of SSDs for $105 is not a bad deal.

Usage / Experience

• I had originally intended to use one as ESXi boot + datastore and use the other for SLOG / L2ARC, but ultimately headed a different route.
• Used the WarpDrive Utility to:
  • Check drive health (both report 100%)
  • Update firmware (reference below)
  • Format all drives / Over provision @ 28% ("Performance over provisioning" @ 50% also available) / Enable UNMAP
• After researching SLOG / L2ARC more, and considering my server configuration and use case, I ended up using as follows:
  • Drive 1: ESXi boot + datastore0 (note they show as 4 x 93GB drives in ESXi, but of course can be aggregated into a single datastore).
  • Drive 2: Configured pass through to FreeNAS, set up new zpool and striped the 4 drives. Created zvol (compression = off / size = 300 GB / sparse volume = checked). Configured iSCSI. Created datastore1 in ESXi.
• Tested using "Drive 1" = Not Amazing. NOTE: Bottom right is a point of reference only and not this disk.

The write speeds reported by CrystalDiskMark (Writes/Seq Q32T1) look really low.

• Tested using "Drive 2" = Much Better (as expected, but still not as great as I would expect). NOTE: Bottom right is a point of reference only and not this disk.

• Miscellaneous Remarks:
  • Quite happy with the purchase, although I expected performance to be stronger. Note that I included a benchmark on my zpool which consists of 3 2 HDD drive mirrors.
  • While I have a fully functional ESXi AIO server with pfSense, FreeNAS, etc. I have MUCH to learn in regards to ESXi optimization (as well as iSCSI). Compared to what the drive is capable of, I would assume that I haven't configured everything properly and to that end, appreciate any feedback / advice that can be offered.
  • The drive includes low profile brackets so I had to source a couple of full height brackets for my SC836 on eBay. A standard full height LSI bracket works just fine. You lose visibility to the 3 status LEDs (no big deal).
  • Based upon other users' remarks (STH) these drives can run very hot. Since I added it to @Seren 's FreeNAS temperature graphing script (which I can't thank him enough for), I've seen temps in the range of 48°C - 53°C, which causes me no concern considering my HDD temps are managed by script to a max of 40°C. Before they arrived, I contemplated whether I may need to add a fan to cool these drives & my LSI, but they seem to be just fine.

Reference

• Nytro WarpDrive Utility Management (115.00.15.00) can be used to flash firmware / manage drive.
• FW 2.0 download requires an Oracle Support account, but can be found here: F40 / F80 Firmware

 

[Ericloewe]

Another LSI SAS2008-based SSD? Interesting. Intel has one (in fact, several people have managed to accidentally flash it with stock LSI firmware, kinda rendering it useless).

 

[svtkobra7]

Another LSI SAS2008-based SSD? Interesting. Intel has one (in fact, several people have managed to accidentally flash it with stock LSI firmware, kinda rendering it useless).

It is OLD (2012) ... Identical to the LSI Nytro WarpDrive BLP4-400 (which certainly you have heard of). While both were originally created to serve as flash accelerators, the fact that it is SAS2008-based means re-purposing it for a completely different use case works out just fine for me! Just wish I could figure out why I'm not seeing better #s.

 

[Ericloewe]

Well, SSDs have come a long way, mostly due to improved controllers. That's why these things where attached to an SAS2008 in the first place, to provide more interesting performance as a group.

 

[falchionwielder78]

Does anyone know if it’s possible to use an LSI Warpdrive as an install drive? What would the best use of one of these be in a FreeNas server?

 

[Ericloewe]

A 30-buck SSD sounds like a better option in almost every way... I mean, the LSI SAS2008 alone uses some 10 W.

 

[falchionwielder78]

It would be except it’s a Supermicro x11ssm-f build with 8x4tb drives. The board only has 8 satas so my options are: boot from usb, use a sas pcie card or install to the warp drive.

My question is mainly pertaining to feasibility, though I am interested in the drawbacks as well so thank you for the power consumption consideration as I honestly hadn’t considered that.

 

[Ericloewe]

boot from usb, use a sas pcie card or install to the warp drive.

Or get a cheap NVMe M.2 SSD plus an adapter card for not much more than the SATA SSD.

 

[falchionwielder78]

Or get a cheap NVMe M.2 SSD plus an adapter card for not much more than the SATA SSD.

That's a good idea... I have a spare m.2 drive that I could use for that purpose.

One idea that keeps jumping out at me is whether I would gain anything by using the Warp drive as a ZIL or SLOG, or maybe both since the 480gb of data is broken out into 4 visible ssds by the OS. Thoughts?

 

[Rototype]

Just picked up one of these from eBay, 1.86TB for less than $200 - Not sure M2 or SATA drives for that capacity are any cheaper so I'll persevere with it. (I already have a 400GB one that I'm using as a W*****s 10 boot drive on another PC, that when I first loaded it was stunning in performance and even now is almost as quick as a lot of modern SSDs). Going to add the 1.86TB as a secondary RAID in my box, the first consisting of 5 x 6TG WD Reds. Might make a good video playback drive.