Best use of a PCI E Intel 750 400Gb SSD

[Darren Myers]

Just picked up another Intel 750 400Gb PCI-E SSD for cheap and am considering adding it to my FreeNAS server, my server typically servers Plex media via Plex plugin, and serves up CIFS shares.

Whats the best way to utilize this SSD

Specs:

FreeNAS-9.10.1 (d989edd)
Supermicro X9DR3-LN4F+ , dual E5-2670 @ 2.60Ghz
96GB of RAM
9211-8i , IT Phase 20
Dual 80GB Intel SSD's for boot
x7 4TB WD Red's RAIDz3
x14 2TB WD RE4's Striped RAIDz3
x14 2TB WD RE4's Striped RAIDz3
Chelsio T420-CR 10GB card
Intel quad port gigabit NIC
Supermicro 847E16-R1400LPB case

 

[Spearfoot]

Just picked up another Intel 750 400Gb PCI-E SSD for cheap and am considering adding it to my FreeNAS server, my server typically servers Plex media via Plex plugin, and serves up CIFS shares.

Whats the best way to utilize this SSD

Specs:

FreeNAS-9.10.1 (d989edd)
Supermicro X9DR3-LN4F+ , dual E5-2670 @ 2.60Ghz
96GB of RAM
9211-8i , IT Phase 20
Dual 80GB Intel SSD's for boot
x7 4TB WD Red's RAIDz3
x14 2TB WD RE4's Striped RAIDz3
x14 2TB WD RE4's Striped RAIDz3
Chelsio T420-CR 10GB card
Intel quad port gigabit NIC
Supermicro 847E16-R1400LPB case

It would make a good SLOG device... but you may not even need one. You could use it for L2ARC, too, but best practice is to max our RAM before worrying about L2ARC.

 

[Mirfster]

Just picked up another Intel 750 400Gb PCI-E SSD for cheap and am considering adding it to my FreeNAS server, my server typically servers Plex media via Plex plugin, and serves up CIFS shares.

Whats the best way to utilize this SSD

By wrapping it up nice and neat; then sending it to me for *indefinite evaluation*... ;)

Since you have 96 GB of RAM, can try it as a L2ARC. Heck or even try it as a SLOG. With either of these, you can add and remove them from a Volume without concerns, so feel free to give it a whirl and see where it works best for you.

 

[Stux]

By wrapping it up nice and neat; then sending it to me for *indefinite evaluation*... ;)

Since you have 96 GB of RAM, can try it as a L2ARC. Heck or even try it as a SLOG. With either of these, you can add and remove them from a Volume without concerns, so feel free to give it a whirl and see where it works best for you.

Why not both?

Oh I know, pre-existing wisdom is that you don't want to use an SSD for both SLOG and L2ARC. This is because of performance bottleneck problems, cost concerns, and power loss concerns. Thing is, this is a high performance SSD with PLP, shouldn't suffer from those bottleneck issues, and we're not talking about gigabit... at least I'm not. We're talking 10gbe.

Oh look. I asked the same thing before

Was thinking of using an Intel P750 400MB

It's overkill for a SLOG, but has PLP. Would make more sense if I also used it as mostly L2ARC.

It seems like the concern is that a low end SSD would get bottlenecked with the reads/writes, but I'm not sure that is the case with a high end non gumstick ssd.

BUT, I can get 1+GB/s out of two cheap striped sata SSDs, so as you suggest that might be the way to go, at least at first for L2ARC

I still think that using a P750 for both a SLOG and L2ARC is in fact a good idea, the only issue is that you'll have to do some command line kung fu to get it to work.

as per @jgreco

Well, another reason is that FreeNAS doesn't support combined SLOG and L2ARC. You can hack around that with manual ZFS commands, but it really isn't a good idea.

The Intel 750 is a nice choice for a budget SLOG device.

The command line kung fu is actually fairly simple. The most complicated part is finding the right gptid.

And for those wondering, I also asked about the reasons for not using SLOG/L2ARC on the same device

While we're discussing L2ARC/SLOG

Why is partitioning a single NVMe device into a small SLOG and a large L2ARC discouraged?

In my situation, I'm planning on running iSCSI and filesharing over 10gbe. Although there will be a large static file base, the working set would be just a few hundred GB.

 

[Mirfster]

I still think that using a P750 for both a SLOG and L2ARC is in fact a good idea, the only issue is that you'll have to do some command line kung fu to get it to work.

Possibly, but I would think that it will put a lot more wear on the same drive. Out of curiosity, I looked up the specs of the P750 and was comparing it to my S3500 (160GB) and S3710 (200GB) drives.

What I found interesting is that while I did not see an "Endurance Rating" for Writes Per Day for the S3500 (they do have total - TBW); the P750 and S3710 were significantly different:

P750 = 70GB Writes Per Day (127 TBW)
S3710 = 10 Drive Writes Per Day; So my 200GB can do 2000GB Per Day (3.6 PBW)

So going off that, I would wonder how close one would be to the "Writes Per Day" if the P750 were just used as a SLOG, let alone it being used as a L2ARC as well?

Not knocking the P750, but looking at it from a "numbers" perspective may warrant a little concern regarding life span; especially when it is used as both SLOG and L2ARC...

 

[HoneyBadger]

I still think that using a P750 for both a SLOG and L2ARC is in fact a good idea, the only issue is that you'll have to do some command line kung fu to get it to work.

It's possible but I don't know that I'd call it a "good idea" because of the unpredictable elements it introduces. Will an incoming burst of SLOG writes cause your L2ARC reads to take too long? Or will a bunch of L2ARC reads make SLOG writes pile up, trashing your latency?

The reason it's considered silly is because while SLOG has some very stringent requirements regarding power-failure protection and low write latency, L2ARC can be basically any cheap SSD you can get your hands on. Why potentially screw up performance for the sake of saving $100 on some MLC NAND?

 

[MatthewSteinhoff]

I understand the dream. The advantage of using a single, high-quality, data-center-grade PCIe-based device for SLOG/L2ARC is speed, for sure, but also physical space.

Drive bays are gold. It's hard to give up a slot to a puny SSD which could be used for a high-capacity hard disk. If I could use one PCI slot to host a device that is five times faster than a regular SSD, I would carve it up into three or four SLOGs (and maybe some L2ARC) that are still faster than three or four SATA devices that eat up three or four drive bays. Even with the significant price penalty, I'd gladly pay for PCIe-based device if it gave me back a meaningful number of bays.

So, one of these days, I hope someone will test and bless this configuration.

Cheers,
Matt

 

[HoneyBadger]

"In theory" if you could somehow ensure that the combined IOPS of "SLOG writes" and "L2ARC reads" would never be more than "total IOPS the device is capable of serving" then it would work.

In the case of a single device being partitioned into multiple SLOGs, you might be able to somehow bodge that into existence by modifying or tuning the ZFS write throttle (since the typical "write_limit" tunables affect TXG size and staging to permanent vdevs, not the initial ingest to SLOG).

But "in theory" communism works.

 

[MatthewSteinhoff]

Like I said, @HoneyBadger, I understand the dream.

Cheers,
Matt

 

[HoneyBadger]

Sounds like what we need is an NVMe version of the Sun F40 (4x100GB eMLC modules on a PCIe card, presented via an internal SAS HBA)

 

[Stux]

Drive bays are gold. It's hard to give up a slot to a puny SSD which could be used for a high-capacity hard disk. If I could use one PCI slot to host a device that is five times faster than a regular SSD, I would carve it up into three or four SLOGs (and maybe some L2ARC) that are still faster than three or four SATA devices that eat up three or four drive bays. Even with the significant price penalty, I'd gladly pay for PCIe-based device if it gave me back a meaningful number of bays.

Exactly,

You're talking 20000..200,000 iops for the p750

 

[wtfR6a]

If test it as L2arc and slog in your system with your typical usage. I'd expect CIFS and plex to not be worth the hassle and additional complexity. I benchmarked my system with P3700s and 750s and it really wasn't with the trouble. Your mileage may vary etc.
Certainly the benefits of having round the GUI are questionable to combine l2arc and slog on one device for the gains.

 

[Stux]

If test it as L2arc and slog in your system with your typical usage. I'd expect CIFS and plex to not be worth the hassle and additional complexity. I benchmarked my system with P3700s and 750s and it really wasn't with the trouble. Your mileage may vary etc.

Were you using 10gbe?

 

[wtfR6a]

Were you using 10gbe?

Yes.