Is a SSD for L2ARC worth it for me

[Itamar Croitoru]

Hello,

Here's my current setup:
Server: Dell PowerEdge R310
CPU: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz
RAM: 48 GB ECC 1333
Boot drive: 60 GB Commercial SATA SSD
FreeNAS: Latest production 9.10
Storage Controller: HP rebrand of LSI 9207-8E flashed with IT build
Storage Array: Compelent 24 bay SAS array DAS
Drives: 14 x 146 GB 15,000 RPM SAS drives
zpool: 1 x RAID 10
Shared as iSCSI through 2 x Dell PowerConnect 6248 stacked switches
Used as Datastores for 2 x VMware ESXi 5.5 Update 3e

I'm currently seeing pretty good performance with the above setup. Don't get me wrong, it took a lot of work, research and testing to get it working, but I'm considering performance upgrades.

My question:
After finding some good articles, it seems that for my usage case above, a 120 GB SSD in the storage array acting as L2ARC can sometimes be a performance boost and put simply I'm asking if that's the case.

Figuring it's based on my currently ARC status, here are the stats I can think now to provide:
ARC Size: 25.6 GB Avg 37.6 GB Last
ARC Hit Ratio: 53.4% Avg 53.4% Last

From what I've read, I think it may be a performance improvement to spend on the SSD for L2ARC but I'm looking for confirmation.

Can you tell from the above?
Do you need other info?
What do you think?

* I am not seeing performance problems but I do have some budget to improve the storage now, so why not invest before it's an issue?

Thanks

 

[Spearfoot]

Hello,

Here's my current setup:
Server: Dell PowerEdge R310
CPU: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz
RAM: 48 GB ECC 1333
Boot drive: 60 GB Commercial SATA SSD
FreeNAS: Latest production 9.10
Storage Controller: HP rebrand of LSI 9207-8E flashed with IT build
Storage Array: Compelent 24 bay SAS array DAS
Drives: 14 x 146 GB 15,000 RPM SAS drives
zpool: 1 x RAID 10
Shared as iSCSI through 2 x Dell PowerConnect 6248 stacked switches
Used as Datastores for 2 x VMware ESXi 5.5 Update 3e

I'm currently seeing pretty good performance with the above setup. Don't get me wrong, it took a lot of work, research and testing to get it working, but I'm considering performance upgrades.

My question:
After finding some good articles, it seems that for my usage case above, a 120 GB SSD in the storage array acting as L2ARC can sometimes be a performance boost and put simply I'm asking if that's the case.

Figuring it's based on my currently ARC status, here are the stats I can think now to provide:
ARC Size: 25.6 GB Avg 37.6 GB Last
ARC Hit Ratio: 53.4% Avg 53.4% Last

From what I've read, I think it may be a performance improvement to spend on the SSD for L2ARC but I'm looking for confirmation.

Can you tell from the above?
Do you need other info?
What do you think?

* I am not seeing performance problems but I do have some budget to improve the storage now, so why not invest before it's an issue?

Thanks

Welcome to the forums!

Did you mean to say your zpool is a RAIDZ2? Or is it some kind of mirror setup? I hope you're using mirrors because they give the best performance when providing block storage.

I ask because 'RAID 10' terminology doesn't apply with ZFS. Passing through a RAID-10 array to FreeNAS is not recommended as doing so prevents it from performing very important housekeeping tasks to test and maintain the pool's health.

With respect to L2ARC, the rule of thumb is to maximize the RAM in your system first and only then consider adding an L2ARC device if performance is still inadequate. Have you maxed out the RAM on this system?

When providing block storage, you want to turn on synchronous writes for the dataset, and this means you need a SLOG device or else performance will be awful. Good SLOG devices have power protection, high durability, low latency, and fast writes. The Intel DC S3700 SATA SSD is a good entry-level SLOG device, with the Intel 750 and DC P3700 NVMe devices being better and best, respectively, while also being more expensive.

TL;DR: Maximize RAM first, turn synchronous writes on for the block storage dataset, install a good SLOG device... and then think about adding an L2ARC.

Good luck!

 

[Itamar Croitoru]

I did mean to say RAID 10 for my zpool, as the UI in 9.10 does offer a RAID 10 option which in reality is the exact same configuration of drives as a RAID 10 but using ZFS directly. So the RAID 10 equivalent for FreeNAS, or 'mirrors'.

I have maxed out the R310 with 48 GB of ECC RAM, so I'm asking if a SSD such as the Samsung 750 EVO 120GB, plugged into the array or the server and used for L2ARC would improve my performance:

As for the SLOG, that means setting up a ZIL? If so, why such expensive drives? I get some performance but do they really have battery backup?

Would an Intel 540S 240 GB drive work or the Intel S3500 80 GB?

 

[Spearfoot]

I did mean to say RAID 10 for my zpool, as the UI in 9.10 does offer a RAID 10 option which in reality is the exact same configuration of drives as a RAID 10 but using ZFS directly. So the RAID 10 equivalent for FreeNAS, or 'mirrors'.

Yes, sir. Pardon my asking, but I've never seen 'RAID 10' in the volume layout dropdown list; only mirrors. And we get a lot of folks here who use a RAID controller and pass various classic RAID arrays to FreeNAS... :)

I have maxed out the R310 with 48 GB of ECC RAM, so I'm asking if a SSD such as the Samsung 750 EVO 120GB, plugged into the array or the server and used for L2ARC would improve my performance.

It might indeed! But from a performance standpoint and based on your use-case (providing block storage to ESXi) you'll be better off setting up a SLOG device first.

As for the SLOG, that means setting up a ZIL? If so, why such expensive drives? I get some performance but do they really have battery backup? Would an Intel 540S 240 GB drive work or the Intel S3500 80 GB?

All FreeNAS servers have a ZIL, located in memory. The SLOG device is a 'Separate LOG' device that works with it.

The drives are a little pricey, but this is because they're designed with characteristics tailored for use as a SLOG device. FWIW, you can pick up new 100GB Intel S3700s on eBay for $100-150US. You can use an S3500, but the S3500 is optimized for reads and general-purpose use; the three devices I mentioned are optimized for writes -- which is all a SLOG device does. A common garden-variety SSD won't work well as a SLOG device, as it doesn't have the features I mentioned. The power protection is provided by small capacitors which store enough charge to give the gizmo time to flush its buffers if the system goes down. Here's some good reading on the subject of ZIL and SLOG devices:

https://forums.freenas.org/index.php?threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/
http://nex7.blogspot.com/2013/04/zfs-intent-log.html

 

[Itamar Croitoru]

In that case I'll be setting up both.

I can purchase the L2ARC drive tomorrow so I'll get that first and see what I can find on EBay for the Intel DC S3700.

If I recall my reading correctly, a single S3700 of say 400 GB being used for both SLOG and L2ARC would not be smart, right? It's better that each part run on its own SSD?

I ask because I may have a line on a new Intel S3700 but it's the 400 GB drive and it would be a waste to buy that for a few gigs of used SLOG..... actually, how big of a SLOG should I be setting up?

 

[Spearfoot]

In that case I'll be setting up both.

I can purchase the L2ARC drive tomorrow so I'll get that first and see what I can find on EBay for the Intel DC S3700.

If I recall my reading correctly, a single S3700 of say 400 GB being used for both SLOG and L2ARC would not be smart, right? It's better that each part run on its own SSD?

You're right - it's not a good idea to use the same device for both SLOG and L2ARC.

I ask because I may have a line on a new Intel S3700 but it's the 400 GB drive and it would be a waste to buy that for a few gigs of used SLOG..... actually, how big of a SLOG should I be setting up?

The SLOG doesn't need to be very large - details available here. I know that it seems like a waste, but one advantage of using larger SSDs is that they're typically faster than the smaller ones.

 

[Arwen]

To be clear, the ZIL, (ZFS Intent Log), is actually in the pool. It's used to make a persistant
copy of synchronously written data before the client is told the write is done. A copy of the
data still remains in RAM until the next ZFS write transaction group is written to the actual
vDevs for permenant storage. Then the ZIL entry is no longer needed.

So a ZIL and SLOG are write only unless the FreeNAS server crashes. Then the intent log,
(SLOG if configured, ZIL if SLOG is not used), is played back to update the vDevs.

Basically, "sync=always" is very important for VM's backing store's data integrity. And if
that makes things a bit slow, then a SLOG can help.

A bit confusing I know. It will get worse in a year or 2 whence ZFS Metadata cache devices
become available.

 

[Itamar Croitoru]

I have been looking for deals on one of these drives and I finally found a deal on two BNIB Intel S3710 400 GB SSDs for less then one at any retail location. So I picked up two.

I'm currently looking to install one as the SLOG but now need to purchase the mounting bracket and then I'll start setting that up.

I do want to ask though what is the best use of my second one. Here are the options I have thought of:
1. I could install it as well in the same server and use it for L2ARC.
2. I could wait for my new hardware to come in where I will have 12 x 7,200 RPM 2 TB SATA drives in a similar configuration as my larger but slower iSCSI datastore in VMware. Then I could use the second S3710 for this server but then that begs the question about if it should be used there also for SLOG or L2ARC.

Currently I am putting the medium-high performing DBs on the higher speed iSCSI server so the other one would handle all other VM drives such as web servers, OS drives, low-perf DBs, etc... VM swap is handled by local SSD so not needed on there.

Anyone have opinions on what would be the best usage of my BNIB S3710?

 

[marshalleq]

As far as I knew, a slog device should be mirrored no?

 

[sretalla]

As far as I knew, a slog device should be mirrored no?

That "requirement" went away some time back... battery backed and superfast performance are the only requirements.

 

[sretalla]

that begs the question about if it should be used there also for SLOG or L2ARC

If we're still talking (more than 2 years after the OP) about a box with less than 64GB of RAM, L2ARC is unlikely to make a positive shift in performance.

 

[marshalleq]

Wow, thanks for your update - that changes things quite a bit!

 

[HoneyBadger]

This is a necropost certainly.

A missing SLOG device in import no longer causes pool failure, but you will still have data loss if there are pending writes that haven't been committed to the vdevs. You can force the import by discarding the pending txg.

Mirrored SLOG is still recommended in situations where any data loss at all is considered unacceptable.