Interesting Post on Experimental iSCSI Performance

[DaveFL]

I came across this and found it to be an interesting read since it sets the expectations of what one could expect with a typical home user NAS setup.

What I do find odd is that their read performance decreases significantly with the new driver. Perhaps one of the pros here can share their thoughts.

http://pivotallabs.com/high-performing-mid-range-nas-server-part-2-performance-tuning-iscsi/

Full Disclosure:

I have no affiliation with the author of this post.

 

[cyberjock]

Nice read. He seems to make a valiant effort to do a thorough job, but there's a few things I don't like about the post:

1. He uses bonnie++ but doesn't provide the actual parameters he used.
2. Putting the L2ARC and SLOG on the same device is just stupid. The L2ARC is designed to only be used when you aren't using the disk for other activities. Surprise! An SLOG, by its very nature and if used properly (which is the only time you should have an SLOG at all) would mean you're locking out the L2ARC. This is a noob mistake and pretty much proves he's not the expert ZFS user he might sound like from his read.
3. He's running RAIDZ2. Most people doing iSCSI are, by the nature of iSCSI, going to be IOPS bottlenecked. This also means that anyone that understands ZFS would go with mirrors and not RAIDZ(x). So he literally tested a configuration that is not likely to ever reflect real-world values except to people that don't really understand ZFS (or think they are going to save money by going with RAIDZ2 over mirrors).
4. Things like block size, file-based versus zvol-based have a TREMENDOUS impact on throughput. zvol is a guaranteed win over file-based when doing benchmarks, yet he made no mention of which he used.
5. We know from experience that pretty much every SSD out there except the most expensive Samsungs and the Intel S3700/330 don't make great SLOGs. He used a Corsair. :(

Overall, it seems like he tried really hard, but is still missing so many fundamental aspects of ZFS that he really didn't prove a thing because his configuration would never ever be used in a production environment, except by someone that doesn't know any better. Unfortunately, this also literally invalidates all of his results because a given percent improvement for some non-standard non-proper configuration means absolutely nothing.

Very well written by him though and I give him props for trying.

Just for the record, I don't try to benchmark ZFS any more than I have to because it is a disaster if you aren't a ZFS "God" (yes, with a capital-G). I don't consider myself to be that level yet. I can find obvious flaws in benchmarking, but I bet if I did my own benchmarking I'd probably screw it up too.

 

[DaveFL]

He actually did include all of his commands.

https://github.com/cunnie/freenas_benchmarks


Thanks for sharing your thoughts. Good to know that if one has any intent of using iSCSI they should be using RAID10. I still do find it odd how poorly the kernel iSCSI driver performed in comparison to the user land iSCSI driver for reads on the baseline. None of your points touch on that.

 

[cyberjock]

Thanks for sharing your thoughts. Good to know that if one has any intent of using iSCSI they should be using RAID10. I still do find it odd how poorly the kernel iSCSI driver performed in comparison to the user land iSCSI driver for reads on the baseline. None of your points touch on that.

Please don't confuse the complexity of the situation. It's not a foregone conclusion (at least not exactly) that if you do iSCSI you *must* use mirrors. The issue depends on the workload. More than 99% of the time the workload is random I/O. Random I/O only works even remotely reasonable with mirrors unless your workload is nearly zero 99.999% of the time.

RAID10 is a hardware RAID type, mirrors is for ZFS. You might not want to say RAID10 as you might get lynched in here for using improper terminology.

 

[mav@]

I just committed patch for kernel iSCSI to FreeNAS 9.3, that should fix reported problem with sequential read speed. It should be available in next nightly build. It works good on my test, but it would be good to have more test reports before official 9.3 release.

I short, kernel iSCSI appeared to be too fast/multithreaded for present ZFS read-ahead implementation. :)

 

[DaveFL]

I just committed patch for kernel iSCSI to FreeNAS 9.3, that should fix reported problem with sequential read speed. It should be available in next nightly build. It works good on my test, but it would be good to have more test reports before official 9.3 release.

I short, kernel iSCSI appeared to be too fast/multithreaded for present ZFS read-ahead implementation. :)

Do FreeNAS and FreeBSD share any code base with respect to the iSCSI driver?

 

[mav@]

Do FreeNAS and FreeBSD share any code base with respect to the iSCSI driver?

They share most. iSCSI of FreeNAS 9.3 is almost equal to FreeBSD HEAD's.

 

[jgreco]

Random I/O only works even remotely reasonable with mirrors unless your workload is nearly zero 99.999% of the time.

Write workload. Read workloads with RAIDZn are generally fine but may be faster with mirrors. It is the write workload which is truly problematic. And you're exaggerating, though I appreciate the sentiment.

You might not want to say RAID10 as you might get lynched in here for using improper terminology.

I'll quote something like that back at you the next time you say ZIL when you mean SLOG. ;-)

 

[depasseg]

Write workload. Read workloads with RAIDZn are generally fine but may be faster with mirrors. It is the write workload which is truly problematic. And you're exaggerating, though I appreciate the sentiment.



I'll quote something like that back at you the next time you say ZIL when you mean SLOG. ;-)

And who put RAID10 in the Wizard?

 

[jgreco]

Not me.

 

[Ericloewe]

And who put RAID10 in the Wizard? :)
View attachment 5735

He who finds it gets to write the ticket for new nomenclature for FreeNAS 10.1.