Database performance and vfs.zfs.cache_flush_disable

[Mlovelace]

Looking at the "readme" included in the P16 firmware package makes me think FreeNAS should move off this particular driver/firmware version. LSI recommends moving to P17 in the documentation:

"Integrated RAID (IR) configuration can show an HDD as offline under UEFI during OS installation, causing the installation to fail

Required Configuration:

Phase 16 IR Firmware

Volume over 2TB under any OS but only with UEFI

Due to the possibility of this issue being experienced in the field, LSI recommends customers with the aforementioned configuration update their IR firmware at the earliest opportunity to Phase 17".


EDIT: I spoke with LSI tech support and this is a known issue with the Phase 16 firmware and they recommend moving to the current Phase 20 firmware/driver combination for the best drive support and optimization, however Phase 17 should be adopted at a minimum.

I also included this in the bug report.

 

[mjws00]

Wonder if that applies to IT firmware as well?

 

[Mlovelace]

Wonder if that applies to IT firmware as well?

I asked that and he said the warning was for Phase 16 firmware, to include IR and IT, however he had not received a call about IT specifically since the issue was identified.

I don't know if that means no news is good news about the IT side of things, but he was insistent that Phase 17 should be the minimum.

 

[Ericloewe]

It only affects people who boot from LSI controllers, the way I read it. OSes see the drives just fine, it's just the UEFI environment that has issues (regulars will certainly remember a case or two of people asking why their LSI boot ROM won't show their drives, even though FreeNAS works fine - we have our answer). Everyone affected raise their hand, please.

*crickets*

 

[cyberjock]

And the last 4 posts sum up the problem PERFECTLY. Not all LSI bugs affect IT firmware and usablity. The fact that the LSI recommendation says "for IR" should be your clue that the recommendation isn't applicable for IT firmware. In fact, the *entire* FreeBSD LSI community yses p16 by default. Being that FreeBSD 9.3 came out like 2 months ago I think it's safe to say that P16 is probably a good place to be.

Honestly, not sure why we are still having this discussion. The developers have decided what we are going to use and unless you can provide a reproducible example of a bug in the P16 IT firmware that actually affects people using the card as it is designed in FreeNAS then your comments are without merit. The IR bugs from P16 to P20 have been fairly serious in 2 or 3 cases. But we aren't using IR mode, we aren't using the hardware RAID, and we aren't booting from the LSI controller.

 

[Mlovelace]

The fact that the LSI recommendation says "for IR" should be your clue

That was my first clue which is why I called LSI to confirm the IT firmware was unaffected. Their response was somewhat ambiguous as he said all Phase 16 but hadn't seen an issue with IT.

 

[cyberjock]

LSI, as a general rule, has a hard time buying into the "IT" crowd. Why would you buy a RAID controller, then disable the reason for buying a RAID card. So no only are their answers somewhat ambiguous but they still can't believe people would do IT mode. The only good metric you has is the fact that with P16 drivers and P16 firmware the number of issues that have been identified to be directly attributable to firmware is nearly zero. Even on cases like the one you linked in the bug report, that doesn't prove the problem is with LSI drivers or firmware. The problem could squarely be in the face of Seagate. But neither you or I are able to make that definitive a conclusion.

I used to side with you. Back in 8.x we were on P13 drivers and firmware and P16/P17 was the latest. I was disgusted that we weren't "on the latest". Ultimately the answer is "let the butthurt flow through you, then realize it's not something you have control of or should really care about". So I let the butthurt flow through me for a while, then accepted reality. The harsh reality is that unless you can firmly point the finger directly at LSI firmware/drivers the "old stuff" isn't really "bad". The race to have the newest and "bestest in the whole wide world" is a misnomer. Every time a new Windows version comes out it's crap for a while. For something that you want (and expect) to be rock solid you *don't* want the latest and greatest. You want it to work, and the P16 has done just that.

 

[Mlovelace]

I complete appreciate where you're coming from and agree. I don't even run that controller currently, I just had a passing interest from that bug report and thought I'd look at it.

Sorry for making everyone kick the proverbial dead horse again.

 

[cyberjock]

I complete appreciate where you're coming from and agree. I don't even run that controller currently, I just had a passing interest from that bug report and thought I'd look at it.

Sorry for making everyone kick the proverbial dead horse again.

You must be new here. Kicking dead horses is what we do like 70% of the time! :D

 

[EdvBeratung]

Even this thread has been hijacked for an LSI discussion ;) I still want to share with you my latest test results for those who are curious.

I recently saw a test of the quite new Samsung Server SSD 845DC Pro online and not only it was basically the fastest SSD in this test (Intel S3700 was listed there as well) it was also mentioned that this SSD delivered the most consistent results they have seen so far.
So I bought me a Samsung 845DC Pro 400GB and I was running the tests that I did before (and also the forum member titan_rw did). these are the results:

Code:

sync=always
cache flush enabled (vfs.zfs.cache_flush_disable not set)

Record Size 4 KB
File size set to 2097152 KB
Command line used: iozone -i 0 -r 4k -s 2g
Output is in Kbytes/sec

      KB  reclen   write rewrite
2097152       4   28069   28245



sync=always
cache flush disabled (vfs.zfs.cache_flush_disable=1)

Record Size 4 KB
File size set to 2097152 KB
Command line used: iozone -i 0 -r 4k -s 2g
Output is in Kbytes/sec

      KB  reclen   write rewrite
2097152       4   40864   41443

As you can see the performance during "normal" operation (with cache flush enabled) is about 50% higher than the Intel S3700 in the results that titan_rw posted (and btw around 1000% than the Samsung 850 Pro but I learned that already that they are no good ZIL).
Considering the 845DC Pro has all the nice features like capacitor cache protection, end to end protection, etc. for me the decision is made in terms of ZIL: I'll use the Samsung. Around 50% higher performance compared to S3700 for (in Germany) around 65% of the price of the Intel. The decision couldn't be any easier for me. :)

 

[mjws00]

Thanks for the testing. The specs and price on the 3700 have never really done it for me. Even if it's a proven workhorse. Someone needs to eat their lunch, imho.

 

[Ericloewe]

The S3700 was never designed to be fast. That didn't keep them from pricing it outrageously, unfortunately.