LSI 9300-8i random read/write performance gap using SATA SSD

[PhaseMelter]

Hello FreeNAS users,

I have been reading up on FreeNAS for several years now and finally decided to built my own. Currently I am testing with the hardware I have at hand. To extend my testing, I recently bought an LSI 9300-8i card with the intention to run multiple SATA SSDs off it. I ran some benchmarks with a single Samsung 840 EVO connected to the LSI card and to the chipset SATA controller (Intel X99).

The results for sequential read and write are as expected, similarly for random read, but the random write tests perform an order of magnitude worse when connected to the LSI card. See the benchmark results using AS SSD Benchmark on Windows 10 below. The first screenshot is of the Samsung 840 EVO connected to the chipset and the second screenshot is of the LSI card.

The LSI card is flashed with IT firmware version 15. I believe the latest version is 16. I did not see anything in the release notes that led me to believe version 16 would fix the problems.

I found several sources on problems related to the Samsung 840 series SSDs in combination with LSI 92xx cards, but nothing on the 93xx cards:

https://forums.servethehome.com/index.php?threads/lsi-9212-4i4e-raid0-2xsamsung-840pro.2094/
https://forums.servethehome.com/ind...random-read-write-performance-with-ssds.2239/

For comparison, I also tested the same configuration with CrystalDiskMark and got different results. They are not as 'bad' as the previous benchmark, but it still shows a performance gap on the random read and write tests. The left screenshot is the chipset SATA again and the right screenshot is the LSI card.

Is such a performance gap expected? Does the LSI card add that much overhead?

Besides the performance gap, the Samsung 840 EVO firmware number is also not correctly passed by the LSI card, see the AS SSD Benchmark screenshots. I am not sure if this has any consequences, but I thought I mention it.

If anyone could shed some light on the matter, that would be much appreciated

:)

 

[Ericloewe]

Is the card connected to the PCH or the CPU?

 

[PhaseMelter]

Is the card connected to the PCH or the CPU?

Since the system I was initially testing on was an Asus X99 WS-E motherboard, with PLX chips on all PCIe lanes (both from the CPU and the PCH), swapping the LSI card between slots did not result in different scores.

The goal was to move away from this board anyway, and I ended up testing the LSI card on a Supermicro X10SLL-F with a Xeon E3 1230 v3. I plugged the LSI card into the PCIe slot with access to the CPU PCIe lanes and the following are my results using a Samsung 850 EVO 1TB connected to the LSI card. I noticed the bad partition offset. The reason is that I only had this SSD free at the time of testing, which already housed multiple partitions created on macOS.

As you can see, the AS SSD Benchmark scores went up significantly compared to the LSI card on the Asus X99 WS-E. Especially the write score. However, it is still less performant (while being the only device plugged into the LSI card) than it being connected to a PCH SATA port.
The read score is in the same ballpark as for the PCH connected Samsung 840 EVO. It is not a fair comparison (840 vs 850), but it is all I could get for the moment.

The CrystalDiskMark score is significantly worse compared to the 840 on the LSI card on the Asus X99 WS-E. This sounds odd to me. Up to a factor of two difference, for example both the 4KiB Q8T8 and Q12T1 write scores. Then again, the 4KiB Q1T1 write score sits in between the PCH and LSI connected scores for the 840.

Overall, the order of magnitude difference as first observed with the AS SSD Benchmark is gone, be it by using an entirely different system. This lead me to believe that the PLX chips on the Asus X99 WS-E have something to do with it. If anyone has an alternative interpretation, I am all ears.