Tuning BIOS towards low latency and improve SLOG/sync write performance

[Ender117]

As we all know, sync writes are very sensitive to latency, and people are constantly looking for the best SLOG device with the lowest latency. However, SLOG is only part of the chain. I tried following this guide to tune my BIOS and found out it doubled my P3700's performance at lower recordsizes:

Code:

root@freenas:~ # diskinfo -wS /dev/nvd0
/dev/nvd0
	   4096			# sectorsize
	   42999996416	 # mediasize in bytes (40G)
	   10498046		# mediasize in sectors
	   131072		  # stripesize
	   0			   # stripeoffset
	   INTEL SSDPEDMD400G4	 # Disk descr.
	   CVFT7112001A400LGN	  # Disk ident.

Synchronous random writes:
		 4 kbytes:	 22.1 usec/IO =	176.7 Mbytes/s
		 8 kbytes:	 24.2 usec/IO =	322.4 Mbytes/s
		 16 kbytes:	 29.3 usec/IO =	533.3 Mbytes/s
		 32 kbytes:	 39.4 usec/IO =	793.8 Mbytes/s
		 64 kbytes:	 72.4 usec/IO =	862.7 Mbytes/s
		128 kbytes:	147.8 usec/IO =	845.8 Mbytes/s
		256 kbytes:	250.1 usec/IO =	999.5 Mbytes/s
		512 kbytes:	490.6 usec/IO =   1019.2 Mbytes/s
	   1024 kbytes:	961.4 usec/IO =   1040.1 Mbytes/s
	   2048 kbytes:   1926.7 usec/IO =   1038.1 Mbytes/s
	   4096 kbytes:   3806.6 usec/IO =   1050.8 Mbytes/s
	   8192 kbytes:   7432.6 usec/IO =   1076.3 Mbytes/s

Code:

root@freenas:~ # diskinfo -wS /dev/nvd0
/dev/nvd0
		4096			# sectorsize
		42999996416	 # mediasize in bytes (40G)
		10498046		# mediasize in sectors
		131072		  # stripesize
		0			   # stripeoffset
		INTEL SSDPEDMD400G4	 # Disk descr.
		CVFT7112001A400LGN	  # Disk ident.

Synchronous random writes:
		   4 kbytes:	 11.4 usec/IO =	342.0 Mbytes/s
		   8 kbytes:	 13.8 usec/IO =	566.7 Mbytes/s
		  16 kbytes:	 19.0 usec/IO =	821.3 Mbytes/s
		  32 kbytes:	 33.0 usec/IO =	945.8 Mbytes/s
		  64 kbytes:	 65.3 usec/IO =	957.3 Mbytes/s
		 128 kbytes:	131.2 usec/IO =	952.4 Mbytes/s
		 256 kbytes:	244.2 usec/IO =   1023.6 Mbytes/s
		 512 kbytes:	482.2 usec/IO =   1036.9 Mbytes/s
		1024 kbytes:	950.8 usec/IO =   1051.7 Mbytes/s
		2048 kbytes:   1927.8 usec/IO =   1037.4 Mbytes/s
		4096 kbytes:   3815.4 usec/IO =   1048.4 Mbytes/s
		8192 kbytes:   7662.8 usec/IO =   1044.0 Mbytes/s

You can see that it doubled the 4K speed and bring it to the range of a P4800X w/o tuning.

Here are the changes that I made, some will undoubtedly decrease CPU performance but FreeNAS are rarely CPU bounded anyway. Every BIOS is different so you may need to find yours but a lot of things are generic:

I should also note that the idle power increased 60% (from ~100W to ~170W, excluding HDDs), and fans start to run at full blast. so the trade-off may or may not be worth it to you.

I hope you find this interesting, let me know the result if you tried it on your own machine.

 

[HoneyBadger]

Did you record the settings that were used beforehand? I'd be interested in isolating the specific setting or settings that had the highest impact.

 

[Ender117]

Did you record the settings that were used beforehand? I'd be interested in isolating the specific setting or settings that had the highest impact.

I didn't write it down but if memory serves me well the settings I changed are: logical processor(HT) and everything below system profile expect for memory frequency.

Update, turning CPU power management to DBPM dropped 4K speed to 186Mbyte/s, and interestingly set system profile to performance per watts (DAPC) increase it back to 200

 

[Ender117]

Did you record the settings that were used beforehand? I'd be interested in isolating the specific setting or settings that had the highest impact.

After some tinkering I found that disable HT and let FreeNAS to manage CPU frequency (CPU power set to OS managed instead iDRAC managed) to have the biggest impact. With these settings I can still achieve a 300MB/s 4K speed in diskinfo -wS and a generally acceptable power usage (80W idle, 150W busy writing) and noise.