slower than expected write speeds

[pious_greek]

I have my pool configured as raidz2 with compression off for testing. When performing a write test through SSH, i'm getting a write speed of about 192 megabytes per second. I was expecting about twice that. When i perform a read of that data, i'm getting 485 megabytes per second which is consistent with my expectations. (expectations were based on https://calomel.org/zfs_raid_speed_capacity.html which indicated: w=429MB/s , rw=71MB/s , r=488MB/s )

Code:

 sudo dd if=/dev/zero of=/mnt/pool/dataset/testfile bs=1024 count=1000000
1024000000 bytes transferred in 5.322653 secs (192385256 bytes/sec)

while running the test, the CPU does not appear to be taxed to any great extent, so how would i diagnose where the bottleneck is? i dont think my hardware selection is inadequate. I've run burn in tests on the CPU, RAM and on the hard drives with no alarming results, although i did experience one bank of drives running the testing at a slower rate than the other bank, which i was advised was not unusual.

https://forums.freenas.org/index.ph...urn-in-tests-bend-in-cable.59770/#post-424116


my hardware selection is in my signature below

 

[pious_greek]

following jgreco's advice here, https://forums.freenas.org/index.php?threads/new-to-freenas-slow-write-speeds.61316/#post-436148 i increased the block size to 16384, which resulted in write speeds at almost a gigabyte per second; which does not seem realistic.

Code:

sudo dd if=/dev/zero of=/mnt/pool/dataset/testfile bs=16384 count=500000
8192000000 bytes transferred in 8.376937 secs (977923037 bytes/sec)

 

[tvsjr]

Your output file is "/testfile", which would be in the root of the boot drive. All of your storage volumes should be under /mnt/.

 

[pious_greek]

Thanks tvsjr. The output file path I initially indicated was incorrect. I wanted to redact volume and dataset names not worth sharing. Anyhow, the test file was being created in a dataset on the storage volume, with a path comparable to: of=/mnt/pool/dataset/testfile

ls showed the file was successfully created in the dataset, and the FreeNAS gui's storage tab reflected that data had been created.

 

[wblock]

Using dd, never use a bs of less than 64K. It just results in more overhead. Also, note that dd understands units like "k" and "m", so bs=64k or bs=1m work.

That said, writing only zeros, even with ZFS compression turned off, is probably going to get compressed in that I/O stream somewhere and show unrealistic numbers.