I'm playing around with FreeNAS 9.3 and have found sync writes aren't as fast as I expected from my setup. First I used dd to test the max sequential write speed of my SLOG SSD:
This is slower than the drive's write spec (170 Mbyte/s), but close enough to the limit of my gigabit Ethernet that it'd be fine if my pool could write that fast. But when I add the SSD to my pool as an SLOG and do the same write to the pool, I get this:
I.e. about 40% slower. My RAM is 32GB, so my transaction group size should be big enough to accept that whole 256MB write into the SLOG, right? What could be causing this? Are SLOG writes non-sequential or multi-threaded or something that would make them run slower than a single-thread sequential write to the same device?
This is somewhat academic, as when I was searching for an answer to this, I discovered that my SLOG SSD doesn't have a super-capacitor, so (as I understand it) sync writes won't give me any extra protection in this setup anyway - but I am still curious to understand.
Hardware:
Dell PE2970
2 x AMD Opteron 2389 (2.90Ghz quad-core)
32GB ECC 1067MHz
6 x SATA 7200 RPM 320GB - RAIDZ2
1 x Intel X25-E 32GB SLC - SLOG
[root@freenas] ~# dd if=/dev/zero of=/dev/ada0 bs=128k count=2000
2000+0 records in
2000+0 records out
262144000 bytes transferred in 2.431943 secs (107791992 bytes/sec)
2000+0 records in
2000+0 records out
262144000 bytes transferred in 2.431943 secs (107791992 bytes/sec)
This is slower than the drive's write spec (170 Mbyte/s), but close enough to the limit of my gigabit Ethernet that it'd be fine if my pool could write that fast. But when I add the SSD to my pool as an SLOG and do the same write to the pool, I get this:
[root@freenas] ~# zpool add store log ada0
[root@freenas] ~# zfs set sync=always store
[root@freenas] ~# dd if=/dev/zero of=/mnt/store/tempfile bs=128k count=2000
2000+0 records in
2000+0 records out
262144000 bytes transferred in 4.080628 secs (64241090 bytes/sec)
[root@freenas] ~# zfs set sync=always store
[root@freenas] ~# dd if=/dev/zero of=/mnt/store/tempfile bs=128k count=2000
2000+0 records in
2000+0 records out
262144000 bytes transferred in 4.080628 secs (64241090 bytes/sec)
I.e. about 40% slower. My RAM is 32GB, so my transaction group size should be big enough to accept that whole 256MB write into the SLOG, right? What could be causing this? Are SLOG writes non-sequential or multi-threaded or something that would make them run slower than a single-thread sequential write to the same device?
This is somewhat academic, as when I was searching for an answer to this, I discovered that my SLOG SSD doesn't have a super-capacitor, so (as I understand it) sync writes won't give me any extra protection in this setup anyway - but I am still curious to understand.
Hardware:
Dell PE2970
2 x AMD Opteron 2389 (2.90Ghz quad-core)
32GB ECC 1067MHz
6 x SATA 7200 RPM 320GB - RAIDZ2
1 x Intel X25-E 32GB SLC - SLOG