Processor: AMD 3700x
Memory: 128GB
Disks: 20 disks, 10vDevs (Mirrors), all SSD's - Flash
TrueNAS-SCALE-Bluefin
no log or cache enabled on the pool.
Below is the fio. Is there a better way to test?
root@tn[/mnt/Tank]# fio --filename=test --direct=1 --rw=randrw --randrepeat=0 --rwmixread=100 --iodepth=128 --numjobs=12 --runtime=60 --group_reporting --name=4ktest --ioengine=psync --size=4G --bs=4k
Memory: 128GB
Disks: 20 disks, 10vDevs (Mirrors), all SSD's - Flash
TrueNAS-SCALE-Bluefin
no log or cache enabled on the pool.
Below is the fio. Is there a better way to test?
root@tn[/mnt/Tank]# fio --filename=test --direct=1 --rw=randrw --randrepeat=0 --rwmixread=100 --iodepth=128 --numjobs=12 --runtime=60 --group_reporting --name=4ktest --ioengine=psync --size=4G --bs=4k
Code:
4ktest: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=128 ... fio-3.25 Starting 16 processes 4ktest: Laying out IO file (1 file / 4096MiB) Jobs: 16 (f=16): [r(16)][100.0%][r=613MiB/s][r=157k IOPS][eta 00m:00s] 4ktest: (groupid=0, jobs=16): err= 0: pid=3476836: Sun Dec 25 17:05:48 2022 read: IOPS=157k, BW=612MiB/s (641MB/s)(35.8GiB/60001msec) clat (usec): min=2, max=44261, avg=98.45, stdev=344.53 lat (usec): min=2, max=44261, avg=98.64, stdev=344.64 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 18], | 30.00th=[ 23], 40.00th=[ 31], 50.00th=[ 48], 60.00th=[ 122], | 70.00th=[ 151], 80.00th=[ 172], 90.00th=[ 188], 95.00th=[ 202], | 99.00th=[ 247], 99.50th=[ 343], 99.90th=[ 2114], 99.95th=[ 7373], | 99.99th=[16188] bw ( KiB/s): min=467157, max=838431, per=100.00%, avg=626911.93, stdev=3761.33, samples=1904 iops : min=116789, max=209606, avg=156724.97, stdev=940.33, samples=1904 lat (usec) : 4=0.01%, 10=3.35%, 20=21.29%, 50=26.00%, 100=4.57% lat (usec) : 250=43.82%, 500=0.69%, 750=0.08%, 1000=0.03% lat (msec) : 2=0.05%, 4=0.03%, 10=0.04%, 20=0.03%, 50=0.01% cpu : usr=4.80%, sys=83.94%, ctx=153163, majf=0, minf=206 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=9393058,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=128 Run status group 0 (all jobs): READ: bw=612MiB/s (641MB/s), 612MiB/s-612MiB/s (641MB/s-641MB/s), io=35.8GiB (38.5GB), run=60001-60001msec