Hi,
I have TrueNAS in proxmox with 8 cores, 32GB of ram and pci passtrough of the SATA controller.
My issue is I have really fast read speed, but horrible write. I am running RAIDZ1.
TrueNAS version: TrueNAS-SCALE-22.02.4
Proxmox version: 7.3-3
Proxmox config:
My configuration:
Intel Xeon Silver 4210 CPU
96GB Samsung DDR4 2933 Mhz ECC Ram
4x SEAGATE HDD Mobile Barracuda25 Guardian (2.5/ 2TB/ SATA 6Gb/s/ rmp 5400) - (its not a real nas drive, but it will get upgraded soon), these are in RAIDZ1
Here is the result of write and read speed in TrueNAS shell:
Any advice or help is appreciated!
I have TrueNAS in proxmox with 8 cores, 32GB of ram and pci passtrough of the SATA controller.
My issue is I have really fast read speed, but horrible write. I am running RAIDZ1.
TrueNAS version: TrueNAS-SCALE-22.02.4
Proxmox version: 7.3-3
Proxmox config:
My configuration:
Intel Xeon Silver 4210 CPU
96GB Samsung DDR4 2933 Mhz ECC Ram
4x SEAGATE HDD Mobile Barracuda25 Guardian (2.5/ 2TB/ SATA 6Gb/s/ rmp 5400) - (its not a real nas drive, but it will get upgraded soon), these are in RAIDZ1
Here is the result of write and read speed in TrueNAS shell:
Code:
pool1: (groupid=0, jobs=1): err= 0: pid=30860: Sat Dec 3 10:15:41 2022 write: IOPS=27, BW=27.7MiB/s (29.0MB/s)(5120MiB/184910msec); 0 zone resets slat (usec): min=12, max=7850, avg=38.36, stdev=110.32 clat (usec): min=187, max=37853k, avg=36074.89, stdev=1035687.52 lat (usec): min=211, max=37853k, avg=36113.25, stdev=1035687.27 clat percentiles (usec): | 1.00th=[ 212], 5.00th=[ 223], 10.00th=[ 231], | 20.00th=[ 255], 30.00th=[ 289], 40.00th=[ 334], | 50.00th=[ 347], 60.00th=[ 359], 70.00th=[ 371], | 80.00th=[ 392], 90.00th=[ 453], 95.00th=[ 515], | 99.00th=[ 938], 99.50th=[ 3589], 99.90th=[12012487], | 99.95th=[17112761], 99.99th=[17112761] bw ( KiB/s): min=133120, max=1681408, per=100.00%, avg=895958.00, stdev=543132.74, samples=11 iops : min= 130, max= 1642, avg=874.91, stdev=530.46, samples=11 lat (usec) : 250=18.28%, 500=76.15%, 750=3.98%, 1000=0.59% lat (msec) : 2=0.29%, 4=0.33%, 10=0.16%, 20=0.02%, 2000=0.04% lat (msec) : >=2000=0.16% cpu : usr=0.12%, sys=0.03%, ctx=5924, majf=7, minf=49 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,5120,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=27.7MiB/s (29.0MB/s), 27.7MiB/s-27.7MiB/s (29.0MB/s-29.0MB/s), io=5120MiB (5369MB), run=184910-184910msec fio --name=pool1 --size=5g --rw=read --ioengine=posixaio --direct=1 --bs=1m pool1: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1) pool1: (groupid=0, jobs=1): err= 0: pid=56990: Sat Dec 3 10:26:53 2022 read: IOPS=2717, BW=2718MiB/s (2850MB/s)(5120MiB/1884msec) slat (nsec): min=1379, max=458656, avg=6651.71, stdev=6769.14 clat (usec): min=291, max=1858, avg=359.85, stdev=40.09 lat (usec): min=297, max=1864, avg=366.50, stdev=41.51 clat percentiles (usec): | 1.00th=[ 330], 5.00th=[ 334], 10.00th=[ 338], 20.00th=[ 343], | 30.00th=[ 347], 40.00th=[ 351], 50.00th=[ 351], 60.00th=[ 359], | 70.00th=[ 363], 80.00th=[ 371], 90.00th=[ 388], 95.00th=[ 404], | 99.00th=[ 461], 99.50th=[ 515], 99.90th=[ 873], 99.95th=[ 996], | 99.99th=[ 1860] bw ( MiB/s): min= 2718, max= 2732, per=100.00%, avg=2724.19, stdev= 6.98, samples=3 iops : min= 2718, max= 2732, avg=2724.00, stdev= 7.21, samples=3 lat (usec) : 500=99.43%, 750=0.43%, 1000=0.10% lat (msec) : 2=0.04% cpu : usr=3.51%, sys=3.13%, ctx=5352, majf=0, minf=47 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5120,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=2718MiB/s (2850MB/s), 2718MiB/s-2718MiB/s (2850MB/s-2850MB/s), io=5120MiB (5369MB), run=1884-1884msec
Any advice or help is appreciated!