Hi All,
I currenty bought a Truenas Mini XL+ unit for video editing in 2K resolution purpose
8 cores 2.2GHz Intel Atom CPU
64G ECC memory
And Added:
8 x WD 14TB RED NAS Pro disks
1 Pool with Stripe 2 vDev, each vDev consists of 4 disks in RAID-Z1, Lz4 compression, Record size 128K
1 x 512G SSD as cache, 1 x 512G SSD as LOG
Switch: Netgear XS708T 8 ports 10G switch
Test Client Side:
MSI Stealth laptop. windows 11 pro
11th Gen Intel(R) Core(TM) i7-11375H @ 3.30GHz 3.30
64G Memory
ATTO Thunderbolt 3 to Dual 10G adapter
All ports, cards and switch set to MTU 9000 jumbo frame
iperf test speed result:
Connecting to host 192.168.1.140, port 5201
[ 4] local 192.168.1.151 port 53533 connected to 192.168.1.140 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.08 GBytes 9.32 Gbits/sec
[ 4] 1.00-2.00 sec 1.08 GBytes 9.29 Gbits/sec
[ 4] 2.00-3.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 3.00-4.00 sec 1.09 GBytes 9.37 Gbits/sec
[ 4] 4.00-5.00 sec 1.08 GBytes 9.24 Gbits/sec
[ 4] 5.00-6.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 6.00-7.00 sec 1.09 GBytes 9.36 Gbits/sec
[ 4] 7.00-8.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 8.00-9.00 sec 1.08 GBytes 9.32 Gbits/sec
[ 4] 9.00-10.00 sec 1.07 GBytes 9.23 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 10.9 GBytes 9.33 Gbits/sec sender
[ 4] 0.00-10.00 sec 10.9 GBytes 9.33 Gbits/sec receiver
Data looks goo to me
with DD test:
Write
dd if=/dev/zero of=/mnt/POOL1/sambademo/speed/test.dd bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 25.610992 secs (1637696832 bytes/sec)
...................................................................................................................................
Read
dd of=/dev/zero if=/mnt/POOL1/sambademo/speed/test.dd bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 14.354601 secs (2921923151 bytes/sec)
.....................................................................................................................................
Also seems to be OK
With real application, I only got 8xxMB/s in write and 4xxMB/s in read with HD/2K video streams.
I also use frametest command in client machine and here is the result:
WRITE
frametest -w 2k -n 1000 -t 4 x:/speed/
Test parameters: -w12512 -n1000 -t4
Test duration: 15 secs
Frames transferred: 990 (12096.563 MB)
Fastest frame: 15.683 ms (779.09 MB/s)
Slowest frame: 145.292 ms (84.10 MB/s)
Averaged details:
Open I/O Frame Data rate Frame rate
Last 1s: 8.205 ms 31.42 ms 14.73 ms 829.63 MB/s 67.9 fps
5s: 7.392 ms 31.76 ms 14.78 ms 826.43 MB/s 67.6 fps
30s: 7.338 ms 33.07 ms 15.35 ms 796.16 MB/s 65.2 fps
Overall: 7.338 ms 33.07 ms 15.35 ms 796.16 MB/s 65.2 fps
Histogram of frame completion times:
50% |
|
|
| *
| *
| *
| ***
| ***
| * *****
| *****************
+|----|-----|----|----|-----|----|----|-----|----|----|-----|----|
ms <0.1 .2 .5 1 2 5 10 20 50 100 200 500 >1s
Overall frame rate .... 64.97 fps (832435187 bytes/s)
Average file time ...... 60.103 ms
Shortest file time ..... 15.683 ms
Longest file time ...... 145.292 ms
Average create time .... 7.309 ms
Shortest create time ... 1.324 ms
Longest create time .... 69.035 ms
Average write time ..... 33.2 ms
Shortest write time .... 13.0 ms
Longest write time ..... 116.1 ms
Average close time .... 19.625 ms
Shortest close time ... 0.520 ms
Longest close time .... 91.696 ms
Test parameters: -w12512 -n1000 -t4
Test duration: 15 secs
Test parameters: -r -z12512 -n1000 -t4
Test duration: 31 secs
Frames transferred: 972 (11876.625 MB)
Fastest frame: 37.853 ms (322.80 MB/s)
Slowest frame: 216.701 ms (56.39 MB/s)
READ
frametest -r 2k -n 1000 -t 4 x:/speed/
Averaged details:
Open I/O Frame Data rate Frame rate
Last 1s: 23.739 ms 90.88 ms 30.73 ms 397.67 MB/s 32.5 fps
5s: 20.864 ms 91.90 ms 31.71 ms 385.37 MB/s 31.5 fps
30s: 20.073 ms 93.30 ms 32.23 ms 379.13 MB/s 31.0 fps
Overall: 20.007 ms 93.90 ms 32.40 ms 377.12 MB/s 30.9 fps
Histogram of frame completion times:
50% |
|
|
|
|
| *
| ***
| ****
| ******
| * **********
+|----|-----|----|----|-----|----|----|-----|----|----|-----|----|
ms <0.1 .2 .5 1 2 5 10 20 50 100 200 500 >1s
Overall frame rate .... 30.91 fps (395990652 bytes/s)
Average file time ...... 113.839 ms
Shortest file time ..... 37.853 ms
Longest file time ...... 216.701 ms
Average open time ...... 20.069 ms
Shortest open time ..... 7.049 ms
Longest open time ...... 91.108 ms
Average read time ...... 93.7 ms
Shortest read time ..... 29.4 ms
Longest read time ...... 185.7 ms
Average close time .... 0.032 ms
Shortest close time ... 0.015 ms
Longest close time .... 0.144 ms
I would like to push the read be faster , may be up to 600-800MB/s, anyone have experienceand suggestion in how to make the read speed be faster?
Thanks,
Curly
I currenty bought a Truenas Mini XL+ unit for video editing in 2K resolution purpose
8 cores 2.2GHz Intel Atom CPU
64G ECC memory
And Added:
8 x WD 14TB RED NAS Pro disks
1 Pool with Stripe 2 vDev, each vDev consists of 4 disks in RAID-Z1, Lz4 compression, Record size 128K
1 x 512G SSD as cache, 1 x 512G SSD as LOG
Switch: Netgear XS708T 8 ports 10G switch
Test Client Side:
MSI Stealth laptop. windows 11 pro
11th Gen Intel(R) Core(TM) i7-11375H @ 3.30GHz 3.30
64G Memory
ATTO Thunderbolt 3 to Dual 10G adapter
All ports, cards and switch set to MTU 9000 jumbo frame
iperf test speed result:
Connecting to host 192.168.1.140, port 5201
[ 4] local 192.168.1.151 port 53533 connected to 192.168.1.140 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.08 GBytes 9.32 Gbits/sec
[ 4] 1.00-2.00 sec 1.08 GBytes 9.29 Gbits/sec
[ 4] 2.00-3.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 3.00-4.00 sec 1.09 GBytes 9.37 Gbits/sec
[ 4] 4.00-5.00 sec 1.08 GBytes 9.24 Gbits/sec
[ 4] 5.00-6.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 6.00-7.00 sec 1.09 GBytes 9.36 Gbits/sec
[ 4] 7.00-8.00 sec 1.09 GBytes 9.38 Gbits/sec
[ 4] 8.00-9.00 sec 1.08 GBytes 9.32 Gbits/sec
[ 4] 9.00-10.00 sec 1.07 GBytes 9.23 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 10.9 GBytes 9.33 Gbits/sec sender
[ 4] 0.00-10.00 sec 10.9 GBytes 9.33 Gbits/sec receiver
Data looks goo to me
with DD test:
Write
dd if=/dev/zero of=/mnt/POOL1/sambademo/speed/test.dd bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 25.610992 secs (1637696832 bytes/sec)
...................................................................................................................................
Read
dd of=/dev/zero if=/mnt/POOL1/sambademo/speed/test.dd bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 14.354601 secs (2921923151 bytes/sec)
.....................................................................................................................................
Also seems to be OK
With real application, I only got 8xxMB/s in write and 4xxMB/s in read with HD/2K video streams.
I also use frametest command in client machine and here is the result:
WRITE
frametest -w 2k -n 1000 -t 4 x:/speed/
Test parameters: -w12512 -n1000 -t4
Test duration: 15 secs
Frames transferred: 990 (12096.563 MB)
Fastest frame: 15.683 ms (779.09 MB/s)
Slowest frame: 145.292 ms (84.10 MB/s)
Averaged details:
Open I/O Frame Data rate Frame rate
Last 1s: 8.205 ms 31.42 ms 14.73 ms 829.63 MB/s 67.9 fps
5s: 7.392 ms 31.76 ms 14.78 ms 826.43 MB/s 67.6 fps
30s: 7.338 ms 33.07 ms 15.35 ms 796.16 MB/s 65.2 fps
Overall: 7.338 ms 33.07 ms 15.35 ms 796.16 MB/s 65.2 fps
Histogram of frame completion times:
50% |
|
|
| *
| *
| *
| ***
| ***
| * *****
| *****************
+|----|-----|----|----|-----|----|----|-----|----|----|-----|----|
ms <0.1 .2 .5 1 2 5 10 20 50 100 200 500 >1s
Overall frame rate .... 64.97 fps (832435187 bytes/s)
Average file time ...... 60.103 ms
Shortest file time ..... 15.683 ms
Longest file time ...... 145.292 ms
Average create time .... 7.309 ms
Shortest create time ... 1.324 ms
Longest create time .... 69.035 ms
Average write time ..... 33.2 ms
Shortest write time .... 13.0 ms
Longest write time ..... 116.1 ms
Average close time .... 19.625 ms
Shortest close time ... 0.520 ms
Longest close time .... 91.696 ms
Test parameters: -w12512 -n1000 -t4
Test duration: 15 secs
Test parameters: -r -z12512 -n1000 -t4
Test duration: 31 secs
Frames transferred: 972 (11876.625 MB)
Fastest frame: 37.853 ms (322.80 MB/s)
Slowest frame: 216.701 ms (56.39 MB/s)
READ
frametest -r 2k -n 1000 -t 4 x:/speed/
Averaged details:
Open I/O Frame Data rate Frame rate
Last 1s: 23.739 ms 90.88 ms 30.73 ms 397.67 MB/s 32.5 fps
5s: 20.864 ms 91.90 ms 31.71 ms 385.37 MB/s 31.5 fps
30s: 20.073 ms 93.30 ms 32.23 ms 379.13 MB/s 31.0 fps
Overall: 20.007 ms 93.90 ms 32.40 ms 377.12 MB/s 30.9 fps
Histogram of frame completion times:
50% |
|
|
|
|
| *
| ***
| ****
| ******
| * **********
+|----|-----|----|----|-----|----|----|-----|----|----|-----|----|
ms <0.1 .2 .5 1 2 5 10 20 50 100 200 500 >1s
Overall frame rate .... 30.91 fps (395990652 bytes/s)
Average file time ...... 113.839 ms
Shortest file time ..... 37.853 ms
Longest file time ...... 216.701 ms
Average open time ...... 20.069 ms
Shortest open time ..... 7.049 ms
Longest open time ...... 91.108 ms
Average read time ...... 93.7 ms
Shortest read time ..... 29.4 ms
Longest read time ...... 185.7 ms
Average close time .... 0.032 ms
Shortest close time ... 0.015 ms
Longest close time .... 0.144 ms
I would like to push the read be faster , may be up to 600-800MB/s, anyone have experienceand suggestion in how to make the read speed be faster?
Thanks,
Curly