Hi,
I am looking to understand what should be expected speeds of transfer inside the dataset and between two datasets (on two difeerent pools). It looks like my speeds are quite low and can't find the reason why.
Specs:
Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
Supermicro x10SRi-F
3x 4TB WD Red (...EFAX ones, SMR's)
2x 4TB Segate Ironwolf
Seasonic GC-650
I've just build the TrueNAS box and I am playing with it to understand it's behaviour before I will put there any data for good. I have created the RAIDZ2 Pool with all 5 drives.
I copied 200GB file from my PC to TrueNAS through cable (stable 110 MB/s). Then I took this file and created copy of it inside the same folder (the same dataset and pool). I had stable copy speed around 130-140 MB/s. This was using Windows to copy. I repeated this excercise with 'cp' command inside the Shell and copy speed were the same. Those are pretty crappy speeds and I thought it may be these SMR 4TB WD Reds that are messing with the whole pool.
To understand if that's the case, I destroyed the 5 disc pool and created three smaller pools. 2x WD Reds pool (mirror), 2x Segate pool (mirror), 1x WD Red pool.
I did simmilar excercise. I copied 200GB file inside each pool and also from one mirror to second mirror. I connected TrueNAS to monitor and used Shell and 'cp' command. Speeds are as below:
- Inside WD Mirror - 87 MB/s
- Inside Segate Mirror - 71 MB/s
- Inside WD Single - 81 MB/s
- WD mirror to Segate Mirror - 153 MB/s
- Segate Mirror to WD Mirror - 138 MB/s
- Segate Mirror to WD Single - 138 MB/s
I'd like to think that this excercise excludes any issues with Windows because I used Shell and exclude any issues with heterogeneous pool as I tested mirror pools as well.
Please see below output from 'dd' command. I dont really understand how this command works so let me know if better to use different chunks or amounts.
What else I can do to check where is the issue. I would expect that 'inside' the pool I would get sth around 800-1500MB/s. Am I being naive?
Thanks,
Pawel
I am looking to understand what should be expected speeds of transfer inside the dataset and between two datasets (on two difeerent pools). It looks like my speeds are quite low and can't find the reason why.
Specs:
Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
Supermicro x10SRi-F
3x 4TB WD Red (...EFAX ones, SMR's)
2x 4TB Segate Ironwolf
Seasonic GC-650
I've just build the TrueNAS box and I am playing with it to understand it's behaviour before I will put there any data for good. I have created the RAIDZ2 Pool with all 5 drives.
I copied 200GB file from my PC to TrueNAS through cable (stable 110 MB/s). Then I took this file and created copy of it inside the same folder (the same dataset and pool). I had stable copy speed around 130-140 MB/s. This was using Windows to copy. I repeated this excercise with 'cp' command inside the Shell and copy speed were the same. Those are pretty crappy speeds and I thought it may be these SMR 4TB WD Reds that are messing with the whole pool.
To understand if that's the case, I destroyed the 5 disc pool and created three smaller pools. 2x WD Reds pool (mirror), 2x Segate pool (mirror), 1x WD Red pool.
I did simmilar excercise. I copied 200GB file inside each pool and also from one mirror to second mirror. I connected TrueNAS to monitor and used Shell and 'cp' command. Speeds are as below:
- Inside WD Mirror - 87 MB/s
- Inside Segate Mirror - 71 MB/s
- Inside WD Single - 81 MB/s
- WD mirror to Segate Mirror - 153 MB/s
- Segate Mirror to WD Mirror - 138 MB/s
- Segate Mirror to WD Single - 138 MB/s
I'd like to think that this excercise excludes any issues with Windows because I used Shell and exclude any issues with heterogeneous pool as I tested mirror pools as well.
Please see below output from 'dd' command. I dont really understand how this command works so let me know if better to use different chunks or amounts.
root@truenas[~]# dd if=/dev/zero of=/mnt/Segate/Segate/ddfile1 bs=2048k count=10000
10000+0 records in
10000+0 records out
20971520000 bytes transferred in 4.239338 secs (4946886038 bytes/sec)
root@truenas[~]# dd if=/dev/zero of=/mnt/WDmirror/WDmirror/ddfile1 bs=2048k count=10000
10000+0 records in
10000+0 records out
20971520000 bytes transferred in 4.149101 secs (5054473098 bytes/sec)
root@truenas[~]# dd if=/dev/zero of=/mnt/WDsingle/WDsingle/ddfile1 bs=2048k count=10000
10000+0 records in
10000+0 records out
20971520000 bytes transferred in 4.051045 secs (5176817903 bytes/sec)
What else I can do to check where is the issue. I would expect that 'inside' the pool I would get sth around 800-1500MB/s. Am I being naive?
Thanks,
Pawel