I am setting up a new system and I am seeing horrendous performance from the pool I create.
Specs:
I have tried RaidZ2, Mirror of 2 drives, Mirror / Stripe 3x2, and single drive.
All produce within 1MB/s of that speed.
When I ran the badblocks burn-in it only took 3 hours to read / write the drive 4 times or ~ 1.2 GB / sec per drive.
nvmecontrol gives:
I know I have something configured / set wrong but I have never seen that level of degradation in performance.
Now that I think about it, that is such a small file (1GB) for the dd test it should be cache / RAM speed and badblocks / nvmecontrol are likely direct IO.
I ran a memory test for 24 hours with no errors.
Other odd notice. When I run nvmecontrol it never change the cpu utilization in top, but dd goes to 100% as did rsync (90%) copying the file locally.
I have much less powerfully systems running much faster than this and see NVMe systems in the forums topping GB/s speeds.
Thanks for all the help and work on a great product.
Jimmy
Specs:
- FreeNAS-9.10.2-U1 (86c7ef5)
- 2xIntel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
- 128GB RAM
- SuperMicro X10DRU-i+
- 6 X Intel P3700 NVMe
Code:
[root@freenas] /mnt/data# dd if=/dev/zero of=/mnt/data/bifgile bs=1024 count=1024000 1024000+0 records in 1024000+0 records out 1048576000 bytes transferred in 6.095852 secs (172014671 bytes/sec)
I have tried RaidZ2, Mirror of 2 drives, Mirror / Stripe 3x2, and single drive.
All produce within 1MB/s of that speed.
When I ran the badblocks burn-in it only took 3 hours to read / write the drive 4 times or ~ 1.2 GB / sec per drive.
nvmecontrol gives:
Code:
[root@freenas] /mnt/data# nvmecontrol perftest -n 32 -o write -s 512 -t 30 nvme3ns1 Threads: 32 Size: 512 WRITE Time: 30 IO/s: 13636 MB/s: 6
I know I have something configured / set wrong but I have never seen that level of degradation in performance.
Now that I think about it, that is such a small file (1GB) for the dd test it should be cache / RAM speed and badblocks / nvmecontrol are likely direct IO.
I ran a memory test for 24 hours with no errors.
Other odd notice. When I run nvmecontrol it never change the cpu utilization in top, but dd goes to 100% as did rsync (90%) copying the file locally.
Code:
[root@freenas] /mnt/data# rsync -vvaiP bifgile bifgile1 sending incremental file list delta-transmission disabled for local transfer or --whole-file >f+++++++++ bifgile 10,485,760,000 100% 183.71MB/s 0:00:54 (xfr#1, to-chk=0/1) total: matches=0 hash_hits=0 false_alarms=0 data=10485760000 sent 10,488,320,087 bytes received 102 bytes 188,978,742.14 bytes/sec total size is 10,485,760,000 speedup is 1.00
I have much less powerfully systems running much faster than this and see NVMe systems in the forums topping GB/s speeds.
Thanks for all the help and work on a great product.
Jimmy