Writing speed depends on install location

Status
Not open for further replies.

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
Hey FreeNAS!

Yesterday I installed FreeNAS 11 on two thumb drives attached to a server that handles 12 sata disks behind a storage controller configured there as a raid0 array and in FreeNAS itself configured as an raidZ3.

Somehow the writing speed that it produces was around 40 mb/s, the same writing speed that the USB2 produces. After which I installed FreeNAS again on a mini-hdd, which produces a writing speed of around 100mb/s.

In short my question is: Why is FreeNAS it's writing speed changed as soon as the used medium on which FreeNAS is installed is changed.
 
Last edited by a moderator:

IceBoosteR

Guru
Joined
Sep 27, 2016
Messages
503
In a nutshell: Just read a little bit around in the forum. Present the disk as they are, physical, not logical or virtual. Use flashed SAS-HBAs in IT mode to do this.
 

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
In a nutshell: Just read a little bit around in the forum. Present the disk as they are, physical, not logical or virtual. Use flashed SAS-HBAs in IT mode to do this.
Ah I think I understand. I shall look into the subject some more, thanks for giving me a push in the right direction ^^
 

IceBoosteR

Guru
Joined
Sep 27, 2016
Messages
503
No problem mate, afterwards, if you have a straight overview of this, we can help you more on this.
To go with a starter, have a look in the resource section.
https://forums.freenas.org/index.php?resources/
And normally the installation media should not slower your disk speed, this is independant from it.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
And normally the installation media should not slower your disk speed, this is independant from it.
If the system dataset is stored on the boot pool and they are testing with samba, some samba components would be read/written to the boot pool. I doubt it would have that effect but it's worth mentioning.
The system dataset stores debugging core files and Samba4 metadata such as the user/group cache and share level permissions.
FreeNAS Doc - System Dataset
 

IceBoosteR

Guru
Joined
Sep 27, 2016
Messages
503
If the system dataset is stored on the boot pool and they are testing with samba, some samba components would be read/written to the boot pool. I doubt it would have that effect but it's worth mentioning.
FreeNAS Doc - System Dataset
Ok yeah agree to that. Normally my thumb drives are havin writes/read at about 500k/s so nothing really heavy.
Good that you mentioned that.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
In short my question is: Why is FreeNAS it's writing speed changed as soon as the used medium on which FreeNAS is installed is changed.

How are you measuring this?
 

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
How are you measuring this?
Two ways:
  • Through bash we run dd and write a file in the pool it's folder (locally on the FreeNAS server)
  • Through a network share, copying a large file over the network (gigabit connection from a Windows computer to the FreeNAS server).
Pardon my late reply.
 
Last edited:

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
If the system dataset is stored on the boot pool and they are testing with samba, some samba components would be read/written to the boot pool. I doubt it would have that effect but it's worth mentioning.
FreeNAS Doc - System Dataset

Although I believe (could be wrong) Samba is also responsible for the Windows share that we used for testing, we are also testing it locally on the FreeNAS itself.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
can you provide the commands used for testing?
 

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
can you provide the commands used for testing?
FreeNAS: dd if=/dev/zero of=/mnt/mainstorage bs=1024 count=1000000

Windows desktop: drag and drop 10gig file to a network share of /mnt/mainstorage.

Result: ~ 30-40 mb/s.

Edit
I have experimented with different block sizes, although most of them gave the same results, one in particular was ruling out that the speed was being limited by the thumb drives. Block size 4096 had a speed of around 200mb/s. This could indicate that perhaps FreeNAS is getting wrong values by the raid controller
 
Last edited:

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
When talking about block size do you mean 4096k or just 4096? Also when you say 200mb/s do you mean 200MB/s?
raid controller
Thats most of your issue. When you setup the "RAID0" drives did you disable write cache and read ahead?

Try your DD with a block size of at least 512k check you raid controller settings and report back. Please be sure to use correct units
 

Hitmoky

Cadet
Joined
Apr 24, 2018
Messages
7
When talking about block size do you mean 4096k or just 4096? Also when you say 200mb/s do you mean 200MB/s?

Thats most of your issue. When you setup the "RAID0" drives did you disable write cache and read ahead?

Try your DD with a block size of at least 512k check you raid controller settings and report back. Please be sure to use correct units

The block size notation is always in bytes so I did not include it, pardon. The speed of dd is in megabytes.
All the drives are configured as raid0, and write cache and read ahead has been disabled so that FreeNAS won't get all confused.

I just checked with DD and a block size of 512kb, which gave a speed of ~ 1.5 / 2 GB/s. I guess we can conclude from that that the raid controller is making things difficult for FreeNAS.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
I just checked with DD and a block size of 512kb, which gave a speed of ~ 1.5 / 2 GB/s. I guess we can conclude from that that the raid controller is making things difficult for FreeNAS.
Performance wise, this sounds normal. With DD each IO operation is set of syscalls to when writing 10GB at 1024 byte chunks takes a lot more work than working with 512 KB chunks. I (from here) looks like your storage is performing adequately.

Next we need to test the network performance. Take a look at using iperf to test network throughput.

Once we know the storage and network are adequate, we can look into why CIFS is so slow.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Also, we will need the exact specs of your FreeNAS system including model numbers for items like the CPU, Motherboard (unless Dell or similar OEM), HBA (in your case the RAID card), and drives.
 
Status
Not open for further replies.
Top