What kind of read/write performance should I be getting?

[djdwosk97]

I feel like the performance I'm getting is lower than what it should be. I've done a few tests with different file sizes and they all show similar results. The most recent test I did was with a 200MB~ file and I got the following results when copying from one place to the other:
computer -> server : 5MB/s
server -> computer: 14MB/s
server drive 1 -> server drive 2: 3MB/s

I've ran smart tests on all the drives and everything looks good, and the only other thing running is a Plex media server plugin (nothing is streaming). All the reporting graphs are basically at idle -- the cpu usage is at 0%, all the drives are sitting around 0% (except when I try writing/reading during the file test). (Other graphs in reporting section: http://imgur.com/a/s8Qx2 )

Hardware:
Pentium G3250
MSI H81m-p33
8gb non-ECC ram
3x 4TB WD Reds (not in any RAID config)
2x 1TB (WD Green, Seagate barracuda in Raid 1)

 

[gpsguy]

What chipset does your NIC use. If Realtek, that's part of your problem. Get an Intel Pro/1000 CT.

With your mobo choice and lack of ECC RAM you might want to swap them out for better hardware. See the stickies for suggestions. With your drive setup, 16GB RAM might be a better starting point.

And we don't know about the rest of your network hardware, i.e. Ethernet switches, cabling, ...


Sent from my phone

 

[djdwosk97]

What chipset does your NIC use. If Realtek, that's part of your problem. Get an Intel Pro/1000 CT.

With your mobo choice and lack of ECC RAM you might want to swap them out for better hardware. See the stickies for suggestions. With your drive setup, 16GB RAM might be a better starting point.

And we don't know about the rest of your network hardware, i.e. Ethernet switches, cabling, ...


Sent from my phone

Asus AC68u. Both my computers are connected via wifi (out of necessity) -- one is an Asus PCE-56u and the other is whatever AC card is in my Mac. Neither should be limiting.

My motherboard uses a Realtek RTL8111G. At some point I'll probably be throwing my 4690k and Z87m Gaming into the server -- which uses a Killer E2205, would that be better? But, the realtek NIC aside, even the inter-server transfers are slower than they should be.

I don't think memory is the cause of the poor read/write performance though -- it's probably leading to the low hit rate (80%~), but I don't think it's the cause of my performance issues. Also, what kind of read/write performance should I be expecting?

 

[djdwosk97]

What would happen if I shut down the system and disconnected two drives and turned the system back on? Would there be any complications if I wanted to re-add those drives at some future point? and how would I go about re-adding those two drives (would I just have to re-add the volumes in the volume manager? -- the two drives are setup in ZFS Raid1).

 

[gpsguy]

Wifi is a large part of your performance issues coupled with the Realtek NIC. Do a forum search for wifi or Realtek for more information.

Before you remove any drives, please open the shell and do a zpool status

Post the results in code tags or take a picture and post it here. Formatting is very important


Sent from my phone

 

[djdwosk97]

Wifi is a large part of your performance issues coupled with the Realtek NIC. Do a forum search for wifi or Realtek for more information.

Before you remove any drives, please open the shell and do a zpool status

Post the results in code tags or take a picture and post it here. Formatting is very important


Sent from my phone

Again, since I'm also getting equally poor (if not worse) performance when doing a transfer completely within the server it's obvious the network isn't my main issue (assuming it's one at all).

http://imgur.com/a/AT27D

 

[gpsguy]

How are you copying files from server drive 1 to #2? The traffic might actually pass over your wireless connection via your computer and back again.

8GB is the bare minimum for FreeNAS. Given your striped volume + Plex, a minimum of 16GB would be recommended.



Sent from my phone

 

[JDCynical]

At some point I'll probably be throwing my 4690k and Z87m Gaming into the server -- which uses a Killer E2205, would that be better?

Um... no. Not in a million years more than likely, no.

https://forums.freenas.org/index.php?threads/killer-e2205-nic-help.21900/

I get the idea of reusing spare hardware, I do. But FreeNAS is long past the days of 'using old spare hardware' for it.

 

[gpsguy]

Since the mirrored drives are in a separate volume, you can safely detach the volume (follow the instructions in the manual) and remove it. If you have data on them that you don't want to lose, don't check the box that says "Mark disks as new". At some point in the future you could re-add them, by "auto importing the volume". If you mark the disks as new (destroying the data), you'd need to reconfigure your volume, like you did initially.

Your other 3x4TB disks are independent volumes with no redundancy. It's safer than striping the three disks together, in that, should you lose one disk, it doesn't affect the others. Most of our users use some form of RAIDz. With 4TB drives, RAIDz2 is recommended. For that, you'd need at least 4 disks.

What would happen if I shut down the system and disconnected two drives and turned the system back on? Would there be any complications if I wanted to re-add those drives at some future point? and how would I go about re-adding those two drives (would I just have to re-add the volumes in the volume manager? -- the two drives are setup in ZFS Raid1).

 

[djdwosk97]

How are you copying files from server drive 1 to #2? They might actually pass over your wireless connection via your computer and back again.

8GB is the bare minimum for FreeNAS. Given your striped volume + Plex, a minimum of 16GB would be recommended.

I transferred it from the command line.

But what kind of wired performance and what kind of inter-server performance should I be expecting?

And that's why the three 4tb drives are independent. I calculated the likelihood of a URE when rebuilding a RAID 5 array and decided it wasn't worthwhile and I didn't want to spend more money for a RAID 6 array, so I decided to just leave the drives independent rather than striping them even though it would be nice if FreeNAS had a JBOD Span option as well.

Um... no. Not in a million years more than likely, no.

https://forums.freenas.org/index.php?threads/killer-e2205-nic-help.21900/

I get the idea of reusing spare hardware, I do. But FreeNAS is long past the days of 'using old spare hardware' for it.

The Realtek NIC that's in the system now doesn't seem to be causing any problems.

 

[Steve Brown]

There is a lot of variables to pin point the root cause. I would start with a wired connection to see what that does. Also is it going from USB flash drive to the freenas? is it USB2.0? There are so many other questions that have to be answered first.

 

[djdwosk97]

Since the mirrored drives are in a separate volume, you can safely detach the volume (follow the instructions in the manual) and remove it. If you have data on them that you don't want to lose, don't check the box that says "Mark disks as new". At some point in the future you could re-add them, by "auto importing the volume". If you mark the disks as new (destroying the data), you'd need to reconfigure your volume, like you did initially.

Your other 3x4TB disks are independent volumes with no redundancy. It's safer than striping the three disks together, in that, should you lose one disk, it doesn't affect the others. Most of our users use some form of RAIDz. With 4TB drives, RAIDz2 is recommended. For that, you'd need at least 4 disks.

I restarted my server and on wired I'm now getting about 60MB/s when transferring to or from my server via AFP shares. So I don't know if that's normal or if I should still be expecting higher speeds or not.

 

[Steve Brown]

60 MB/s for a 1GB connection is definitely better. To fully saturate it would be more like 90 MB/s. Now I am guessing you would see a performance boost if you have more RAM or with an INTEL NIC.

 

[djdwosk97]

60 MB/s for a 1GB connection is definitely better. To fully saturate it would be more like 90 MB/s. Now I am guessing you would see a performance boost if you have more RAM or with an INTEL NIC.

Well, I guess I'll have to decide at some point soon whether or not I want to spend $300 to upgrade to 16gb of ECC or just throw in an additional 8gb stick for $30.

 

[Steve Brown]

on your reports what is it showing under "Memory"?

Here is mine with 32GB of ECC. I have 7 HDD of 2TB WD REDs in Z2. As you can see I have less than half available and the machine was recently restarted.

 

[djdwosk97]

on your reports what is it showing under "Memory"?

Here is mine with 32GB of ECC. I have 7 HDD of 2TB WD REDs in Z2. As you can see I have less than half available and the machine was recently restarted.
View attachment 9989

Doesn't FreeNAS never free memory -- so isn't the graph is kind of meaningless?

[img=[URL]http://i.imgur.com/u6LK5PN.jpg][/URL]

 

[gpsguy]

I don't believe your mobo is ECC compatible.

That's why I suggested you look at our recommendations.

Well, I guess I'll have to decide at some point soon whether or not I want to spend $300 to upgrade to 16gb of ECC or just throw in an additional 8gb stick for $30.

Sent from my phone

 

[djdwosk97]

I don't believe your mobo is ECC compatible.

That's why I suggested you look at our recommendations.

I know it's not, which is why upgrading to ECC would cost me about $300 vs. just grabbing another 8gb stick, which is about $30; and considering that the server is mainly just for plex, I can't really rationalize that kind of expenditure (especially since any bitrot will most likely not even be noticeable).

 

[SweetAndLow]

1. Grab another 8GB of memory
2. Grab a $30 intel network card
3. to test write speed run this on a dataset that has compression disabled.

Code:

dd if=/dev/zero of=./21GB.dat bs=128k count=163840

4. to test read speed run this on a dataset that has compression disabled and also has the 21GB file created in the previous test.

Code:

dd of=/dev/null if=./21GB.dat bs=128k count=163840

Don't expect anything magical since your hardware is very 'iffy' and not recommended for freenas. For best results stick to recommended server hardware.

 

[djdwosk97]

1. Grab another 8GB of memory
2. Grab a $30 intel network card
3. to test write speed run this on a dataset that has compression disabled.

Code:

dd if=/dev/zero of=./21GB.dat bs=128k count=163840

4. to test read speed run this on a dataset that has compression disabled and also has the 21GB file created in the previous test.

Code:

dd of=/dev/null if=./21GB.dat bs=128k count=163840

Don't expect anything magical since your hardware is very 'iffy' and not recommended for freenas. For best results stick to recommended server hardware.

1. I will probably pick up another 8gb stick at some point soon, I'm just going to think about upgrading to ECC for a bit longer first.
2. I'll be doing basically all of my transfers over wifi anyway, so there's no real point in getting an Intel NIC if my Realtek NIC isn't causing trouble. (Plus the only times I really write to the drive is when I'm adding more content)
3/4. When I copied a file from one drive to another after restarting the 200MB test file I was using was moved almost immediately -- so whatever the issue was seems to be fixed.

Well, like I said in my previous post, true server hardware would have ran me an additional $300~, and I can't really justify that when my main use is for media storage/plex server. I'd love to do a proper upgrade to a Supermicro X10s, 32gb ECC memory, 5 Seagate's 8tb archival drives in raidz2, and a Define R4/R5, but I don't really want to spend $1800.