FreeNAS transfer speed drops after 30 minutes of uptime

[Pietroos]

Anyway do you think I can do something about the problem? I don't have other disks to test

 

[HoneyBadger]

The first i used is an old Kingston with 32GB of space, and the other one is a Samsung 860 QVO 1TB

I'm not surprised that the old Kingston will choke under heavy load, but a 1TB QVO should be able to handle those writes for much longer before buckling. Have you tried running iperf to check the raw network bandwidth? (Do this and your other performance testing wired - wireless is inherently vulnerable to interference and inconsistent results.)

Anyway do you think I can do something about the problem? I don't have other disks to test

If the problem does come down to your vdevs, then there really isn't a bandaid over "SMR has poor sustained performance" - it's the nature of the drives. Let's rule everything else out before we go replacing any hardware though. As I mentioned, the QVO should at least be able to sustain gigabit-speed writes for longer than a few seconds.

 

[Pietroos]

a 1TB QVO should be able to handle those writes for much longer before buckling.

I think is not a issue of the drives, but a software issue or a misconfiguration

Have you tried running iperf to check the raw network bandwidth?

I haven't tried but I'm 100% sure that isn't a network issue, when the speeds are normal a transfer saturates the server link (Which is a 1Gbit)

 

[Pietroos]

Hi, I have an update, I still have this problem, i tried to use 16GB of RAM and nothing, i also tried to change disks to others i have lying around and nothing, also I tried to change the switch and the network cables but nothing.

Given all the tests I've done, I think this is a software problem. Do you have any advice?

 

[MikeyG]

@Pietroos Have you done any testing directly on the pool (the one with the 1TB SSD) or has it all been via the network? Sorry if I missed it earlier.

 

[Pietroos]

@Pietroos Have you done any testing directly on the pool (the one with the 1TB SSD) or has it all been via the network?

I have done the tests only via network, but as I said in previous posts, I don't think is a network issue. I have also done some Iperf tests and and all the results saturate the bandwidth available

 

[Samuel Tai]

Have you tried making a pool with just your 860 SSD, and then running transfers to/from that, to eliminate the shingled drives as a factor?

 

[MikeyG]

I have done the tests only via network, but as I said in previous posts, I don't think is a network issue. I have also done some Iperf tests and and all the results saturate the bandwidth available

If it's not the network, doing some local testing on your pool would let you verify that your disks are performing as expected.

 

[Pietroos]

Have you tried making a pool with just your 860 SSD, and then running transfers to/from that, to eliminate the shingled drives as a factor?

Yes, i've tried but nothing

 

[Pietroos]

If it's not the network, doing some local testing on your pool would let you verify that your disks are performing as expected.

How I can do a local test? Anyway I don't think the problem are the disks, all disks I tried are pretty new and can support more than 100MB/s of writing speed

 

[Samuel Tai]

all disks I tried are pretty new and can support more than 100MB/s of writing speed

Unfortunately, I think what's going on is your disks actually can't sustain 100MB/s. You have so little RAM, you're using swap on your drives at the same time as the transfer. When your shingled drives start remapping sectors, your write performance for both swap and data transfers sinks to almost nothing.

You'll need to:

1. Add RAM
2. Switch to drives that aren't SMR.

 

[Pietroos]

Unfortunately, I think what's going on is your disks actually can't sustain 100MB/s. You have so little RAM, you're using swap on your drives at the same time as the transfer. When your shingled drives start remapping sectors, your write performance for both swap and data transfers sinks to almost nothing.

You'll need to:

1. Add RAM
2. Switch to drives that aren't SMR.

I've tried all sorts of disks, SSD and HDD, even at the same time, ALL the disks are having this issue (If i create two pools, one with an SSD and one with an HDD the speeds initially are good, but after some times i DON'T use the nas the speeds drop, but if i reboot samba the speeds are back to normal)

And the RAM is not the problem, I've tried to add up to 16GB and nothing changes, after few minutes of NOT doing any transfer the speed drops, but if I do any transfer (without stopping) on any disk the speed will remain normal

 

[Pietroos]

And you can't tell me that a samsung qvo does not hold 1000MB/s and needs to go at 1MB/s

 

[Pietroos]

Hi, I have some updates about the topic, the problem sometimes it comes back, but it's not a constant thing, and as I said in earlier posts if I reboot the SMB service, everything comes back to normal.

The only thing I changed from before is the CPU, from Pentium G620 to a Pentium G630, nothing else.

 

[Samuel Tai]

Are you using encryption? Neither the G620 or the G630 have AES-NI in hardware, so if you're using encryption, you may be causing the CPUs to overheat.

 

[Pietroos]

No, the CPU isn't overheating. On the server I have a temp monitor which will send me a notification if the CPU goes above 80°C

 

[no_connection]

What power settings do you have? Does it change clock speed when idle?
I let my CPU run at base clock all the time and just let core park do it's job. I sacrifice some 5W maybe but it's a lot snappier.
Wonder if your CPU get stuck at 200MHz or something and stay there.

 

[Pietroos]

What power settings do you have? Does it change clock speed when idle?
I let my CPU run at base clock all the time and just let core park do it's job. I sacrifice some 5W maybe but it's a lot snappier.
Wonder if your CPU get stuck at 200MHz or something and stay there.

Sorry for the late reply. The power settings are all at the best performance possible, however I don't think is the CPU, even if I use the NAS is barely used

 

[Samuel Tai]

Ok, so this behavior manifests with all disks. It looks like something with the Ethernet controller, then. Which driver are you using? The em or igb drivers? Have you enabled or disabled offload?

 

[subhuman]

And you can't tell me that a samsung qvo does not hold 1000MB/s and needs to go at 1MB/s

A Samsung 860 QVO 1TB drive will NEVER do 1000MB/sec. Nor does SATA-3 support that speed. At best, an 860 QVO does a little over 500MB/sec until its cache fills, after which it drops to around 80MB/sec. That's less than 1/12 of what you seem to be expecting out of this drive.
See the first chart on this page: https://www.tomshardware.com/reviews/samsung-860-qvo-ssd,5920.html