FreeNAS Discord

[Snake3y3s]

Is there a FreeNAS Discord server?

I need help with my server... it has all of a sudden started performing very slowly and I cant understand why.
I have not made any changes to the system...

thank you

 

[Heracles]

Hey @Snake3y3s,

This forum is the place to start learning and asking questions. There is also a chat section even if it is not used that much. People who can support you are not on live call to answer whenever you need help. That kind of direct and instant support is what you can get from IX when you buy commercial support.

So first thing will be to read this.

Then post your details accordingly, including all the details about the setup and a maximum of info about the situation. It is not that useful to just say that something is slow or not working. Give measure, error details and more.

 

[Snake3y3s]

Thank you for the response Heracles.

I already have a thread running where I give a whole lot of information and I did have Yorick give me a few solutions, but it has not helped so far.
His main suggestion was to upgrade/transfer the OS from a thumbdrive to a SSD (which I have now done)

My server was working perfectly okay, have it running as a basic NAS with the added advantage of a PLEX server.
it is a older version of freeNAS, but it was serving me just fine. Then all of a sudden it has started performing EXTREMELY slow. I made no changes to the device (except for possibly adding some more files, but the drives are no where near full capacity.)
(it MIGHT be one of the drives, but the GUI is still saying all is well)
so I need some help or a checklist or someone to point me in the correct direction on what to look for that might be causing the slowdown

 

[Heracles]

I already have a thread running where I give a whole lot of information

Then it will be better to either keep working from that thread or at least, link to it in a new one like this one.

but the drives are no where near full capacity.

No need to be that high for performance to drop. Actually, performance will start dropping from as low as 10%. Down to 50%, you will have a very significant drop compared to an empty pool.

The reason is that ZFS is copy-on-write, so does a lot of writes. When ZFS needs to write stuff, it must first find an empty spot. If the drives are loaded up to 10%, that means FreeNAS will find a free spot first shot 90% of the time and only 10% of the time it will need a second try. At 50%, it means that half the time, FreeNAS will have to try a second time for finding a spot. That second try also have 50% of failing. At then end, you will need 3 tries (50% ; 75% ; 87.5%) to be about 90% sure that you will have a free spot to write to. So 3 shots instead of a single one : that is 3 times as many for going from 10% used space to 50% used space.

Performance testing is more than tricky. You will need to test everything in a way that will be meaningful. Just read - write tests to the pool is not easy to define and achieve. Performance may vary significantly just between testing few big files vs many small ones. The easiest thing to test would be the network, testing from RAM to RAM between two computers. You can read from /dev/chargen and write to /dev/null between 2 netcat instances for example.

So tell us more about these performances drops, what you achieved so far, what you tested so far, what measured, ... The more you tell us, the more we will be able to help.

 

[ornias]

I already have a thread running where I give a whole lot of information and I did have Yorick give me a few solutions, but it has not helped so far.

The best place to ask is here, as most of the people with real knowhow are here... There is no magical discord/slack with more knowhow...

That being said, i'm not going to get into the details of it... Because thats offtopic.

 

[Dan Tudora]

hello
is NOT magic here in this forum ??
I am go away from that kind of forum !!
on topic now
some hardware test/HDD test ??

My server was working perfectly okay

when ??
make some hardware change, OR have hardware failure in near past ??
maybe is time to change that old piece of hardware!!
maybe https://www.advancedclustering.com/act_downloads/breakin/ this kind of software testing help you
looks like is from yours server age

 

[Chris Moore]

Code:

Old school HP ProLiant Microserver (N36L) Gen 7
• AMD Athlon(tm) II Neo N36L Dual-Core Processor
• 8025mb of RAM
• OS version - FreeNAS-11.0-U4 (54848d13b)
• On-board ethernet is dead so have a intel I210 NIC installed in one of the PCI slots
• 5x 3TB WD red SATA

Slow, old dual core processor and only 8GB of memory will make the system struggle ... What is the pool configuration, RAIDz2?

 

[Heracles]

Just noticed that you had WD Red drives... Are they SMR ??? If they are, you will have to get rid of them.

 

[Chris Moore]

Is there a FreeNAS Discord server?

This forum is where we handle community support of FreeNAS and TrueNAS. That is why iXsystems, the people that make the software, created the forum to begin with.

You need to read the forum guidelines:

Forum Rules

Welcome to the TrueNAS Community! Updated 11/28/2023 First and foremost I want to thank you for joining the TrueNAS community! My name is JoshDW19 and I'm the Community Manager here at TrueNAS. The TrueNAS Community is a friendly place to discuss open source software, technology, and products...

www.truenas.com

Then provide a greater degree of detail regarding your system configuration so we can know hot to help you instead of having to guess at what your problem is.

 

[Snake3y3s]

Thank you all for the responses...

For more information on my system, check in my signature or on this thread

My system was working fine about 2 or 3 months back:
As in I could play a video from the server to my local machine and be able to scrub the timeline and it would be responsive or play AND use plex on the server on the playstation in the lounge. Now... plex takes forever just to start up, and even the posters take forever to load, I cant play any videos through plex at all, or it just takes forever, but if i try to scrub it locks up again. even just normal browsing files on the server is slow (just entering my username and password seems to take a good 5 minutes to verify)

I made zero hardware changes.

Just noticed that you had WD Red drives... Are they SMR ??? If they are, you will have to get rid of them.

I doubt that they are SMR as I bought them a number of years ago (before the whole RED SMR debacle) but i could give you the serial numbers of the drives to check maybe?

How/where would I check what RaidZ I have it set up... I have forgotten how I set it all up as I did it a good 3 or 4 years back.

one thing I have noticed is that ada3 does seem to be on 100% disk usage (when checking disk busy under reporting) and the disk latency also seems very high... could this be the culprit? what could it be doing? is there a way to check?

 

[Snake3y3s]

could I run a smart test on ada3 and output the results to a log file or a text file and get some assistance?

 

[Heracles]

To figure what what your pool layout is, use the zpool status command. Post this with your smart tests results and we will look at it. Remember to use the proper Code tags in your post to mark your results. It makes it million time easier to read it for us.

 

[Snake3y3s]

Code:

[root@NAS ~]# zpool status                                                                                                         
  pool: NASvol1                                                                                                                     
 state: ONLINE                                                                                                                     
  scan: scrub repaired 0 in 5h13m with 0 errors on Sun Nov 15 05:13:26 2020                                                         
config:                                                                                                                             
                                                                                                                                    
        NAME                                            STATE     READ WRITE CKSUM                                                 
        NASvol1                                         ONLINE       0     0     0                                                 
          raidz2-0                                      ONLINE       0     0     0                                                 
            gptid/1a1054dd-c7ee-11e7-9d5a-a0369fd4e01a  ONLINE       0     0     0                                                 
            gptid/1b47af73-c7ee-11e7-9d5a-a0369fd4e01a  ONLINE       0     0     0                                                 
            gptid/1c83e873-c7ee-11e7-9d5a-a0369fd4e01a  ONLINE       0     0     0                                                 
            gptid/1db63dbb-c7ee-11e7-9d5a-a0369fd4e01a  ONLINE       0     0     0                                                 
            gptid/1ee63fe8-c7ee-11e7-9d5a-a0369fd4e01a  ONLINE       0     0     0                                                 
                                                                                                                                    
errors: No known data errors                                                                                                       
                                                                                                                                    
  pool: freenas-boot                                                                                                               
 state: ONLINE                                                                                                                     
  scan: none requested                                                                                                             
config:                                                                                                                             
                                                                                                                                    
        NAME        STATE     READ WRITE CKSUM                                                                                     
        freenas-boot  ONLINE       0     0     0                                                                                   
          da0p2     ONLINE       0     0     0                                                                                     
                                                                                                                                    
errors: No known data errors       

So that would be Raidz2 I take it.

with regards to the smart test results, how would I acquire these and post them? (doing the whole status thing is still very new to me)

also, (with regard to ada3 situation) how can I get all the reporting graphs as a screenshot so that I can add it here? or would i need to take individual shots of each graph and stich it together in photoshop or something?
or is there a way I can end any process or activity happening on ada3 so that i can see if that is indeed the problem.

 

[Snake3y3s]

hmmm... okay, so did some further research.
Could my NAS be performing a Scrub?
is a scrub per volume or per drive?

if so... how could I check if it is performing a scrub and is there a way to check at what percentage said scrub is at?

also, if I restart/shutdown the NAS, does the scrub cancel or does it just resume from where it left off, or does it start all over when I boot it up again?

 

[Alecmascot]

The scrub status is in "zpool status"

 

[Snake3y3s]

so then its not performing a scrub (according to the zpool status that I posted earlier)
how can I figure out what is causing 100% disk activity on ada3??

 

[Alecmascot]

If ada3 is still very busy then take it offline and see if the performance improves.
The pool will go degraded but you can live with that for a little while.
Then you can bring it online again; there will be a little resilver and the pool status will be restored.
Use the documentation applicable to your release to see how do this

 

[Snake3y3s]

I could give that a shot, but what if the problem persists? is there no way to see what activity is happening on the drive so that i could try end the process?

This is what my hardware reports for the drives looks like:

 

[Alecmascot]

The pool drives run as an array and the activity should look the same.
ada3 looks like it is internally busy, possibly doing some kind of correction/recovery.
you could display the smart data for ada3:

smartctl -a /dev/ada3

 

[Snake3y3s]

I think I did try that command and didn't see anything out of the ordinary...
Anyway it would seem that the activity has now dropped from a constant 100% to now spiking between 60% and 80% on ada3
the rest of the drive are still going between 2% and 5% activity