Cannot Reach WEBGUI or SSH, MAXPROC LIMIT EXCEEDED Showing on console

[Unixsysadmin]

Hello,

I have just inherited a freenas box that is having issues. It is a supermicro chassis with freenas 9.10.2-U2 currently installed. Atleast I assume that is the version as its sister NAS that was setup at the same time is on that version, sister NAS is working great.

Anyways, I cannot get to the machine via web gui or SSH. When I browse to the OOB IP of the supermicro chassis, I can see on the console MAXPROC LIMIT EXCEEDED by uid 0 (pid 10176; see tuning(7) and login.conf(5)

I am not onsite, but will be flying out early next week so I will actually be able to troubleshoot this locally.

But I wanted to get some insight from the forum before I arrive onsite. Thank you ahead for any assistance.

Regards,
Jacob S.

Edit: All shares and other storage access to the appliance is still functioning.

 

[Chris Moore]

I cannot get to the machine via web gui or SSH.

Edit: All shares and other storage access to the appliance is still functioning.

The combination of these two factors usually indicate a failed boot drive. Do you know exactly what hardware is in the system?

 

[Unixsysadmin]

I have not been able to put hands on it yet. And nothing was documented at this site as it is a new acquisition and the previous admins must not have known what documentation was.

Anyways, as far as i know it is a just a supermicro box with JBOD "just a bunch of disks."

So I assume some sort of raid was setup and does not use a specific boot drive.

It is possible we do not have the right IP for this machine. I will not know for sure until I get onsite and in the console.

Very concerned with the maxproc limit error.

This is what its sister box looks like in terms of drives. I assume potentially those first two drives are boot. But cant say for sure.

 

[Chris Moore]

Anyways, as far as i know it is a just a supermicro box with JBOD "just a bunch of disks."

So I assume some sort of raid was setup and does not use a specific boot drive.

How familiar are you with FreeNAS / ZFS?

There are so many ways to configure this, it depends on exactly what options were selected and what vendor it was ordered from or if the components were purchased and assembled in-house.

Based on the sister box, it looks like you have a pair of 240GB drives in a mirror (da0 + da1) for the boot pool using ZFS for the mirroring. We don't do any hardware RAID. Then the other drives would likely be in a RAIDz2 or pool of mirrored vdevs. Depending on what the usage is.

 

[Chris Moore]

The number in the serial column is supposed to be the actual drive serial number and the ones for da0 and da1 look kind of like the ones on Intel SSDs. It would be unusual to have a mirrored pair of Intel SSDs fail, but I don't know how long the system has been in service.

 

[Chris Moore]

Very concerned with the maxproc limit error.

Do you have any additional functionality configured on these systems or are they solely for file storage?

 

[Unixsysadmin]

It has been in service as far as I can see, since 2012.

As far as I know this is an in house build. Supermicro chassis.

Have you see this maxproc limit error before?

Thank you for the help Chris Moore.

 

[Unixsysadmin]

Do you have any additional functionality configured on these systems or are they solely for file storage?

As far as I know its solely for file storage. Hard for me to say for sure because I cant get into the console and I did not set these up.

 

[Chris Moore]

Thank you for the help Chris Moore.

Your welcome.

Have you see this maxproc limit error before?

I have only heard of it happening. I have not seen it myself.
The situation was associated with a runaway process that was using up all the memory.
It may be solved as easily as a reboot, if that is all that is wrong.
If you are able to remotely restart the system, you might want to give that a try, but it is a risk because it may not come back up.
FreeNAS is usually installed to a pair of drives and sometimes, when one of the drives develops a problem, the system will not automatically switch to the other drive. Still, it is not guaranteed to be a boot disk fault.

 

[Unixsysadmin]

Your welcome.

I have only heard of it happening. I have not seen it myself.
The situation was associated with a runaway process that was using up all the memory.
It may be solved as easily as a reboot, if that is all that is wrong.
If you are able to remotely restart the system, you might want to give that a try, but it is a risk because it may not come back up.
FreeNAS is usually installed to a pair of drives and sometimes, when one of the drives develops a problem, the system will not automatically switch to the other drive. Still, it is not guaranteed to be a boot disk fault.

I do want to restart although I cant remotely. I am worried about the production machines that utilize the NAS having issues while I reboot.

 

[Chris Moore]

I do want to restart although I can't remotely. I am worried about the production machines that utilize the NAS having issues while I reboot.

With the age of this system (2012) it might be time to consider a life-cycle replacement. The company that makes FreeNAS, iXsystems, also makes a line of systems called TrueNAS and some of them have redundant heads so that if one fails the other automatically takes over. They are a bit more cost but it protects the production systems from down time. Would you like to know more?

 

[Unixsysadmin]

With the age of this system (2012) it might be time to consider a life-cycle replacement. The company that makes FreeNAS, iXsystems, also makes a line of systems called TrueNAS and some of them have redundant heads so that if one fails the other automatically takes over. They are a bit more cost but it protects the production systems from down time. Would you like to know more?

That sounds pretty slick. But we are already sizing the environment for two new Nimble SAN arrays (one for DB's one for prod VM's on the xenserver hypervisors) that will replace these FreeNAS and current EMC SAN's.

 

[Chris Moore]

You might want to consider the iXsystems solution instead. They have many options and often for a more competitive price.

Sent from my SAMSUNG-SGH-I537 using Tapatalk