Syslog-ng crashes pools, forces reboot

jrp

Dabbler
Joined
Jan 2, 2023
Messages
39
I configured syslog-ng in the settings - > advanced tab in web Gui by entering the IP address of my Splunk server

I received an error when trying to save the configuration but the configuration did stick.

I received no logs to my Splunk server however.

Several hours later sync routines kicked off and generated sync failures

I ended up receiving 20 some alert emails of which most included references to syslog-ng

Code:
Failed to check for alert SyslogNg: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/plugins/alert.py", line 774, in __run_source alerts = (await alert_source.check()) or [] File "/usr/lib/python3/dist-packages/middlewared/alert/base.py", line 223, in check return await self.middleware.run_in_thread(self.check_sync) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1209, in run_in_thread return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1206, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run result = self.fn(*self.args, **self.kwargs) File "/usr/lib/python3/dist-packages/middlewared/alert/source/syslog_ng.py", line 15, in check_sync p1 = subprocess.Popen(["/usr/sbin/service", "syslog-ng", "status"], stdout=subprocess.PIPE, File "/usr/lib/python3.9/subprocess.py", line 829, in __init__ errread, errwrite) = self._get_handles(stdin, stdout, stderr) File "/usr/lib/python3.9/subprocess.py", line 1593, in _get_handles c2pread, c2pwrite = os.pipe() OSError: [Errno 24] Too many open files


The Gui became unresponsive but managed to reboot over SSH

I removed the Splunk server IP from the syslog-ng configuration and although I received the same timeout error noted above the IP address did get removed.

Has anyone seen this or have had success /failure in sending syslog to an external server?

Thanks in advance
 

jrp

Dabbler
Joined
Jan 2, 2023
Messages
39
Ooops, my home installation is bluefin, this one is at the nonprofit I volunteer at which is currently angelfish
 
Top