[baggins]

Hi Robbie,

Sorted out the nems-update issue: this was related to the fact that I set a fixed IP address with Webmin (I cannot set this in my ISP owned router/dhcp server). I forgot to set the gateway

With that OK, I ran nems-update and this time no errors.
I also ran nems-quickfix and did the NEMS SST saving stuff.

At the moment I can't see errors in the sendemail log, but I also do not receive mail notifications even though I induced one of my servers to a critical service state which is reported by Nagios Core and Notifications are enabled...


Edit: It took some time, but the error is back...
BR

[Robbie Ferguson]

Okay thanks, baggins! We're making progress anyways. And I'm glad to hear the update issue was your fault, not mine - haha!  Be careful of that kinda thing... not sure how I can prevent users from making the same mistake.

baggins & alerty - thanks for helping with some tests and reporting back. I will continue working away on this... probably will spend a fair bit of time over the weekend too, and see how quickly I can get this one fixed.

Cheers! Thanks for your patience, all!

Robbie

[norbertooka]

This is the bug for me, please send info.

Manny Thx
norbert

[Robbie Ferguson]

Thanks norbert. I also received your backup.nems file, so I will investigate this further.

I have again checked my own NEMS server and am still unable to replicate the issue -- but because it's affecting a small handful of users, I am making this a high priority. Hopefully having another backup.nems of an affected system will help me track it down.

Cheers,
Robbie

[Robbie Ferguson]

norbert - it looks like your NEMS server has been running since October - so I'm just curious, is this problem something that suddenly appeared? Or has it always been there and you simply didn't notice it?

EDIT: I see from the log this started just a few days ago, but everything looks like it was working perfectly up till then.

Did you do anything to change the NEMS server (eg., new configs, or any changes in your settings) or did this simply start happening on its own?

This log is very helpful - thank you.

[Robbie Ferguson]

Alright gang, just an update...

Because my own NEMS Servers are not exhibiting this problem, and I have yet to be able to replicate it, I am going to flash a new copy of NEMS 1.3 and boot it up like a "new user" and go through the normal init and setup to see if I am able to get it to replicate the issue you are having.

If any of you have any insight at all such as "the problem started after I did __this__" that'd be really helpful. Otherwise, fingers are crossed that I'm able to re-create the problem on my own system and fix it for you quickly.

Thanks!

[baggins]

Hi Robbie,

I may be totally off track here, but while investigating this on  my system I came across the following in the nagios.log file after I rebooted NEMS:

Code:

[1517158341] Successfully shutdown... (PID=1578)
[1517158343] livestatus: Socket thread has terminated
[1517158343] Event broker module '/usr/lib/mk-livestatus-1.2.8p13/src/livestatus.o' deinitialized successfully
.
[1517158389] Error: NULL variable value - Line 11 of resource file '/etc/nagios3/resource.cfg'  <<<<=========
[1517158389] Nagios 3.5.1 starting... (PID=1445)

Now in my resource.cfg file, line 11 (if I also count the comment lines) is $USER3$ which in my case is blank as is the following line $USER4$.
$USER13$ and $USER14$ are also blank.

Could it be that nagios starts reading the resource file and crashes on the first NULL value and stops reading, because the next line is $USER5$ which is the 'From' address, so that would not be picked up, resulting in the error that we have...

BR

[Robbie Ferguson]

Great news: I was able to replicate the issue having re-flashed to NEMS 1.3 (I've been running the development version of 1.3.1 for a while now and was unable to re-create it... I should buy another Pi so I can always have a current "public" version running).

baggins - thank you for the info. I'm testing your theory now!

Robbie

[Robbie Ferguson]

baggins!!!! YOU NAILED IT! Well done, sir! And THANK YOU!

Before patching:
Jan 28 15:21:58 nems sendemail[16524]: ERROR => You must specify a 'from' field! Try --help.

After patching:
Jan 28 15:35:00 nems sendemail[5350]: Email was sent successfully! From: <i@robbief.com> To: <robbie@category5.tv> Subject: [** PROBLEM Service Alert: Nagios Enterprise Monitoring Server/Swap Usage is CRITICAL **] Server: [mail.promail.ca:587]

I'm writing the patch to NEMS SST now and will notify you all here when it is ready for you.

[Robbie Ferguson]

Okay, the patch has been rolled out.

Please type: sudo nems-quickfix

Then, open NEMS SST and press "Save".

Let me know if this has fixed the issue for you. Thanks all!