Alert notifications

From YobiWiki
Jump to navigation Jump to search

This is a generic page about reporting all kind of misbehaviours from a server.
This is draft, to be implemented :-)

Data collection

  • Syslog
  • Munin
  • acct & atop
  • procinfo
  • exim stats
  • apache stats
  • netstat -l...

What to filter for what kind of alert?

Mail alerts

Jabber/SMS alerts

You can use sendxmpp

apt-get install sendxmpp
echo "Here is a very cool notification agent"|\
   sendxmpp -u <user> -p <passwd> -j <server> -s <subject> <destinataires>

Note that v1.13-1 is buggy, apply this patch:

--- /usr/bin/sendxmpp    2008-09-21 00:51:15.000000000 +0200
+++ /usr/bin/sendxmpp    2008-09-21 00:47:32.000000000 +0200
@@ -281,7 +281,7 @@
                connectiontype  => 'tcpip',
                componentname   => $comp
        };
-       $arghash->{port} = $port if (!$port);
+       $arghash->{port} = $port if ($port);
        if (!$port) {
                @res = $cnx->Connect(%$arghash);
                error_exit ("Could not connect to server '$host': $@") unless @res;
  • Hardware damages
    • temp, fans
    • raid
  • Software damages
    • HD capacity
    • CPU load 100% for more than X mins
      The easiest is to take the third field of /proc/loadavg which is a mean over the last 15 mins, here with 2 CPUs:
awk '$3 > 2 {print "alert"}' /proc/loadavg
    • network load > X for more than Y mins
    • exim load > X mails sent per min

From sendxmpp author:

if test "$cpuload" -gt "$CPULOADMAX"; then
 top -b -n 1 | sendxmpp -s "wake up! cpu load $cpuload at `hostname`" someone@jabber.org  
fi

To get all logcheck mail and mails with subject starting with JABBER I added this to my procmail:

:0 c                                                                                                                                                                      
* ^From: .*logcheck|\                                                                                                                                                     
  ^Subject: JABBER                                                                                                                                                       
* ^Subject: \/.*                                                                                                                                                          
| formail -k -X From:|sendxmpp -u sendxmpp -p xmpp -j <jabber server> -s "$MATCH" <my@jabber.account>