[Nagiosplug-devel] Bug in check_by_ssh

[Earl C. Ruby III]

I had a test server that was being checked by Nagios using check_by_ssh. Last 
night I formatted the hard drive on the test server and re-installed the OS 
(SuSE Linux 8.2), using the same IP number as before.

Since the IP number remained the same, all of the Nagios ping tests succeed. 
Since sshd on the test server is running, check_ssh returns OK.

However, since the RSA host key on the test server is now different, all ssh 
keys have to be reset and known_hosts2 files need to be recreated -- all 
check_by_ssh checks should fail, and they do, but they fail with "warning" 
alarms, not "critical" alarms. 

If I try to ssh to the test server from the Nagios server I see the following:

nagios at nagioshost:~/bin> ssh root at 192.168.1.101
3136: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
3136: @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
3136: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
3136: IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
3136: Someone could be eavesdropping on you right now (man-in-the-middle 
attack)!
3136: It is also possible that the RSA host key has just been changed.
3136: The fingerprint for the RSA key sent by the remote host is
0e:67:02:7b:3d:d3:23:39:d1:ff:0e:fe:e8:f0:db:6b.
3136: Please contact your system administrator.
3136: Add correct host key in /usr/local/nagios/.ssh/known_hosts to get rid of 
this message.
3136: Offending key in /usr/local/nagios/.ssh/known_hosts:38
3136: RSA host key for 192.168.1.101 has changed and you have requested strict 
checking.
3136: Host key verification failed.

In my opinion, any time the remote host identification changes check_by_ssh 
should sound a critical alarm. check_by_ssh SHOULD fail with a CRITICAL alarm 
if ssh returns anything via STDERR. If ssh itself is generating errors, that 
is a critical alarm.

In my opinion, this code in check_by_ssh.c:

        /* WARNING if output found on stderr */
        if (fgets (input_buffer, MAX_INPUT_BUFFER - 1, child_stderr)) {
                printf ("%s\n", input_buffer);
                return STATE_WARNING;
        }
        (void) fclose (child_stderr);

... should be changed to this:

        /* CRITICAL if output found on stderr */
        if (fgets (input_buffer, MAX_INPUT_BUFFER - 1, child_stderr)) {
                printf ("%s\n", input_buffer);
                return STATE_CRITICAL;
        }
        (void) fclose (child_stderr);

Also, this:

                if (!(fp = fopen (outputfile, "a"))) {
                        printf ("SSH WARNING: could not open %s\n", 
outputfile);
                        exit (STATE_UNKNOWN);
                }

... should be changed to this:

                if (!(fp = fopen (outputfile, "a"))) {
                        printf ("SSH CRITICAL: could not open %s\n", 
outputfile);
                        exit (STATE_CRITICAL);
                }

-- 
Earl C. Ruby III
Senior Systems Engineer / Developer
Switch Management