Oct 102010
 

Hi,

In a previous; post we covered the installation of satellite in LAN. It is usefull if you have to monitor machines from distant sites. Generaly these sites are accessible with your local network or through a VPN but sometimes, the site is far away and is only accessible via internet connection. In most case, it’s located behind a firewall.
We will try to describe how we could set a secured connexion between distant satellite and our central monitoring server using SSH connection.

It will looks like something like that:

satelitte-ssh

First thing to do is to open an SSH access on both sides. In my case, I used an IPCOP firewall but it’s quite similar if you have an endian firewall or something else… The main idea is to forward the SSH port 22 to the Central centreon server on one side and to your Satelitte on the other side.

So, on Central IPCOP server side I’ll have two settings:

And external access allowed on port 22.
Port forwarding on TCP port 22 to port 22 on my internal machine 192.168.3.22 only allowed from 10.10.10.10 sender.

On the other side, on Satelittes IPCOP I’ll have also:

And external access allowed on port 22.
Port forwarding on TCP port 22 to port 22 on my internal machine 192.168.0.166 only allowed from 11.11.11.11 sender.

In my example, I’ve restricted the access to my own IP and I highly recommend you proceed like that. It’s more secure (and prevent jackass to try to connect to your system).

To verify that your SSH access is enabled, try to connect with SSH on your satellite to the central Centreon and vice versa (use the external IP adress as destination).

Now that the SSH port have been openned, let’s start with the satellite install.

Satelitte install

The installation process is quite similar to our previous satellite install except some IP port which will be different.

Here is a spreadsheet for the different login and passwords that will be used for the installation process (some of them are used in further post):

Account role Login Password
Nagios nagios nagaccess03
User account activlan remoteaccess03
Web Nagios nagiosadmin nagwebaccess03

This part is done on the distant site:

Insert you CD, boot from it and proceed for a minimal install.
For the partition scheme let the system choose one for you (I recommend using LVM because in case of “out of disk space” it will be easier to extend disk).
When asked, disable the automatic update.
DO NOT INSTALL ANY ROLE EXCEPT SSH.

System update

Once it’s installed, proceed with a system update

sudo aptitude update
sudo aptitude full-upgrade -y

Switch to fixed IP address

sudo nano /etc/network/interfaces

Please verify that you are working with fixed IP address. If not, please change your settings. You should see something like this at the end of the file (the value should be different for you):

auto eth0
iface eth0 inet static
address 192.168.0.166
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.254

REBOOT

Compilation component install

sudo aptitude install build-essential -y
sudo aptitude install linux-headers-$(uname -r) -y

The headers should be already present.
If you are using a virtual machine, it’s time to install the appropriate tools (it’s up to you).

System

sudo aptitude install sysutils policykit ntp -y

Web and php5

sudo aptitude install apache2 apache2-mpm-prefork libapache2-mod-php5 -y
sudo aptitude install php5 php5-mysql -y
sudo aptitude install php-pear php5-ldap php5-snmp php5-gd -y

RRDTool

sudo aptitude install rrdtool librrds-perl -y

Perl

sudo aptitude install libconfig-inifiles-perl libcrypt-des-perl libdigest-hmac-perl libdigest-sha1-perl libgd-gd2-perl -y

SNMP

sudo aptitude install snmp snmpd libnet-snmp-perl libsnmp-perl -y

GD-Libraries

sudo aptitude install libgd2-xpm libgd2-xpm-dev libpng12-dev -y

Other needed library

sudo aptitude install libldap2-dev libssl-dev libsnmp-dev -y

Now we are ready to install Nagios.

User install

PS: use “nagaccess03” as password for nagios account.

sudo useradd -m -s /bin/bash nagios
sudo passwd nagios
sudo groupadd nagcmd
sudo usermod -G nagios,nagcmd nagios
sudo usermod -a -G nagios,nagcmd www-data

Nagios Core

cd /usr/local/src
sudo wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.2.tar.gz
sudo tar xvzf nagios-3.2.2.tar.gz
cd nagios-3.2.2
sudo ./configure --prefix=/usr/local/nagios --with-nagios-user=nagios --with-nagios-group=nagios --with-command-group=nagcmd --enable-nanosleep --enable-event-broker
sudo make all
sudo make install
sudo make install-init
sudo make install-commandmode
sudo make install-config
sudo update-rc.d -f nagios defaults
cd ..

Nagios plugins 1.4.15

sudo wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz
sudo tar xvfz nagios-plugins-1.4.15.tar.gz
cd nagios-plugins-1.4.15
sudo ./configure --prefix=/usr/local/nagios --with-nagios-user=nagios --with-nagios-group=nagios
sudo make
sudo make install
cd /usr/local/src

Ndoutils 1.4b9

sudo wget http://prdownloads.sourceforge.net/sourceforge/nagios/ndoutils-1.4b9.tar.gz
sudo tar xvfz ndoutils-1.4b9.tar.gz
cd ndoutils-1.4b9
sudo ./configure --prefix=/usr/local/nagios/ --enable-mysql --disable-pgsql --with-ndo2db-user=nagios --with-ndo2db-group=nagios
sudo cp ./config/ndo2db.cfg-sample /usr/local/nagios/etc/ndo2db.cfg
sudo cp ./config/ndomod.cfg-sample /usr/local/nagios/etc/ndomod.cfg
sudo chown nagios:nagios /usr/local/nagios/etc/ndo*

It appears that with large monitoring system, ndoutils use a lot of IO and slows down drasticly the system. It’s better to installe the patched version.

cd /usr/local/src
sudo wget -r -k -np http://svn.centreon.com/trunk/Patch-NDOutils/
cd svn.centreon.com/trunk/Patch-NDOutils
sudo bash ./configure --prefix=/usr/local/nagios/ --enable-mysql --disable-pgsql --with-ndo2db-user=nagios --with-ndo2db-group=nagios
sudo make
sudo cp src/ndomod-3x.o /usr/local/nagios/bin/ndomod.o
sudo cp src/ndo2db-3x /usr/local/nagios/bin/ndo2db
sudo chmod 775 /usr/local/nagios/bin/ndo2db
sudo chmod 755 /usr/local/nagios/bin/ndomod.o
sudo chown nagios:nagios /usr/local/nagios/bin/ndo2db
sudo chown nagios:nagios /usr/local/nagios/bin/ndomod.o

Nagios web interface

I also recommend installing Nagios web interface as it could help you comparing your centreon checks and nagios exported configuration.

Use “nagwebaccess03” as password

cd /usr/local/src/nagios-3.2.2
sudo make install-webconf
sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

SNMP settings

Please edit /etc/default/snmpd

sudo nano /etc/default/snmpd

and change these sections:

#SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1'
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1 192.168.0.166'

Replace 192.168.0.166 by your IP address.

#TRAPDRUN=no
TRAPDRUN=yes

Edit snmpd configuration file:

sudo nano /etc/snmp/snmpd.conf

and change this section:

#       sec.name  source          community
#com2sec paranoid  default         public
com2sec readonly  default         public
com2sec readonly  127.0.0.1         public
com2sec readonly  192.168.0.0/24         public
#com2sec readwrite default         private

Edit ndomod configuration file:

sudo nano /usr/local/nagios/etc/ndomod.cfg

find and replace these sections:

#instance_name=default
instance_name=Central
output_type=tcpsocket
#output_type=unixsocket
output=127.0.0.1
#output=/usr/local/nagios/var/ndo.sock
#reconnect_warning_interval=15
reconnect_warning_interval=900
#config_output_options=2
config_output_options=3

Edit sudoers grant file:

sudo nano /etc/sudoers

and add nagios user account after root account:

nagios   ALL=NOPASSWD: /etc/init.d/nagios restart
nagios   ALL=NOPASSWD: /etc/init.d/nagios stop
nagios   ALL=NOPASSWD: /etc/init.d/nagios start
nagios   ALL=NOPASSWD: /etc/init.d/nagios reload
nagios   ALL=NOPASSWD: /usr/sbin/nagiostats
nagios   ALL=NOPASSWD: /usr/sbin/nagios *

Create missing folder:

sudo mkdir -p /var/lib/centreon/centplugins/
sudo chown nagios:nagios /var/lib/centreon/centplugins/

The following steps have to be done on CENTRAL CENTREON SERVER

If you didn’t generate ssh key yet, proceed as follow.
Log you on Centreon Central server and switch to nagios account.

su nagios
cd ~
ssh-keygen

Here is an example of produced output (just press ENTER to all questions):

Generating public/private rsa key pair.
Enter file in which to save the key (/home/nagios/.ssh/id_rsa): 
Created directory '/home/nagios/.ssh'.                                                                                     
Enter passphrase (empty for no passphrase):                                                                                
Enter same passphrase again:                                                                                               
Your identification has been saved in /home/nagios/.ssh/id_rsa.
Your public key has been saved in /home/nagios/.ssh/id_rsa.pub.
The key fingerprint is:
17:f1:f4:f3:2a:de:94:eb:36:8a:82:84:d3:cb:cf:ee nagios@act-centreon-test
The key's randomart image is:
+--[ RSA 2048]----+
|          . .    |
|           + .   |
|          . . o  |
|           .   o |
|     o  S .     .|
|    o o  .     o |
|     + o    . +  |
|      +..  o +o. |
|       +E.. o++. |
+-----------------+

After that, copy your public key to the satellite:

ssh-copy-id -i ~/.ssh/id_rsa.pub nagios@{SATELITTE_FIREWALL_EXTERNAL_IP_ADDRESS}

Use nagaccess03 for accessing your satellite (or use your own password).
Verify that you could access to the satellite without password prompt:

ssh nagios@{SATELITTE_FIREWALL_EXTERNAL_IP_ADDRESS}

Exit to came back to your central centreon server.

If it does not work, check your firewall settings. This must work, it’s a PRE REQUIREMENT for going further.

From central centreon
Synchronize commands with the satellite (only if you are using the same architecture on both side 32bits or 64bits):

scp /usr/local/nagios/libexec/* nagios@{SATELITTE_FIREWALL_EXTERNAL_IP_ADDRESS}:/usr/local/nagios/libexec
scp /usr/local/nagios/bin/ndomod.o nagios@{SATELITTE_FIREWALL_EXTERNAL_IP_ADDRESS}:/usr/local/nagios/bin

if you encounter access denied errors like:

scp: /usr/local/nagios/libexec/check_dhcp: Permission denied

Delete files from satellite and resync again with scp.

Optionnaly, if you use rsyslog module in Centreon, redirect your logs:
Edit rsyslog configuration file

sudo nano /etc/rsyslog.conf

Add at the end of the file log events redirection inside your database:

 *.=notice;mail.*;\
       *.=crit;*.=err;\
       *.=warning @@{127.0.0.1}

We use 127.0.0.1 instead of the real IP address. It will be redirected through SSH tunnel. We used the double @ for TCP port (single @ will use UDP access).

Edit 50-default.conf to correct the path

sudo nano /etc/rsyslog.d/50-default.conf

At the end of the file, comment the xconsole section:

#daemon.*;mail.*;\
#       news.err;\
#       *.=debug;*.=info;\
#       *.=notice;*.=warn       |/dev/xconsole

Optionnaly intall NRPE module:

cd /usr/local/src
sudo wget http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.12/nrpe-2.12.tar.gz
sudo tar xzfv nrpe-2.12.tar.gz
cd nrpe-2.12/
sudo ./configure
sudo make all
sudo make install-plugin

SSH Tunelling

The main idea is to redirect through SSH the NDO messages (on TCP port 5668) and for thoose using rsyslog the system messages on our main console (on TCP 514).

Configure our main ssh config file:

sudo nano /etc/ssh/ssh_config

At the end of the file please edit like this:

Host activlan
        Hostname 11.11.11.11
        User nagios
        CheckHostIP no
        LocalForward 5668 127.0.0.1:5668
        LocalForward 514 127.0.0.1:514
        Compression yes

If you are not planning to redirect Rsyslog messages, remove the ‘LocalForwad 514’ line.

You have to generate your SSH key as root. It will be used by our ssh tunelling daemon.

sudo ssh-keygen

Here is an exemple of resulting output:

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
16:44:58:0e:da:f4:19:df:04:df:e4:ba:21:a7:ce:df root@centreon-dejan
The key's randomart image is:
+--[ RSA 2048]----+
|      o+= ... .  |
|     +.= + + +   |
|    . . = . o o  |
|         .   .   |
|        S . +    |
|       .   + o   |
|          . .    |
|         o   .   |
|          o.. E  |
+-----------------+

Then copy the public key to the distant server:

sudo ssh-copy-id -i /root/.ssh/id_rsa.pub nagios@11.11.11.11

Replace 11.11.11.11 by your external IP adress. Enter the nagios password and your key should be copied.
Try to connect to the distant server like that:

sudo ssh nagios@11.11.11.11

You should be logged in without password.

Install autossh on your satellite:

sudo aptitude install autossh

create your launching script:

sudo nano /etc/init.d/autossh-startup.sh

and put this line:

autossh -N activlan&

make it executable and autostart:

sudo chmod +x /etc/init.d/autossh-startup.sh
sudo update-rc.d autossh-startup.sh defaults

Reboot your satellite.
Your ssh tunnel should be active.

Connect to the web interface on central Centreon server and proceed as follow:

Add a new pooler:

Duplicate the ndomod.cfg configuration

Edit it like this:

Duplicate the nagios.cfg configuration

Edit it like this:

Add your new host (which is your satellite):

Go to ‘Configuration / Nagios’ and export like this your settings:

Check on both Central and Satellite that you did not have any error message:


Then you could export your settings:

You should see very soon something like this:

  6 Responses to “Install Centreon Satellite with secured connexion (ssh tunnel)”

  1. What about using the autossh from the Central Centreon and dialing to the Satellite?

    If the Satellite is hacked it would not a easily endanger the the Central Centreon

    Anyways your blog is a fantastic ressource about Nagios, monitoring and Ubuntu.

    Thank you very much!

  2. Hi iiome,

    To be honest, I have not think about this situation. I left of the principle that I initiate the ssh tunneling from the source to the destination (and mainly because all tutorials I’ve found on internet used the tunneling in this way but may be I’m wrong).

    In the tutorial, I recommend to filter the ssh entry on both side (central server and satellite) this way you will minimize the hacking risk. I’ve tried a few minutes unfaltering IP address on the ssh port and a few minutes later I’ve been attacked from North-Korea with “brut force” attack (tons of invalid root loggin).

    I could only advise to not use unfiltered access to your ssh access or maybe use a tutorial for hardening ssh connection (those filters could be also activated in configuration files).

    Cheers

  3. Hi, great tutorial, your effort is highly appreciated.

    I’m having the issue that the command autossh -N activlan doe not work. It’s not complete.

    Just to be sure:
    – ssh keys are exchanged, ssh login without password is succesfull
    – I edited the /etc/ssh/ssh_conf on the centreon server
    – Installed autossh on centreon and satellite
    – tried the command autossh -N activlan on the satellite

    I’m running Centos, shouldn’t make a difference I guess. What am I missing?

    Grtz,
    Roderick

  4. I can create the tunnel now. That’s not the issue. The issue is how to start the tunnel just using the short commandline autossh -N activlan

  5. Hi mrcoffee11

    I’m not familiar with CentOs but the idea should be something like that.

    Create a bash script which will be launched when you start your server (on ubuntu, I’ve created a script here: /etc/init.d/autossh-startup.sh with the corresponding autossh command).
    After that I made the script runnable (sudo chmod +x /etc/init.d/autossh-startup.sh) so it could be started when the server starts.
    And finally I’ve declared this command to start whith the system (sudo update-rc.d autossh-startup.sh).

    To summarize, the bash script declared “as a service” will launch the ssh tunnel when the computer start.

    Regards

  6. Hi Dejan,
    I created an init script, works good.

    You mention to use this command: autossh -N activlan&. This does not work for me, autossh complains that it’s not a correct command. I added the info to /etc/ssh/ssh_conf.

    Grtz
    Roderick