Indestructible Cluster part 1

Tweet Version 1.0 of CMON (gpl license) has now been released.
It is dead easy to use and gives you a lot of helpful information about the state of the cluster. Memory utilization, statistics etc.

You can also get graphical representation using a web server and the php scripts that comes with cmon.

If you use it with the new and improved config tool then you can setup a system with process management (monit) and init.d scripts etc in minutes, and cmon will then automatically coordinate the start procedures of the data nodes in the cluster!

First you have to download cmon and then you can check out the documentation for it.

Then you have to compile it:

tar xvfz cmon-1.0.tar.gz
cd cmon-1.0
sh ./autogen.sh
./configure
make
make install ## -- optional step - will install into /usr/local/bin as default and you need to be root


The easiest and recommended way is to have a mysql server and a management server on the computer where you are installing cmon. This makes things simpler.

Now you can start cmon which will daemonize itself (start with --nodaemon if you don't want this to happen):

/usr/local/bin/cmon ##if you did 'make install' above
or
src/cmon ##if you didn't 'make install' above

This corresponds to starting cmon with the following arguments:

cmon --ndb-connectstring=localhost \
--mysqlhost=localhost \
--mysqluser=root \
--mysqlpasswd="" \
--mysqlsocket="/tmp/mysql.sock"\
--savetime-clusterlog=24 \
--savetime-statistics=24

cmon will:

• automatically create a database called 'cmon' on the localhost the necessary tables.
• log to syslog (tail -f /var/log/message )

If you start up cmon and tail /var/log/message you will get:

Sep 16 23:14:09 ps-ndb01 cmon: Starting cmon with the following parameters: --mysqldatabase = cmon --mysqluser = root --mysqlpasswd = --mysqlhost = localhost --mysqlport = 3306 --mysqlsocket = /tmp/mysql.sock --ndb-connectstring = localhost --savetime-clusterlog = 24 hours --savetime-statistics = 24 hours
Sep 16 23:14:09 ps-ndb01 cmon: If that doesn't look correct, kill cmon and restart with -? for help on the parameters, or change the params in /etc/init.d/cmon
Sep 16 23:14:09 ps-ndb01 cmon: Creating pid file /tmp/cmon.pid
Sep 16 23:14:09 ps-ndb01 cmon: Creating database 'cmon' if it does not exist
Sep 16 23:14:09 ps-ndb01 cmon: Table 'backup' did not exist. Schema incomplete. Going to recreate missing tables.
Sep 16 23:14:09 ps-ndb01 cmon: Recreated missing tables
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'backup'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'backup_log'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'restore'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'restore_log'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'cluster_log'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'cluster_state'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'memory_usage'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'node_state'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'cluster_statistics'
Sep 16 23:14:09 ps-ndb01 cmon: Created table 'node_statistics'
Sep 16 23:14:09 ps-ndb01 cmon: Done
Sep 16 23:14:09 ps-ndb01 cmon: Setting up threads to handle monitoring
Sep 16 23:14:09 ps-ndb01 cmon: Starting MonitorThread
Sep 16 23:14:09 ps-ndb01 cmon: Starting LogEventThread
Sep 16 23:14:09 ps-ndb01 cmon: connected to mysqld
Sep 16 23:14:09 ps-ndb01 cmon: connected to ndb_mgmd on host localhost, port 1186


Using the PHP scripts that are located in the 'www' directory and a webserver you can get nice stats about Cluster. Such as:

..and per node statistics:

..and memory usage:

... and detailed backup and restore statistics:
You are actually getting every stat you can get from MySQL Cluster at this stage. And if you don't want to use the PHP scripts, you can access the same information using plain SQL, since cmon logs everything to normal SQL tables.

And yes, I have only tested this on Linux deployed on a cluster consisting of HP DL385 systems with x86-64 architecture (in this case dual cpu dual core Intel(R) Xeon(R) CPU 5160 @ 3.00GHz and dual cpu quad core Intel(R) Xeon(R) CPU E5345 @ 2.33GHz ). They are a bit old, but good.

Let me know if you have problems or questions on the above:
johan at severalnines dot com