As part of the work on system monitoring ... / 2010-08-10

As part of the work on system monitoring I am looking into monitoring RAID units. The beta-ict department uses a number of raid units and data gets replicated between buildings.

I want a warning when a disk goes down. The 3ware disk controller has a nice webinterface but I can't integrate that (easily..) into zabbix. What I did was install the tw_cli command line utility from the 3ware LSI raid controller site (lookup your type of controller, find 'support and downloads' and you will see cli utils for lots of unix versions), which makes life easy:

# tw_cli show

Ctl   Model        (V)Ports  Drives   Units   NotOpt  RRate   VRate  BBU
------------------------------------------------------------------------
c0    9650SE-16ML  16        15       1       1       1       1      OK

What I want to know is the number of not-optimal disks (yes, indeed one is broken at the moment and needs replacement). That I can monitor in zabbix, when I pick up the value with a script:

#!/bin/sh
# /etc/zabbix/external/3ware.okdisk

sudo /usr/local/sbin/tw_cli show | awk ' /^c0/ { print $6 } '

Root access via sudo which means a line in /etc/sudoers which allows /usr/local/sbin/tw_cli from the zabbix user, and the right setting in zabbix_agentd.conf to bind this script to a user parameter:

UserParameter=3ware.okdisk,/etc/zabbix/external/3ware.okdisk

Now I can program a trigger on the output: 0 is ok, 1 is warning, > 1 is disaster. I added an extra action on the trigger to mail the output of tw_cli '/c0 show' to the admins so we know which disk is broken.

Now to do the same for adaptec (aacraid) based raids.
Update 2011-04-15: We now have a raid unit with a b0rken disk but still in 'optimal' state strangely enough. Updated the script to test for disks reporting something else than 'OK'.