News items for tag computersarebetterat - Koos van den Hout

I walked in this morning at work with some people looking at me expectingly. About the third person was nice enough to explain: home directories and mail were unavailable. A quick look showed me that the home directory server was waiting for the ldap server and the ldap server showed a kernel panic on the console. Strangely enough the root ldap object was still available so the monitoring system did not notice it.
Anyway, server systems should not wait for the systems administrator after a panic in my opinion, they should be available. So I looked it up, and indeed: Linux Kernel panic reboot explains how simple it is to change this setting. So I changed all servers at work to give up after a panic and reboot. That should help availability. I'm not interested in the intimate details of a panic, I want working ldap. Yes, as several people noted to me, there are ways in which this can lead to a reboot-loop, for example when the panic is file-system related. I'll take that risk when it will 'fix' all other problems.

unix - linux - storage unixfoo is good at linux and netapp knowledge. I browsed it for a while and found lots of interesting stuff.

Handy unix utility which I had a hard time remembering today: watch. For some reason this is part of procps /proc file system utilities. I used watch to keep an eye on the number of USB storage devices seen by a computer because I was busy hooking up 28 of them at a time to 4 USB hubs and sometimes things were flaky, resulting in the famous usb 7-3.3: device not accepting address 83, error -32. The solution was to unplug and replug the USB device. Trying another hub helped too.

$ watch 'lsusb -t 2>&1| grep -c 0x090c'

This showed the number of USB storage devices (of the type I used) detected so I could plug them in and see whether detection went right.

After a bit of searching I managed to get my Dell Latitude D630 laptop to use the audio buttons in fvwm.

I did some serious web services programming (in perl) and updated the scripts powering the Friday Afternoon URL page to post new urls via Twitter on Friday. You can follow @fridayaftURL to get a weekly dose of Friday afternoon stuff from all over the web. The urls are now stored in a (postgresql) database and on Friday a script runs which searches for new urls and posts them to Twitter using the Twitter api. When urls need to be shortened it uses the ln -s_ web service to shorten them.

I tried to use the --filter option in rsync but I was a bit baffled by the syntax and the manpage is nice but I couldn't understand. I wanted certain directories completely, other directories default excluded and certain files in one directory but not all. After some trail and error and talking to the teddybear:

rsync -rvv --progress /home/koos/rsyncsource/ /home/koos/rsyncdest --filter='merge /home/koos/rsyncfilter'

And in the filter file name things to include and exclude:

+ /wel/
- /niet/
+ /random/file
- /random/*

And the result is what I want:

$ ~/bin/testrsync 
building file list ... 
[sender] showing directory wel because of pattern /wel/
[sender] hiding directory niet because of pattern /niet/
[sender] hiding file random/niet because of pattern /random/*
[sender] showing file random/file because of pattern /random/file
7 files to consider
delta-transmission disabled for local transfer or --whole-file
random/
random/file
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=4/7)
wel/
wel/file1
           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=2/7)
wel/file2
           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=1/7)
wel/file4
           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=0/7)
total: matches=0  hash_hits=0  false_alarms=0 data=0

sent 319 bytes  received 126 bytes  890.00 bytes/sec
total size is 0  speedup is 0.00

Now to do this on a filesystem with 151000 files.

It's that xsnow time of year. I wanted to compile it for our students and staff to use and found a major Makefile and a real Imakefile (remember those?):

$ wc Makefile  Imakefile 
  957  2413 26799 Makefile
    7    21   172 Imakefile

Trying to find the 'real' problem I managed to reduce all that to:

xsnow: xsnow.o toon_root.o
        gcc -o xsnow xsnow.o toon_root.o -lm -lXpm -L/usr/X11R6/lib

imake gave us somewhat overkill Makefiles...

Going old-school today: I wrote a sed script to massage grub.conf to add a windows partition on a second disk. Searching google for has this been done before yields loads of page with handholding on how to add windows by hand to a grub.conf generated by anaconda but no simple 'automated' solution. I am always in favor of letting the computer do the boring work. But a bit of thinking and testing and now sed does the job:

if [ -b /dev/sdb1 ]; then

        cp /boot/grub/grub.conf /boot/grub/grub.conf.pre

        sed -e 's/timeout=5/timeout=30/' -e '/hiddenmenu/a\
title Windows XP (Service Pack 3)\
        rootnoverify (hd1,0)\
        map (hd0) (hd1)\
        map (hd1) (hd0)\
        makeactive\
        chainloader (hd1,0)+1
' -e '/hiddenmenu/d' < /boot/grub/grub.conf.pre > /boot/grub/grub.conf
fi

Everybody knows sed -e 's/../../' but I had to look up 'insert', 'append' and 'delete'.
Update 2009-11-12: Changed insert to append because the previous version inserted windows multiple times with multiple linux kernels. Once is enough. Also moved it from the post-install instructions to the post-reboot script so linux is fully configured before windows gets booted.

Actual progress in windows printing: windows 2008 with office 2003 will give a warning message about trying to print an A4 document on a printer with Letter as default format. Fixed it by selecting the right papersize for the printer, no letter paper in this building. No PC LOAD LETTER on the printer display!

The tapedrive-with-changer on the homeserver found itself in a wedged state with at the bottom of the dmesg output:

[105715.017656] ch 0:0:1:1: Attempting to queue a TARGET RESET message
[105715.017658] CDB: 0x1b 0x20 0x0 0x0 0x2 0x0
[105715.017663] ch 0:0:1:1: Command not found
[105715.017664] aic7xxx_dev_reset returns 0x2002
[105718.936191]  target0:0:1: FAST-10 SCSI 10.0 MB/s ST (100 ns, offset 15)

And still no access to the scsi tape drive. But, there is a bigger hammer nowadays named sg_reset which can fix this:

# mt -f /dev/nst0 status
/dev/nst0: Input/output error
# sg_reset -b /dev/sg0
sg_reset: starting bus reset        
sg_reset: completed bus reset
# mt -f /dev/nst0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x25 (DDS-3).
Soft error count since last status=0
General status bits on (41010000):
 BOT ONLINE IM_REP_EN

and it's back, not needing a reboot. The list of options says it all:

Usage: sg_reset  [-b] [-d] [-h] [-V] DEVICE
  where: -b       attempt a SCSI bus reset
         -d       attempt a SCSI device reset
         -h       attempt a host adapter reset
         -V       print version string then exit

   {if no switch given then check if reset underway}
To reset use '-d' first, if that is unsuccessful, then use '-b', then '-h'

More than one visitor of my homepage saw an intricate XML parsing error and not the page you all want to see. I never saw the problem myself but my best guess sofar is that the twitter rss feed was malformed, because that is the only XML parsing happening for the page. I fetch the twitter feed automatically every 6 hours, but sometimes twitter is a bit overloaded and probably gives an internal error page (the famous fail whale) and not the valid rss feed.

Solution: Fetch the file to a temporary file, run the parser on it and when the parser does not fail, copy it to where the webserver reads it:

#!/bin/sh

wget -O wwwdata/twitter.rss.pre -o /dev/null http://twitter.com/statuses/user_timeline/19301166.rss

perl -MXML::RSS -e 'my $rss=new XML::RSS; $rss->parsefile("wwwdata/twitter.rss.pre");'

if [ "$?" = "0" ]; then
	cp wwwdata/twitter.rss.pre wwwdata/twitter.rss
fi

So, who is this nobody user anyway? Porting account-management scripts from Centos linux to ubuntu made strange problems show. A short check found interesting differences. After asking around on irc and trying some things I found quite a choice:
Centos linux:

nobody:x:99:99:Nobody:/:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin

Ubuntu linux:

nobody:x:65534:65534:nobody:/nonexistent:/bin/sh

FreeBSD 6.1:

nobody:*:65534:65534:Unprivileged user:/nonexistent:/usr/sbin/nologin

Solaris:

nobody:x:60001:60001:Nobody:/:

Interesting is that creating a file as root on a solaris nfs client will create it as 65534:0. Data Ontap:

pw_name = nobody
pw_passwd = 
pw_uid = 65535, pw_gid = 65535

Computers are better at... doing what you program to, not what you want. There is still an older computer running which once hosted my mailman mailinglists for several domains. I shut down mailman on that machine and migrated the lists. But mailman was never removed from the startup files and the machine rebooted today, resulting in three years of mailman monthly reminders mailed at once. So if you were looking why you got old mailman reminders from virtualbookcase, this is why.

From time to time it annoyed me that the double-click selection in XTerm/UXTerm on the Ubuntu desktop differed from what I was used to before: 1 click is a letter, 2 clicks is a word, 3 clicks is a line. The definition of 'word' seemed to differ a lot with mine, I want to be able to select parts separated by @ and : characters. Inspired by xterm Regex Matching for Cut Selection by Sean ReifschneiderI tried with 'regex' but that did not work as I wanted, because extending a selection fails unexpectedly in that mode. Solution: updating the definition of a 'word' by modifying the XTerm*charClass. My current settings:

XTerm*on4Clicks: line
XTerm*on3Clicks: regex [^	 \n]+
XTerm*on2Clicks: word
XTerm*charClass:

That's a tab in the regex: now I can use triple-click to select a url, e-mail address or something likewise all at once, and double-click selects a word in the way I like it. Duplicated for class UXTerm.

Stuff computers are better at: shutting down at the wanted time. Just a simple:

root@hostname:~# at 23:01
warning: commands will be executed using /bin/sh
at> shutdown -h now
at> <EOT>
job 3 at Mon Dec  1 23:01:00 2008
root@hostname:~# 

The <EOT> was were I pressed ctrl-d. You might say "why not shutdown -h 23:01 ?". Well, your users will be bothered by the announcements between now and the shutdown time.

If you like mutt you might want to have a look at my .muttrc to see how I configured it.