Josh Olson’s Blog

A software raid group disk failed in one of my servers yesterday.

The kernel was spewing SCSI errors:

kernel: ata2: status=0xd0 { Busy }
kernel: SCSI error : return code = 0×8000002

# mdadm --display /dev/md0
# mdadm --display /dev/md1

both reported a failed disk sdb*

The procedure to rebuild the md groups is as follows:

Replace bad disk (sdb in this scenario.) Note that if you do not bring down the server to replace the disk, be sure to “remove” the disk from the raid groups using mdadm.

# mdadm --remove /dev/md0 /dev/sdb0
# mdadm --remove /dev/md1 /dev/sdb1

Read the good disk’s partition table (sda in this scenario.)

# fdisk -l /dev/sda
Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 fd Linux raid autodetect
/dev/sda2 14 19457 156183930 fd Linux raid autodetect

Install identical partition table on newly replaced disk. Create partitions that start and end on the same listed cylinders and are of type “fd.” Be sure to set the boot flag, and don’t forget to write the changes.

# fdisk /dev/sdb

Add partitions back to the appropriate raid groups.

# mdadm --add /dev/md0 /dev/sdb0
# mdadm --add /dev/md1 /dev/sdb1

Ensure the raid groups are rebuilding properly.

# mdadm --display /dev/md0
# mdadm --display /dev/md1

Friday night, good friends arrived with fishing gear in anticipation of our annual expedition. After a night of celebration (and not much sleep) we woke at 04:45 and drove to the Huron river in Skanee, MI. This was our first time fishing this popular area, so we were surprised to see the abundance of other fishermen. The weather was decent, but the water temperature was still too cold for the fish to “run” in full force. Although our party was fruitless, we saw five Steelhead caught over the weekend. With the rising water temperature, I expect the fish to be more prevalent by this weekend. I plan to try my luck at least once more.

This afternoon I was faced with searching a directory tree for large files that have rotated within the last 24 hours - a symptom of a problem we were experiencing with a service.

Here’s what I put together quickly:

# find -iname name-\*.log -mtime 0 -exec du -sh {} \;

Explanation of the switches (from the find man page):

-iname pattern
Base of file name (the path with the leading directories removed) matches case insensitive shell pattern pattern

-mtime n
data was last modified n*24 hours ago.

-exec command {} \;
run the specified command on the matched files


It’s not complex (and probably not post-worthy,) but someone may find it helpful.

In preparation for the bicycle purchase, I ordered a SIGMA BC 1606L DTS bike computer. I haven’t opened it yet, but it looks quite neat. I look forward to obsessing over my speed and distance, and will probably buy the add-on cadence monitor. No bike-shopping updates yet, I’ll be test-riding James’s 2006 Trek 1500 tomorrow after work.

Never have I felt the need to wear a helmet while riding on asphalt - until today. During lunch I experienced my first trip on a cyclocross bicycle, the Kona Jake specifically. I’ve been shopping for a new bicycle for almost a month and am now enticed by ‘cross bikes. The Jake was fast, responsive, and I felt comfortable riding it. I can’t wait to test the 2008 model. Cross country sports should have one built by the end of the month.

I’m open to suggestions.