Breaking News: The Shitty Old IDE Drive Fails!

The 160GB 7200RPM Hitatchi Deskstar IDE hard drive I’ve been using in my server for the past five years has failed.  This is the second drive failure this server has had.  The first indication of trouble was my mom wondering why she had read-only access to the Samba shares on the server.  The second indication of trouble was my mom’s machine refusing to log in since the profile was located on one of those Samba shares.  The third indication of trouble was the HDD activity light lit steady on the server.  I really wish I had a hotswap canister so I could identify a failed drive more easily.

Powering on the console monitor confirmed that /dev/hdc was no more – a million I/O errors told me that.  Things were slow as molasses but I knew that’s because it was still trying to swap to that drive since swap is outside of RAID.  After a swapoff /dev/hdc1 and a restart of Samba, vsftpd, sendmail, spamd, dovecot, and innd, I was back in business.  The nice part is that mysqld and httpd never even blinked.  The array is coping with the loss just fine:

What a degraded RAID-5 looks like in Webmin

What a degraded RAID-5 looks like in Webmin

What a failed drive looks like in /proc/mdstat

What a failed drive looks like in /proc/mdstat

I don’t know if I have a spare 160GB IDE drive available.  I’ve got stacks of brand new 160GB SATA drives.  I plan on looking for a suitable drive tomorrow and if I can’t find it, I’ll order it from TigerDirect.  If I have a suitable drive, I will install it during Sunday evening’s scheduled downtime.  If not, then I’ll postpone the downtime until the new drive gets here.

Advertisements

,

  1. #1 by Joshua on September 23, 2010 - 7:51 PM

    LOL – around 75 minutes after the failure, I just now got the text message on my cell phone. I guess Cingular’s SMTP-SMS gateway is a bit laggy.

  2. #2 by Joshua on September 24, 2010 - 3:01 PM

    UPDATE: I don’t have a spare 160GB IDE drive. I have a 120GB IDE drive but that’s 20GB short. What I do have is a PCI SATA-150 controller card and two spare 160GB SATA drives. So that’s the new plan, power down the machine, pop out the failed drive, pop in the SATA controller, connect the new drive to it, then power back up and make sure it rebuilds the array on the new drive.

  3. #3 by Phillip on September 26, 2010 - 10:30 PM

    AT&T has been really wonky over here. With my phone I’ve had it do it’s alert noise to let me know that I have a voicemail and a text. This particular time, I called in to listen to my voicemail and it went through, I tried calling the person back and it didn’t work, then I couldn’t send a text message either. I’m half blaming that on my phone too though, since it’ll send and call when I reboot it.

    And on kind of a related note, my brother built his first PC and was ready to throw out all the packaging material. I told him that was fine, but he should REALLY keep the stuff for a HDD in case it goes south. So he boots up the computer, installs Windows, gets done installing everything, when all of a sudden it starts going south on him. It’s guaranteed here that a HDD will go south.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: