[PLUG] Booting/disk "problem"?

["eric@lucii.org" <eric@lucii.org>]

I've been called in to do a sort of forensic analysis on a Linux server
that won't boot (oh, they want me to fix it and make it work again too
<grin>.)

The machine is a Compaq server with a RAID array running Red Hat 8.0.  It
refuses to boot citing: "Kernel panic: no init found".  I also see this
error: pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2

I booted it with the CD ROM and it STILL won't run on the existing 
partitions.  I got into a shell and went mucking about.  Here's what
I found with various tools like fdisk, e2label, and fsck:

  device              label    note
  /dev/cciss/c0d0p1   /boot    appears fine
  /dev/cciss/c0d0p2   /usr     appears fine
  /dev/cciss/c0d0p5   /home    appears fine
  /dev/cciss/c0d0p7   /var     appears fine
  /dev/cciss/c0d0p6   /        Problem: -->

     On the p6 partition, the is only: /bin, /boot, /home, /proc,
     /usr and /var Since boot, home, usr, and var are mount points, they
     are empty.  There are a number of files in the bin directory
     including one called "all.tar" which is 122 MB and is truncated.
     The tar file was created about the last time that the machine was
     known to be working.  

    df -h shows:
   
      Size     Used    Available  Use%
      505.9M   505.9M     0       100%  

Also, the UPS went down at some point and may have just taken the server
down. 

Given this limited set of evidence, can anybody come up with a plausible
explanation for what happened? 

I theorize that the partition was too full for the user to build their
all.tar file so they tried to perform a /bin/rm command but executed it
in the wrong directory.  They were logged in as root :-(   By the time
they realized it, it was too late.  

The flaw in this is that it would have deleted the contents (or SOME OF
the contents) of the mounted partitions and that does not seem to have
happened.  I cannot envision a tar process that selectively deletes
files to make room for itself :-P but then I've not seen everything.


Thanks,
Eric

-- 
#  Eric Lucas
#     "Oh, I have slipped the surly bond of earth
#      And danced the skies on laughter-silvered wings...
#                             -- John Gillespie Magee Jr.
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug