rc.sysinit segmentation fault

[otukile]

Morning. We have a client who has Redhat AS4 Update 5 installed on DL380 G4 server. When booting we get error similar to the below:

Welcome to /etc/rc.d/rc.synsinit: line 70: 35 Segmentation fault LC_ALL=C grep -q "Red Hat" /etc/redhat-release Red Hat Linux
Press 'I' to enter interactive startup.
Mounting proc filesystem: [failed]
/etc/rc.d/rc.sysinit: line 90: 42 Segmantation fault LC_ALL=C grep -q .initrd /proc/mounts
Confuguring kernel parameters: Error: /proc must ve mounted
To mount /proc at boot you need an /etcfstab line like: /proc /proc proc defaults
In the meantime, mount /proc /proc -t proc [failed]
Setting clock (localtime): Fi Jan 23 09:06:35 MST 2004 [ ok ]
Loading default keymap (us): [ ok ]
Setting hosname dbcluster01: [FAILED ]
Cat /proc/cmdline: No such file or directory



We have tried replacing memory. Ran fsck from linux rescue mode. We cannot afford to lose data in the system, thats why we dont want to do a fresh installation. Has anybody come across the same and how did they resolve problem?
Thanks in advance

[Matti_Kurkela]

Maybe the system board has failed, causing random command failures because of data corruption in system memory.

Or maybe the OS on the disks has been corrupted so far that the OS installation is no longer viable.

Your *first* objective should be a full backup of your application data. Any attempts to fix the problem must come after that: it is possible that a failed "fix" will make the problem worse.

You'll need to boot from some other media and mount the old disks as "data disks". This would allow you to backup the data.

You could use a Live-CD style Linux distribution, like Knoppix. That would allow you to boot from CD/DVD and still have networking features.

Does the server have any free disk slots? If so, you could add a new disk and install a new OS _on that disk_. If you do this, you must be very careful running the installer, and make sure you don't select any options that would erase the data on the original disks.
This would allow running comparisions between any damaged files on the old disks and a known-good installation of the same OS version on the new disk.

By the way, this is *exactly* the reason why server systems should have separate filesystem(s) dedicated to applications and their data. The OS installer will detect existing filesystems: you could then choose to overwrite only the root, /boot, /usr, /var and /tmp filesystems, leaving any application and data filesystems untouched.

After the installation completes, you could then manually mount the application and data filesystems and be essentially ready to start the application. (Of course, you would still have to restore any custom init scripts and the like... but at that point, the worst of the crisis will be over.)

MK