Operating System - Linux
1748176 Members
4224 Online
108758 Solutions
New Discussion юеВ

Re: Stuck in runlevel 3 with / as read only

 
SOLVED
Go to solution
Oscar Garcia
Regular Advisor

Stuck in runlevel 3 with / as read only

Hi Guys,

I am running Suse SLES 8 in an HP proliant DL385 and after trying to configure nfs and samba I ended up with an unusable machine.
Basically all the filesystems are readonly. I tried to mount / in single user mode, but it comes with:
mount: can't find / in /etc/fstab or /etc/mntab

Any ideas how to make it available? Any thing I tried to do just tells me is read-only...

Thanks in advance.
11 REPLIES 11
xyko_1
Esteemed Contributor

Re: Stuck in runlevel 3 with / as read only

Hi Oscar,

would you please attach boot.log, boot.msg and messages from /var/log ?

Regards,
Xyko
Oscar Garcia
Regular Advisor

Re: Stuck in runlevel 3 with / as read only

Thanks xyko,

I cannot telnet or ssh to that server so getting those files is going to be hard...
I will try to post what seems the mos relevant entries.

Ta
Oscar Garcia
Regular Advisor

Re: Stuck in runlevel 3 with / as read only

Well to make things worst, there is not /var/log only /var/lib and /var/run...

The only filesystems that appear in the fstab are nfs ones.

I think is important to mention that I am using LVM and I have defined only one vg00, where everythings hangs of except /boot and swap.
One of the first errors I notice is something like:
Can't fsck because root is not read-only

and these are the last lines of the boot:
Failed services in runlevel 5: random hotplug sshd nfs xdm
Skipped services in run level 5: isdn nmb sshd cups hwscon postfix smb atd cron nscd inetd
xyko_1
Esteemed Contributor

Re: Stuck in runlevel 3 with / as read only

Oscar,

show us the result of the command

#mount


Xyko
Stuart Browne
Honored Contributor

Re: Stuck in runlevel 3 with / as read only

Yeah, the major issue is that you've fried your '/etc/fstab'.

There should be (at a minimum) a '/' and 'swap' entry in there for local devices.

You've also mentioned that '/boot' is missing.

Now, assuming you've got all your LVM modules loaded, and the machine does a 'vgscan' (and 'vgchange -a y')on boot-up, you can probably try the following:

mount LABEL=/boot /boot

(as a test). Most modern distro's "LABEL" the filesystems so you can use this method.

If that works, then add entries similar to:

LABEL=/ / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2

to your '/etc/fstab', and reboot.

Going back a bit, to explain why you're seeing what you are seeing...

As the system boots (of an initrd etc.), it creates a fake-root of the initrd's details. It then 'pivot_root's to your real '/' device, and continues the booting from there. This then uses your '/etc/fstab' to correctly mount the remainder of your filesystems, usually in '/etc/rc.d/rc.sysinit', including the re-mounting of root (with a command similar to "mount -n -o remount,rw /") into read-write mode. As '/etc/fstab' doesn't exist, the short-hand method of mount (using only the existing mount-point) doesn't work, as there's no map-entry in the fstab.

Unf... yay...

Fix fstab, reboot, all should be well.
One long-haired git at your service...
Matti_Kurkela
Honored Contributor

Re: Stuck in runlevel 3 with / as read only

Note that the list of mounted disks provided by "mount" command comes from /etc/mnttab, which might have stale data if the root filesystem is in read-only state.

In a situation like this, you should check /proc/mounts. It may not have all the details /etc/mnttab has, but if /proc is properly mounted by the startup scripts, the data will be current (/proc is a pseudo-filesystem; its data is generated from the kernel memory structures on demand).

To make the root filesystem writeable, you should run "mount -o remount,rw /". This is often the first command you'll need to use when booting to the single user mode, so it's good to remember it.
MK
Oscar Garcia
Regular Advisor

Re: Stuck in runlevel 3 with / as read only

Thanks guys for your replies.

The result of mount is:
proc on /proc type proc(rw)

The result of mount LABEL=/boot /boot
mount:no such partition

and this is what I have in /proc/mounts :
rootfs / rootfs rw 0 0
/dev/root / reiserfs ro 0 0
proc /proc proc rw 0 0
devpts /dev/pts devpts rw 0 0
shmfs /dev/shm shm rw 0 0
lindev7:/my_remote_dir /my_local nfs rw, v2, reserved,hard,udp,lock, addr=... 0 0

I have changed to init 1 and when trying to mount -o remount,rw / it comes back with this:
mount: can't find / in /etc/fstab or /etc/mtab

So it seems that the only hope is to use /proc, but I do not know what to put there...

Regards,
Matti_Kurkela
Honored Contributor
Solution

Re: Stuck in runlevel 3 with / as read only

OK, so you'll need to figure out your disk structure first...

Begin by checking out /proc/partitions: it's a list of all disk partitions the system currently has detected. Do you see any "dm-" -style entries? If you do, LVM might already be active. If not, you must activate it manually.

As you have a LVM-based system, a "vgscan -v" might help to understand the current situation. Then activate all available volume groups with "vgchange -a y". If the LVM device nodes are missing, "vgscan --mknodes" should fix that. (None of these commands should do any harm, even if they aren't needed.)

You currently seem to have both /etc/fstab and /etc/mtab in unusable state. First you need to try to reconstruct the /etc/fstab, I think.

Have you already checked whether there is a backup version of /etc/fstab still available in the system? (ls -l /etc/fstab*)
MK
Oscar Garcia
Regular Advisor

Re: Stuck in runlevel 3 with / as read only

Matti,

I cannot see any "dm-" -style entry. However there is a list of what seems to be the partition map. It has 15 columns and start with "major minor #blocks name ..."
the first entry is
58 0 1048576 lvma and 11 columns with 0

the last 4 entries show what it seems to be the partitions for c0d0, c0d0p1 /boot, c0d0p2 swap and c0d0p3 lvm.

I ran vgchange -ay with "vgchange -- volume group "vg00" already active" as result.

I also found a /etc/fstab.orig file. I think there are a couple of filesystems missing, but I can afford to loose them.

Ta