topic Re: Remounting filesystem read-only in Operating System - Linux

Remounting filesystem read-only

José M. Navarro Mesegue — Fri, 26 Sep 2008 09:00:52 GMT

A filesystem connected to san is remounted like read-only.

Sep 26 04:02:24 mx2 init: Trying to re-exec init
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1): ext3_free_blocks_sb: bit already cleared for block 32823655
Sep 26 07:57:46 mx2 kernel: Aborting journal on device sda1.
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_truncate: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_orphan_del: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_delete_inode: Journal has aborted
Sep 26 07:57:46 mx2 kernel: __journal_remove_journal_head: freeing b_committed_data
Sep 26 07:57:46 mx2 last message repeated 2 times
Sep 26 07:57:46 mx2 kernel: ext3_abort called.
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1): ext3_journal_start_sb: Detected aborted journal
Sep 26 07:57:46 mx2 kernel: Remounting filesystem read-only
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in start_transaction: Journal has aborted

- Server Blade HP BL20p
- Kerneel: Linux 2.6.9-78.0.1.ELsmp #1 SMP Tue Jul 22 18:11:48 EDT 2008 i686 i686 i386 GNU/Linux
- SAN HP EVA3000

Re: Remounting filesystem read-only

Sandeep_Chaudhary — Fri, 26 Sep 2008 09:30:20 GMT

Please try the steps mentioned in the link . If it doesnt help reboot will solve this problem

Re: Remounting filesystem read-only

Sandeep_Chaudhary — Fri, 26 Sep 2008 09:31:42 GMT

http://answers.google.com/answers/threadview?id=452563

Re: Remounting filesystem read-only

Ivan Krastev — Fri, 26 Sep 2008 10:10:06 GMT

Check from SAN side for any problems. Try to unmount affected filesystems and run fsck to check and reply the ext3 jurnal. After that mount it again.

regards,
ivan

Re: Remounting filesystem read-only

José M. Navarro Mesegue — Fri, 26 Sep 2008 11:08:33 GMT

Hi everybody. (Sorry for my English level)

Thanks everybody for the answers, but what I wanted to know was if anyone had resolved a similar issue and what it had been the cause of the problem.

I know I have to umount, run fsck and then mount again to get the filesystem in rw mode, but I am afraid to get the same error again (it is the second time this week)

I have checked my hardware and no errors or "red leds" have been found.

Any idea about the cause of the problem?

Thank you

Best regards,

Jose

Re: Remounting filesystem read-only

Mike Stroyan — Tue, 30 Sep 2008 15:02:08 GMT

My first guess would be that two different systems have tried to simultaneously mount the same SAN partition. That could produce that kind of error message. The kernel is complaining that completing the journal is finding file system changes already made that should only be done when applying the journal.
The second system to mount a partition would be correct to detect and complain about that.

A slight variation on that would be the same system mounting the SAN partition twice through what looks like two different paths.

Re: Remounting filesystem read-only

John Donovan — Tue, 30 Sep 2008 16:04:04 GMT

The only time we seen this is just like Mike described. We'd failed to umount the fs from the old server before mounting to the new server.

It was a ServiceGuard environment.

Hope this helps...
:-)
--donovan

Re: Remounting filesystem read-only

Uwe Zessin — Tue, 30 Sep 2008 17:09:17 GMT

It is possible that you have been hit by a kernel bug which is triggered by a long SCSI/FC delay. The result is a readonly FS.

http://kb.vmware.com/kb/1001778
the page says that the problem also occurs on physical systems.

Re: Remounting filesystem read-only

José M. Navarro Mesegue — Wed, 01 Oct 2008 14:18:38 GMT

Hi everybody.

Again, thank you very much for all your answers and for your interest about my problem. It is very helpful for me.

The partition is mounted on two servers, but not simultaneously, of course :-)
When the partition is umounted on the first server, there are no error messages. Then, the partition is mounted on the second server, and there are no error messages either. Sometime later, the problem arises.
Idea: I will always run fsck on the partition before mounting in the second server. If the problem persists, I think that there might be a bug in my kernel (as it is pointed by Uwe Zessin)

Best regards,

Jose