System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Remounting filesystem read-only

José M. Navarro Mesegue
Occasional Visitor

Remounting filesystem read-only

A filesystem connected to san is remounted like read-only.

Sep 26 04:02:24 mx2 init: Trying to re-exec init
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1): ext3_free_blocks_sb: bit already cleared for block 32823655
Sep 26 07:57:46 mx2 kernel: Aborting journal on device sda1.
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_truncate: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_orphan_del: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in ext3_delete_inode: Journal has aborted
Sep 26 07:57:46 mx2 kernel: __journal_remove_journal_head: freeing b_committed_data
Sep 26 07:57:46 mx2 last message repeated 2 times
Sep 26 07:57:46 mx2 kernel: ext3_abort called.
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1): ext3_journal_start_sb: Detected aborted journal
Sep 26 07:57:46 mx2 kernel: Remounting filesystem read-only
Sep 26 07:57:46 mx2 kernel: EXT3-fs error (device sda1) in start_transaction: Journal has aborted

- Server Blade HP BL20p
- Kerneel: Linux 2.6.9-78.0.1.ELsmp #1 SMP Tue Jul 22 18:11:48 EDT 2008 i686 i686 i386 GNU/Linux
- SAN HP EVA3000
8 REPLIES
Sandeep_Chaudhary
Trusted Contributor

Re: Remounting filesystem read-only

Please try the steps mentioned in the link . If it doesnt help reboot will solve this problem
Sandeep_Chaudhary
Trusted Contributor

Re: Remounting filesystem read-only

Ivan Krastev
Honored Contributor

Re: Remounting filesystem read-only

Check from SAN side for any problems. Try to unmount affected filesystems and run fsck to check and reply the ext3 jurnal. After that mount it again.

regards,
ivan
José M. Navarro Mesegue
Occasional Visitor

Re: Remounting filesystem read-only

Hi everybody. (Sorry for my English level)

Thanks everybody for the answers, but what I wanted to know was if anyone had resolved a similar issue and what it had been the cause of the problem.

I know I have to umount, run fsck and then mount again to get the filesystem in rw mode, but I am afraid to get the same error again (it is the second time this week)

I have checked my hardware and no errors or "red leds" have been found.

Any idea about the cause of the problem?

Thank you

Best regards,

Jose
Mike Stroyan
Honored Contributor

Re: Remounting filesystem read-only

My first guess would be that two different systems have tried to simultaneously mount the same SAN partition. That could produce that kind of error message. The kernel is complaining that completing the journal is finding file system changes already made that should only be done when applying the journal.
The second system to mount a partition would be correct to detect and complain about that.

A slight variation on that would be the same system mounting the SAN partition twice through what looks like two different paths.
John Donovan
Regular Advisor

Re: Remounting filesystem read-only

The only time we seen this is just like Mike described. We'd failed to umount the fs from the old server before mounting to the new server.

It was a ServiceGuard environment.

Hope this helps...
:-)
--donovan
"I have not failed. I've just found 10,000 ways that won't work." - Thomas Edison
Uwe Zessin
Honored Contributor

Re: Remounting filesystem read-only

It is possible that you have been hit by a kernel bug which is triggered by a long SCSI/FC delay. The result is a readonly FS.

http://kb.vmware.com/kb/1001778
the page says that the problem also occurs on physical systems.
.
José M. Navarro Mesegue
Occasional Visitor

Re: Remounting filesystem read-only

Hi everybody.

Again, thank you very much for all your answers and for your interest about my problem. It is very helpful for me.

The partition is mounted on two servers, but not simultaneously, of course :-)
When the partition is umounted on the first server, there are no error messages. Then, the partition is mounted on the second server, and there are no error messages either. Sometime later, the problem arises.
Idea: I will always run fsck on the partition before mounting in the second server. If the problem persists, I think that there might be a bug in my kernel (as it is pointed by Uwe Zessin)

Best regards,

Jose