System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Online resize2fs: Operation not permitted While trying to add group #14080

SOLVED
Go to solution
Liam Curtis
Frequent Advisor

Online resize2fs: Operation not permitted While trying to add group #14080

Hello All,

I am running Oracle Enterprise Linux x64 5.5 on an HP Proliant DL380 G6 attached to various SANs via fibre channel. I have set up multipath, edited multipath.conf, lvm.conf, and after some trial and error, everything behaves and looks as it should in regard to multipath.

However, when I try to resize a multipath ext3 lvm filesystem, The operation runs for a while, but then dies and I get the above error. I then see this error in /var/log/messages:

EXT3-fs warning (device dm-9): ext3_group_add: No reserved GDT blocks, can't resize

-I am only using the /dev/mapper/ paths to the filesystem (ie, resize2fs /dev/mapper/testvol-lvol1 128G)
-if I unmount the filesystem, resize2fs works fine
-Have tried removing and re-adding journal file via tune2fs with no luck
-The VG has plenty of room in it and is composed of a 100GB and 200GB partition.
-Inode resize is on in the filesystem

I am wondering if online resize2fs is only for small filesystems or not recommended as a best practice or just not generally done??? As I mentioned, the operation works fine offline if I unmount the filesystem, but this won't be an option once system is in production.

Any ideas? Thanks in advance!
14 REPLIES
J. Maestre
Honored Contributor
Solution

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Do you see reserved GDT blocks if you run dumpe2fs on the device?
Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Thanks for the reply. When I create the filesystem, the reserved GDT blocks are at 127. After the error, there are none and even though resize_inode is part of system features, there is no longer even an entry for reserved GDT blocks. Is there a way that I can increase the reserved GDT blocks for the filesystem?
J. Maestre
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

127 reserved GDT blocks should allow your filesystem to grow to roughly 127 times it's initial size. Does your resize operation fit there?


I don't think you can change the number of reserved GDT blocks, you would have to recreate the filesystem and provide a "-E resize=" value.

Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

127 reserved GDT blocks should allow your filesystem to grow to roughly 127 times it's initial size. Does your resize operation fit there?

I don't think you can change the number of reserved GDT blocks, you would have to recreate the filesystem and provide a "-E resize=" value.

I have tried various sizes with same results. In this case I started with a very small size of a few MB and grew it out to 50GB okay, but then it blew up when I tried to resize further. I will try some different initial sizes.

I guess I was certainly spoiled with HP-UX and onlineJFS/vxfs!
J. Maestre
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Also check the filesystem journal size, just in case. A very small journal will also cause online resize to fail.

If that's the case, you can use tune2fs to delete the journal and recreate it. It should automatically assign a proper size (likely ~128MB).
Steven E. Protter
Exalted Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Shalom,

My experience shows resize2fs does work exactly like HP-UX Online JFS

I think the tunefs (you can tune a fs but not a fish), will actually resolve this problem.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Shalom,

My experience shows resize2fs does work exactly like HP-UX Online JFS

I think the tunefs (you can tune a fs but not a fish), will actually resolve this problem.

Shalom, Steven ;)

I find that with OnlineJFS, resizing is pretty much instantaneous, whereas with Linux and resize2fs, the resize operation takes quite a while--several minutes or more until it bombs in my case.

I had tried deleting and recreating journal, but that did not help. It seems that once GDT blocks are exhausted, that is that as far as online resizing. I must then do an fsck.

Am I perhaps missing something?
Matti_Kurkela
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Back in RHEL 4 or so, when there was a specific "ext2online" tool for on-line filesystem extension, there was also a "ext2prepare" tool for increasing the GDT block reserve. Using "ext2prepare" was always an off-line operation, because it needed to move things around inside the filesystem to get more contiguous space for future GDT blocks.

Later the functionality of "ext2online" was integrated to "resize2fs", and currently "ext2prepare" is no longer updated. Perhaps the functionality of "ext2prepare" was also integrated to resize2fs, so that it'll automatically replenish the GDT block reserve whenever you use it to extend the filesystem in off-line mode? (This is just a guess; I haven't tested it. I tried looking at the resize2fs source but it was too complicated for me at the moment (perhaps not enough caffeine?))

When you install a system for production, your original filesystem size choices should be based on informed estimates based on expected usage and data size, not wild guesses. If you originally don't have the information to make those estimates, then that's a good reason to make a complete re-install to make the system as clean and as efficiently laid out as possible before the system goes to production.

Perhaps this would also be a good argument for keeping the testing an production systems completely separate from each other, instead of "promoting" test systems to production as-is.

But sooner or later you *will* need to do some kind of maintenance to the production system too (as nothing human-made is absolutely perfect). If the system is so critical that it absolutely cannot have even pre-scheduled maintenance breaks, then it should also be critical enough to have a totally independent failover system which can instantly take over with a minimum of fuss.

If such a failover cannot be implemented for your application, then the application is clearly unsuitable for continuous mission-critical operation; no excuses.

MK
MK
chris huys_4
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Hi Liam,

LC> I guess I was certainly spoiled with HP-LC> UX and onlineJFS/vxfs!

Yes, you were.

LC> Am I perhaps missing something?

Install (payable) vxfs instead, for the filesystem part, and (payable) vxvm, for the volumemanager part, on the OEL system, and use that, problem solved. ;)

Greetz,
Chris
Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Thanks for the replies everyone. I am kinda disappointed about lackluster online resizing with out of box Linux. Funny how my dirt old version of HP-UX11i still blows away latest Linux in many regards...shame that HP has embraced Itanium so much, but that's just my humble opinion.
Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Before I give up, has anyone out there managed to grow filesystems online by 2-300GB or more within Linux?

For this particular project, I think I am going to just allocate 2-3 years of disk space, but I have been faced with instances of 'we need another 500GB NOW' where OnlineJFS has saved me in a matter of minutes without even a second thought of taking filesystem offline...
John Poff
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

I use resize2fs to grow mounted filesystems all the time with no problems, but I don't use it the way you do. I always do an lvextend on the logical volume first, and then use resize2fs to grow the filesystem.

JP
John Poff
Honored Contributor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

And the last huge filesystem I grew was a few months ago. We had a 1 Tb filesystem and I grew it by an additional 1.5 Tb. It took a few minutes to churn through it but it worked (online) just fine.

JP
Liam Curtis
Frequent Advisor

Re: Online resize2fs: Operation not permitted While trying to add group #14080

Interesting...what the heck could i be doing wrong then...

Could you list the steps you did to create filesystem and extend it? wondering if i am missing something obvious...