Software Raid issue

[Shannon Petry]

I recently had to rebuild a system. In the process I tar'd up 2 20GB files to my RH8.0 server onto a 2x64gb disk stripe of 128GB.

After deleting these files, I am now not able to add any data to the stripe. I get an error that the disk is full, even though it is only 6GB used out of 130GB. THis was verified with both du -sk, and df -k.

Since these were plain files (tar archives) I know that nothing would have had them open. Just to be sure, I rebooted the Linux server, and still can not create data in the stripe with the error "no space left on device"

[root@masterc 1b]# df -k .
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/md1 143347736 5902848 130276160 5% /1b

[root@masterc 1b]# touch test
touch: creating `test': No space left on device

I have looked with lsof, and see no open files on the device.



Any ideas?

Thanks!
Shannon

[Sachin Patel]

Hi Shanon,

Check for quota
#repquota

If you don't want it just turn it off
#quotaoff


how is your raid configuration look like

#lsraid

[Steven E. Protter]

Greetings Shannon,


Here is my output on a working raid 1/0 system.

[root@jerusalem root]# lsraid -A -g -a /dev/md0
[dev 9, 0] /dev/md0 E21817F0.29578AF2.8ED33C38.7B44A0A9 online
[dev 3, 2] /dev/hda2 E21817F0.29578AF2.8ED33C38.7B44A0A9 good
[dev 3, 66] /dev/hdb2 E21817F0.29578AF2.8ED33C38.7B44A0A9 good


Actual use of the lsraid command.

In your context I'd change it to.

lsraid -A -g -a /dev/md1

Hope this helps.

SEP

[Steven E. Protter]

Shannon,

Would you be so kind as to post your results from the command. Just curious as to what can cause your results.

Thanks.

Steve

[Shannon Petry]

quota's are not enabled

There is no such tool as lsraid on Redhat 8, I'll have to check 9. But here is the contents of /etc/raidtab.

raiddev /dev/md0
raid-level 0
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/sdc
raid-disk 0
device /dev/sdd
raid-disk 1

raiddev /dev/md1
raid-level 0
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/sda
raid-disk 0
device /dev/sdb
raid-disk 1

There is no issue that I can see with the raid device, the drivers are loaded, it's on line.

Regards,
Shannon

[Steven E. Protter]

Here is my /etc/raidtab for comparison's sake.

[root@jerusalem root]# more /etc/raidtab
raiddev /dev/md7
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda1
raid-disk 0
device /dev/hdb1
raid-disk 1
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda5
raid-disk 0
device /dev/hdb5
raid-disk 1
raiddev /dev/md0
raid-level 1
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda2
raid-disk 0
device /dev/hdb2
raid-disk 1
raiddev /dev/md4
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda7
raid-disk 0
device /dev/hdb7
raid-disk 1
raiddev /dev/md6
raid-level 1
nr-raid-disks 2
chunk-size 64k
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda8
raid-disk 0
device /dev/hdb8
raid-disk 1
raiddev /dev/md1
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda3
raid-disk 0
device /dev/hdb3
raid-disk 1
raiddev /dev/md3
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
raid-disk 0
device /dev/hdb3
raid-disk 1
raiddev /dev/md3
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda6
raid-disk 0
device /dev/hdb6
raid-disk 1
raiddev /dev/md5
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda9
raid-disk 0
device /dev/hdb9
raid-disk 1



Are you sure about the lsraid command? I can find no desupport notice, though my resuls are admittedly off a red hat 7.3 server.

[root@jerusalem root]# find / -name lsraid
/sbin/lsraid
[root@jerusalem root]# ll /sbin/lsraid
-rwxr-xr-x 1 root root 67272 Apr 15 2002 /sbin/lsraid

Note that its not on the PATH of non-root users.

SEP

[Steven E. Protter]

Red hat search on lsraid.

http://www.redhat.com/whitepapers/rhel/OracleonLinux.pdf

I'm not running oracle on this server, wonder where the darned thing came from.

Will do further research and try and find you an alternate command for checking raid status.

My guess is based on data, the response you posted and prior fun with Red Hat Raid is that there is a problem with raid.

Alternate solution: use fdisk, cfdisk or the gui graphical tool to blow away and re-create the filesystem, see what happens.

SEP

[Sachin Patel]

Hi Shannon,

There is a command lsraid in 8.0.

Try raidstart. what is /proc/mdstat?

Is raidtool rpm install?

Sachin

[Steven E. Protter]

Obviously the lsraid command is not installed on Shannon's system.

Here is where to get it.

http://rpmfind.net/linux/RPM/redhat/8.0/i386/raidtools-1.00.2-3.3.i386.html

Download it, install it, run the command and post the output, it should lead us to a cause.

A pleasure working with you. The answer would have been faster, but itrc was a dog today.

For you Shannon, I'd be happy to arrange a phone call to work this through(your dime)

stevenprotter@juf.org

SEP

[Steven E. Protter]

Sorry, these tools are probably also on your CD's for red hat. That's where mine came from.

SEP

:-)