HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

 
SOLVED
Go to solution
JDR45
Frequent Advisor

dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

Hello,

I have a HP RP3440 running 11.11. It reboots via cron twice a week around 3am. When the system comes up I get this message in the syslog. It sends out about 15 messages, all at 03:57, and then is fine the rest of the time-

Apr 4 03:57:25 vmunix: msgcnt 9 vxfs: mesg 008: vx_direrr: vx_dirscan_2
- /var file system inode 28171 block 4520501 error 6

I ran this command-
find /var -inum 2871 -print

And the output was
/var/dt

/var/dt is an empty directory dated March 19th at 15:29

Is there a way I can fix this?

I've done some searching on mesg 008 and error 6 and found some similar messages but none quite like the one I am seeing.

Thanks!

12 REPLIES
Patrick Wallek
Honored Contributor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

This link comes from Veritas, the folks that write the VxFS file system, and seems to match your error exactly. 

https://support.symantec.com/en_US/article.TECH1539.html

Even before reading this, I would recommend a FULL fsck of that file system.

# fsck -y -o full /dev/vg00/rlvol8

(or whatever the lvol of /var is on your system). 

It may be best to boot into single-user mode to do the fsck since you really don't want the file system mounted while doing that.

 

JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

Reading the link now, thanks!

Here's how /var looks on a bdf-

/dev/vg00/lvol7    36896768 1940896 34683616    5% /var

And once I'm inside /var, the empty "dt" directory looks like this-

drwxr-xr-x   4 root       sys           8192 Mar 19 15:29 dt

As root I used vi to create and place a small test file inside /var/dt, and sure enough got the message "no such device or address" and it would not let my place the test file inside /var/dt.  The rest of /var looks good.

Will look into doing the fsck soon.  

 

Bill Hassell
Honored Contributor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

This is an early warning that the disk may be going bad.

Do you have your boot disk mirrorred?
Are you using Ignite to create a recovery image?



Bill Hassell, sysadmin
JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

So the full file system check ran last night in single user mode.  It didn't find anything, but after booting back up the same message was waiting for us-

Apr 5 02:05:13 vmunix: msgcnt 32 vxfs: mesg 008: vx_direrr: vx_dirscan_2
- /var file system inode 28171 block 4520501 error 6

About one second before those messages appeared, I saw this in the syslog-

Apr 5 02:04:56  automount[742]: Automount abort!!!
Apr 5 02:04:56  automount[742]: Possible cause:
Apr 5 02:04:56 automount[742]: 1)No map provided
Apr 5 02:04:56 automount[742]: 2)Using the same map as the running aut
omount
Apr 5 02:04:56  automount[742]: 3)The maps contain invalid mount points
Apr 5 02:04:57  inetd[761]: Reading configuration

Basically, the server boots, puts out the automount messages, then 1 second later the "vx_direrr: vx_dirscan_2" messages come along, then it boots up, the users hop on the server, and everything runs fine and the syslog stays clear until the next reboot..

I wonder if something was configured wrong with /var/dt vs there being a hardware or filesystem problem?

JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

And for Bill Hassell- yes, this is a mirrored pair of HDs and we do frequent backups.

 

JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

Here's what I was able to pull from the console logs related to the fsck that ran overnight-

/dev/vg00/lvol7 :

vxfs fsck: sanity check: /dev/vg00/lvol7 needs checking

log replay in progress

file system is not clean, full fsck required

pass0 - checking structural files

pass1 - checking inode sanity and blocks

pass2 - checking directory linkage

pass3 - checking reference counts

pass4 - checking resource maps

OK to clear log? (ynq)y

set state to CLEAN? (ynq)y

 

vxfs mount: /dev/vg00/lvol7 is corrupted. needs checking

mount: /dev/vg00/lvol1 is already mounted on /stand

# bdf

Filesystem          kbytes    used   avail %used Mounted on

/dev/vg00/lvol3     983040  138952  791339   15% /

/dev/vg00/lvol1     961051   49962  814983    6% /stand

/dev/vg00/lvol8    206503936 73208248 132256200   36% /JR

/dev/vg00/lvol6    7471104 1139336 5936139   16% /usr

/dev/vg00/lvol5    2097152    2007 1964260    0% /tmp

/dev/vg00/lvol4    5177344 1446421 3497758   29% /opt

# umount /JR

# fsck -y -o full /dev/vg00/rlvol7

fsck: /etc/default/fs is used for determining the file system type

log replay in progress

pass0 - checking structural files

pass1 - checking inode sanity and blocks

pass2 - checking directory linkage

pass3 - checking reference counts

pass4 - checking resource maps

OK to clear log? (ynq)y

 

# fsck -y -m -o full /dev/vg00/rlvol7

vxfs fsck: sanity check: /dev/vg00/rlvol7 OK

 

/dev/vg00/lvol7 :

vxfs fsck: sanity check: /dev/vg00/lvol7 OK

Bill Hassell
Honored Contributor
Solution

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

The HP-UX errno value for 6 means:

from /usr/include/sys/errno.h:
      errno 6 = ENXIO /* No such device or address      */

      [ENXIO]        No such device or address. I/O on a special file refers
                     to a subdevice that does not exist, or is beyond the
                     limits of the device.  It can also occur when, for
                     example, a tape drive is not on line or no disk pack is
                     loaded on a drive.

So this matches what you saw when creating a test file.
It is possibly a bookeeping error in the directory structure, but to verify that there are no bad sectors in /var, do this to read every sector:

# dd if=/dev/vg00/rlvol7 of=/dev/null bs=1024k

The dd comand will take several minutes as it appears that lvol7 is 37 GB in size.
I have attached a copy of the script bdfmegs in case you don't have it. It formats bdf for today's large filesystems:

File-System Mbytes Used Avail %Used Mounted on
/dev/vg00/lvol1 938 48 795 6% /stand
/dev/vg00/lvol3 960 135 772 15% /
/dev/vg00/lvol4 5056 1412 3415 29% /opt
/dev/vg00/lvol5 2048 1 1918 0% /tmp
/dev/vg00/lvol6 7296 1112 5797 16% /usr
/dev/vg00/lvol8 196.9g 69.8g 126.1g 36% /JR

 NOTE: The bdfmegs file has a Windoze extension .jpg so it can be attached to this post. It is a simple text file, so remove the .jpg and copy it to /usr/contrib/bin.

 I would rename the /var/dt directory to /var/dt_BAD and recreate /var/dt.
The dt directory is just temp storage for Xwindows.

 



Bill Hassell, sysadmin
JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

Thanks for such a detailed answer!

 

# dd if=/dev/vg00/rlvol7 of=/dev/null bs=1024k

I'll give this a try afer hours, when there aren't so many users on the system.  

JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

So I ran this command early this morning before the users got on-

# dd if=/dev/vg00/rlvol7 of=/dev/null bs=1024k

And after about 6 minutes the result came back-

36032+0 records in

36032+0 records out

No errors, and the in / out number matched, so that's good news.

Also tried this to remove /var/dt and recreate it, but it would not let me remove /var/dt:

 # cd /var

# ls -ld dt

drwxr-xr-x   4 root       sys           8192 Mar 19 15:29 dt

# rmdir dt

rmdir: dt: Directory not empty

# cd dt

# ls -lar

total 0

# ls -la

total 0

# ll

total 0

# cd /var

# ls -laR dt

total 0

#

Looks like /var/dt is an odd little gremlin in the system.  

 

 

Bill Hassell
Honored Contributor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

You wrote:
>>      # rmdir dt
>>      rmdir: dt: Directory not empty
>>      # cd dt
>>      # ls -lar
>>       total 0

Try: rm -rf /var/dt
You could also find the inode for /var/dt and remove the inode like this:

# ll -i /var/dt
(first column is the inode number)
# cd /var
# find . -inum ### -exec rm {} \;


Bill Hassell, sysadmin
JDR45
Frequent Advisor

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

Well between the fsck and the dd command, it is fixed.  /var/dt is fine and the syslog is clear of messages, even after a reboot or two.  So thanks for both recomendations.   

Is there a way I can select two different posts as "accept as solution?"

 

 

Re: dmesg showing vxfs: mesg 008... error 6 on /var filesystem inode

> Is there a way I can select two different posts as "accept as solution"?

 

No, only the one.  But you can give kudos to each.