Multiple files???

[Peter Quodling]

I am seeing multiple apparent versions of the same files, turning up in a directory.
NETSERVER.LOG;13 15/15
NETSERVER.LOG;12 15/15
NETSERVER.LOG;11 15/15
NETSERVER.LOG;10 15/15
NETSERVER.LOG;9 15/15
NETSERVER.LOG;8 15/15
NETSERVER.LOG;12 15/15
NETSERVER.LOG;11 6/10
NETSERVER.LOG;10 6/10
NETSERVER.LOG;9 6/10
NETSERVER.LOG;8 6/10

Yet, Dir/full returns

Prd> dir/full n*.log;1

Directory WWW_PRD_ROOT:[000000]

NETSERVER.LOG;1 no such file

NETSERVER.LOG;1 no such file

Total of 2 files, 0/0 blocks.

It's a busy large Raid set (6 drive 0+1)
and people are getting worried.

ALso getting a lot of warnings (badhiblk, freespadrift, bad_nameorder and related...)

Mister Q

[Antoniov.]

Hi mister Q,
No such file are the file marked to deleted but not yet really deleted. I guess the deleting process is not terminate. ANAL/DISK could report same information (file marked to delete) and /REPAIR could solve it.
However, I believe you have some network process keep opened these file while some other try to delete them.

Antonio Vigliotti

[David B Sneddon]

Peter,

The last time I saw something like that was on a 4 member RAID 0+1
volume where 2 of the members had dropped out.
Are all 6 disks still OK?
The directory looks to be screwed. DFU might be able to help.
Does the directory command loop on that block?
Try using DUMP/DIRECTORY to examine the contents of the
blocks after the one containing the listed files.
I seem to recall that ANALYZE/DISK was unable to repair
the problem I came across.
How big is the directory containing these files?
It looks like possibly a directory shuffle was
interrupted.

Dave.

[Wim Van den Wyngaert]

Is it possible that WWW_PRD_ROOT is a search list ? Do sho log WWW_PRD_ROOT to find out.
Then the multiple files may be normal (do a dir for each part of WWW_PRD_ROOT to find out where the files really are).

That combined with open files that are deleted could be very normal.

Wim

[Uwe Zessin]

> No such file are the file marked to deleted but not yet really deleted.

Sorry, but I don't agree. This error message implies a 'dangeling directory entry'. It means that the directory entry points to a file header/ revision number combination that is no longer in use.

A simple 'DELETE NETSERVER.LOG;vers' should remove the dangeling entry in recent versions of OpenVMS - else, use '$ set FILE/REMOVE'.

File names or file revisions out of order indicate a corruption of the directory file. It is possible that a maintanance operation was aborted by a system crash.

The people have a good reason to be worried!

You might want to dismount the disk and take a BACKUP/PHYSICAL copy before you start any repairs.

After that, I agree with Antonio to further analyze the problem with '$ ANALYZE /DISK_STRUCTURE'.

[Willem Grooters]

Just can answer on your first concern, having no real experience on the second. Others have already given theirs.

is logical WWW_PRD_ROOT: a searchlist? if you SET DEF WW_PRD_ROOT before, DIR will run down each entry - each directory - and you'll get multiple lines, as shown.
This is standard behaviour; it has nothing to do with the storage architecture.

If not, there IS a serious problem with your system, but then you would be too busy to consult ITRC....

[Jan van den Ende]

Q,

do not yet discards Willem's idea.
If you defined your WWW_PRD_ROOT (obviously a Concealed Device) to be the concealed logical name for multiple directories, and files of the same name appear in different dirs, then the output looks to be out-of-sequence, like what you get. Defining a concealed logical for each directory tree, and defining a (NOT concealed) search list over those ROOTs, will generate dir headers between the various listings.

Your DIR/FULL presenting TWO files with the same name also points in that direction.

The "no such file" issue may well be a separate thing, which DOES need further attention, as indicated by Uwe.

Proost.

Have one on me.

Jan

[Uwe Zessin]

Peter wrote:
> ALso getting a lot of warnings (badhiblk, freespadrift, bad_nameorder and related...)

So, let's take a look:
$ help/message bad_nameorder

[Wim Van den Wyngaert]

Did a test : try to delete a file that is open (shared). And try to find file afterwards with dir.

6.2, 7.3, 8.2 : no files found

So, the message "no such file" could indicate problems.

Wim

[Uwe Zessin]

Of course. If you delete an open file, the system removes the directory link, but keeps the file header. Should the system now crash, you will have a 'lost file' - a header without a directory entry pointing to it -- the opposite of a dangeling directory entry ;-)

[Jan van den Ende]

To quickly generate a "no such file":

$ CREATE X.X
^Z
$ SET FILE /ENTER=Y.Y X.X
$ DELETE X.X.*
$ DIR Y.Y


Proost.

Have one on me.

Jan

[Uwe Zessin]

You would have to ammend your directory command with a qualifier that forces a request of some data from the file header, e.g.
$ directory /size Y.Y

else, directory will just display the file name.

I've just tried this under OpenVMS Alpha V7.3-2, but this games does not work any longer on a disk that has hard links enabled ;-)

[Hein van den Heuvel · Accepted solution]

Hi there mister Q. Long time no see.
(Per still lives on Queens way)

Considering that the sizes are different, we must be speaking about different file. But why speculate if you can be sure... for your next DIR command add in /FILE this will report the FILE ID (which points to the file header). This is the only information in the directory itself. Any other detail (size, date) will come form the header and may cause 'no such file' even if the directory entry is there.

Now in your example you use: "NETSERVER.LOG;1", and you give no 'proof' that this file might simple not exist. That would be a very good reason for "no files found", but not for "no such file", the latter message comes from the broken link, as mentioned earlier. I suspect that for a file it an extension header, a missing extention could cause this also.

DUMP is your friend to analyze further. DUMP/DIR as mentioned, but also DUMP/IDEN to go after the reported file ID and see what it there now.

You may want to blow away the directory (in a soft, recoverable way: set file /nodir, rename) and have DFU or ANA/DISK find back the files.

>>> ALso getting a lot of warnings (badhiblk, freespadrift, bad_nameorder and related

That would worry me all right!

Start doublechecking the backup procedures, logs. Start reviewing the recovery procedure.

Good luck

Hein.

[Lawrence Czlapinski]

Peter,
1. www_prd_root probably has multiple translations. The multiple versions are probably from 2 different directories especially since they are in 2 groups of desending sequence with different sizes. So that part is probably ok. As previously suggested do show log/sys of www_prd_root and do individual dir n*.log.
SYS$SYSROOT is an example of a multiple translation logical.
2. The no such file may be a real problem.
Perhaps a delete of the files was interrupted. Do another dir n*.log;1 just to be sure that you didn't catch it while it was being deleted. It's not like, but just to cover that possibility. If you get the same results, than do as others have suggested to correct it.

[John Gillings]

Hey Q!

I'd look at DIR/FILE then examine the files by FID. This might be an artifact of a search list, or it could be directory corruption (bad_nameorder doesn't look good).

If it is directory corruption, I'd recommend repairing the directories with ANALYZE/DISK/REPAIR *AND* DFU. Make sure your disk is clean, then watch the directory carefully for a few weeks to see if the problem returns.

As you know, there are many possible causes of this type of problem, and it's all but impossible to diagnose retrospectively. We need to at least narrow down a time window to have a good crack at figuring out the cause.

[Ian Miller.]

It looks like a bad directory file.

The repair for bad_nameorder errors is to get rid of the directory, use ANAL/DISK/REP, or DFU to recover the files now not in a directory and then recreate the directory.

[Peter Quodling]

re all...

www_prd_root is not a search list.

show log www_prd_root
"WWW_PRD_ROOT" = "DATA1:[PROD04.WWW_PROD.]" (LNM$SYSTEM_TABLE)
Prd> show log www_prd_root/full
"WWW_PRD_ROOT" [exec] = "DATA1:[PROD04.WWW_PROD.]" [concealed] (LNM$SYSTEM_TABLE)

By File ID I see...
NETSERVER.LOG;13 (402,44510,0) 15/15
NETSERVER.LOG;12 (3109,25741,0) 15/15
NETSERVER.LOG;11 (2686,1568,0) 15/15
NETSERVER.LOG;10 (1444,16908,0) 15/15
NETSERVER.LOG;9 (1030,57182,0) 15/15
NETSERVER.LOG;8 (929,16481,0) 15/15
NETSERVER.LOG;7 (3715,57855,0) 15/15
NETSERVER.LOG;6 (932,29654,0) 15/15
NETSERVER.LOG;5 (3544,5997,0) 15/15
NETSERVER.LOG;4 (1758,20663,0) 15/15
NETSERVER.LOG;3 (479,6900,0) 15/15
NETSERVER.LOG;2 (3511,13613,0) 15/15
NETSERVER.LOG;1 (3466,1304,0) 15/15
NETSERVER.LOG;12 (5752,21700,0) 15/15
NETSERVER.LOG;11 (3534,8418,0) 6/10
NETSERVER.LOG;10 (5112,2341,0) 6/10
NETSERVER.LOG;9 (3425,11024,0) 6/10
NETSERVER.LOG;8 (5127,49434,0) 6/10
NETSERVER.LOG;7 (3825,23594,0) 6/10
NETSERVER.LOG;6 (3379,1160,0) 6/10
NETSERVER.LOG;5 (3378,24912,0) 6/10
NETSERVER.LOG;4 (3375,6237,0) 6/10
NETSERVER.LOG;3 (2858,19727,0) 6/10
NETSERVER.LOG;2 (4097,7333,0) 6/10
NETSERVER.LOG;1 (4118,7704,0) 6/10

Total of 82 files, 451980/452230 blocks.
P

Looks like there goes the upcoming long weekend.

Peter

[Paul Jerrom]

So is DATA1: a logical name search list?

I've seen the "no such file" problems when restoring incremental backups. Until all the incrementals are restored, missing files are flagged as existing in the directory, but give a "no such file" error when you try to look at the file itself (because there is no such file!!).

[Peter Quodling]

No such luck.

Prd> show log/full data1
"DATA1" [exec] = "$2$DRA1:" [concealed,terminal] (LNM$SYSTEM_TABLE)

And there haven't been any restores to this disk in recent times.

q

[Antoniov.]

Hi Q,
attached file is an example how you can see "No such file" message.
The command file creates a lot of TEMP*.TMP files, then spawn a subprocess to delete them while executes dir command.
While subprocess is deleting files, the dir catches some entry of deleted file.
Obviously, this is a merely temporary situation but for some weird reason in your case is not temporary.

Antonio Vigliotti
P.S. If you run attached example, please, execute it in a temporary directory.

[Peter Quodling]

Hi everybody, well after a couple of hours on Sunday, things are looking healthier. The multiple files, and sundry other symptoms are gone.

I am still seeing some allocclr, baddirent,lostheader, and freespadrift errors, but suspect, that might be a vagary of caching not keeping up. Not to check the XFC patches.

Q

[John Gillings]

Q,

>I am still seeing some allocclr, baddirent,lostheader, and freespadrift errors, but suspect, that might be a vagary of caching not keeping up.

Could be. On recent versions of OpenVMS you can ANALYZE/DISK/LOCK to eliminate spurious errors without performing a repair (but it does lock your volume against allocations and deallocations).

Another option is to use DFU VERIFY in addition to ANALYZE/DISK.

[Antoniov.]

Hi Q,
what did you do to solve?

Antonio Vigliotti

[Peter Quodling]

Sorry about delay in response. What did I do to fix? Copied everything in that directory to another, anal/disk/repair, and then dfu/repair.

hasn't played up since.

[Peter Quodling]

All Solved - closing thread.