topic Re: Read a locked file... in Operating System - OpenVMS

Read a locked file...

Juan M. Lazaro — Fri, 28 Jul 2006 04:51:09 GMT

Hello, I've some problems trying to read a locked file, the problem I have is that the file appears like

FILE.LOG 0/144

When I do a DIR FILE.LOG/SIZE=ALL.

This file is created with the command

SET H 0/LOG=FILE.LOG

The BACKUP/IGNORE=INTERLOCK command creates a file with 0 blocks and nothing into the file.

The command CONVERT/SHARE doesn't run here.

The EDIT/TPU/READ results are like the BACKUP command.

Any body knows any way to take the data from the memory region where are this file and put those data to a new file???
And how can I know in wich memory region are the file data???

Many thanks in advance.

Juan

Re: Read a locked file...

Heinz W Genhart — Fri, 28 Jul 2006 05:20:51 GMT

Hi Juan

If this file is the output log from a set host/log command, is the set host session stil active ? If yes, do a logout and you will be able to read the file.

Otherwise you could try a

$ set file/end sethost.log

hopefuly that helps

regards

Heinz

Re: Read a locked file...

Juan M. Lazaro — Fri, 28 Jul 2006 05:30:52 GMT

Hi Heinz,

The problem is that I want read it without a logout, I've tried the command

set file/end file.log

But I've the next errors

%E, error reading GLOG:[OPER.LOG]FILE.LOG;1
%W, file access conflict

And the file appears empty...

Thanks in any case.

Juan

Re: Read a locked file...

Volker Halle — Fri, 28 Jul 2006 05:55:23 GMT

Juan,

the data can be found in memory (in the RMS BDBs) and also on disk (after it has been flushed, which happens from time to time). You could just look at the hex/ASCII dump of the data in the file this way:

$ ANAL/SYS
SDA> SET PROC/IND=
SDA> SHOW PROC/RMS=BDBSUM
SDA> EXA addr;size

Take the value addr from column ADDR and the value for size from the column SIZE

SDA> SHOW PROC/RMS=WCB

This will show you the WCB (Window Control Block) address of the file.

SDA> CLUE XQP/FILE=
will list the File Header LBN (HdLBN) on the disk:

$ DUMP/BL=(COUNT=1,START=%X/FILE disk:

will dump and format the file header, which includes the Map Area (Retrieval pointers), which give you the LBN and size of each extent of the file on disk.

$ DUMP/BL=(count=size,START=%Xlbn) disk:

will then dump HEX/ASCII the contents of the file extents.

It can be done, but it's not easy.

Volker.

Re: Read a locked file...

Juan M. Lazaro — Fri, 28 Jul 2006 06:21:42 GMT

Hi Wolker,

First, many thanks about your answer, but I have had a problem in the last command,

The command output of

DUMP/BL=(COUNT=1,START=%X/FILE disk:

is:

Dump of device $1$DGA107: on 28-JUL-2006 13:15:26.50

Logical block number 4530 (000011B2), 512 (0200) bytes

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (1768,9,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Variable
File organization: Sequential
Record attributes: Implied carriage control
Record size: 0
Highest block: 144
End of file block: 1
End of file byte: 0
Bucket size: 0
Fixed control area size: 0
Maximum record size: 0
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics:
Caching attribute: Writethrough
Map area words in use: 2
Access mode: 0
File owner UIC: [1,4]
File protection: S:RWED, O:RWED, G:RE, W:
Back link file identification: (504,1,0)
Journal control flags:
Active recovery units: None
File entry linkcount: 0
Highest block written: 144
Client attributes: None

Identification area
File name: FILE.LOG;1
Revision number: 3
Creation date: 28-JUL-2006 09:48:16.59
Revision date: 28-JUL-2006 09:55:35.44
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 144 LBN: 1539378

Checksum: 57158

Then the last comand must be?

DUMP/BL=(count=144,START=%X1539378) $1$dga107:

Because it appears empty...

Many thanks again...

Juan

Re: Read a locked file...

Volker Halle — Fri, 28 Jul 2006 06:26:06 GMT

Juan,

sorry, my fault.

The LBN number from DUMP/FILE_HEADER is in decimal, so you must specify:

$ DUMP/BL=(count=144,START=1539378) $1$dga107:

Volker.

Re: Read a locked file...

Hein van den Heuvel — Fri, 28 Jul 2006 06:30:50 GMT

This problem has previously been discussed here.
Google: +site:itrc.hp.com +backup +locked
This gives:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=970391

And to a lesser degree in:

http://forums2.itrc.hp.com/service/forums/questionanswer.do?threadId=656460

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=990678

The data is probably on the disk, except for the last buffer.

IMHO this design choice in backup is wrong / a bug. It should either just copy the block irrespective of the EOF mark as the EOF mark really is strictly optional as far as the file system is concerned. Private applications can set of not set a reasonable EOF as they please. RMS does rely on it.
The should be something like BACK/IGNO=EOF or /ALLOCATED

In the mean time anyone needing a solution is left with the tedious DUMP and ANAL/SYST hacks Volker suggests, or would have to write a program to open the file with FIB$V_NOLOCK active and QIO copy the blocks to a work file. Maybe some rainy afternoon I'll code that up, but I'm afraid I do not have the time just now...

Good luck!
Hein.

Re: Read a locked file...

Juan M. Lazaro — Fri, 28 Jul 2006 06:31:27 GMT

Wolker, many thanks,

It runs perfectly, and it's not your fault, I can see it in the HELP too...

Many thanks for yours replies and the time expent in it...

Cheers,,

Juan

Re: Read a locked file...

Hein van den Heuvel — Fri, 28 Jul 2006 08:00:28 GMT

Just for grins... Here is an other way to 'see' the data.
This follows the file id into indexf.sys, to the header and then dump the data pointed to by the first mapping pointer.
I don't have my file system internals book handy, so for now it has the serious limitation of using a hardcoded 22 bit LBN retrieval pointer format. Yikes.
And it does just 1 pointer to begin with.
To be fixed if and when the procedure needs to be used for real.
For now it is just a 'look what I can do' excercise :-)

$if p1.eqs."" then inquire p1 "file name"
$dir/file/out=tmp.tmp/notrail/nohead/wid=file=1 'p1
$open/read tmp tmp.tmp
$read tmp tmp
$read tmp tmp
$clos tmp
$dele tmp.tmp.
$id = f$elem(0,",",f$elem(1,"(",tmp))
$dev = f$parse(p1,,,"device")
$ibmapsize = (f$getdvi(dev,"maxfiles") + 4095) / 4096
$indexf_bitmap_vbn = (f$getdvi(dev,"cluster") * 4) + 1
$header_vbn = indexf_bitmap_vbn + ibmapsize + id -1
$! dump/bloc=(start:'header_vbn',count=1) 'dev'[000000]indexf.sys
$vbn = "1234"
$vbn[0,32]='header_vbn
$open/read/share=write indexf 'dev'[000000]indexf.sys
$read/key=&vbn indexf header ! Fails if vbn looks like quote :-(
$close indexf
$!show sym header
$name_offset = 2*f$cvui(0,8,header)
$map_offset = 8*2*f$cvui(8,8,header)
$name = f$extr(name_offset,20,header)
$lbn = 256*256*f$cvui(8+map_offset,6,header) !Assume 22 bit pointer format
$lbn = lbn + f$cvui(16+map_offset,16,header) !Swapped low bits
$! lbn_bits = f$cvui(map_offset,32,header)
$! show symb lbn_bits
$! write sys$output "id=''id', vbn=''header_vbn', name=''name', lbn=''lbn'"
$dump/block=(start='lbn',count='f$getdvi(dev,"cluster")') 'dev'

Enjoy!
Hein.

Re: Read a locked file...

Juan M. Lazaro — Mon, 31 Jul 2006 05:05:48 GMT

Hi Hein,

Great work, maybe it was a rainy morning???

Many thanks for your program, work and time...

I've had a little problem, in the last line, the count only gives to me the disk cluster size, in my system is 3, and I've allocated 144 blocks, it can be solved if you replace the last line:

dump/block=(start='lbn',count='f$getdvi(dev,"cluster")') 'dev'

with

dump/block=(start='lbn',count='f$file_attributes(p1,"alq")') 'dev'

To display all the blocks allocated by the file.

In any case your program runs ok, thanks again.

Juan

Re: Read a locked file...

Hein van den Heuvel — Mon, 31 Jul 2006 07:07:25 GMT

Well, it is always safe to dump at least one cluster. That's why I used that shortcut.
I since came across a file systems internal book.
So here is an update which understands the 3 formats.

Hein.
$if p1.eqs."" then inquire p1 "file name"
$dir/file/out=tmp.tmp/notrail/nohead/wid=file=1 'p1
$open/read tmp tmp.tmp
$read tmp tmp
$read tmp tmp
$clos tmp
$dele tmp.tmp.
$id = f$elem(0,",",f$elem(1,"(",tmp))
$dev = f$parse(p1,,,"device")
$ibmapsize = (f$getdvi(dev,"maxfiles") + 4095) / 4096
$indexf_bitmap_vbn = (f$getdvi(dev,"cluster") * 4) + 1
$header_vbn = indexf_bitmap_vbn + ibmapsize + id -1
$dump/bloc=(start:'header_vbn',count=1) 'dev'[000000]indexf.sys
$vbn = "1234"
$vbn[0,32]='header_vbn
$open/read/share=write indexf 'dev'[000000]indexf.sys
$read/key=&vbn indexf header ! Fails if vbn looks like quote :-(
$close indexf
$!show sym header
$name_offset = 2*f$cvui(0,8,header)
$map_offset = 8*2*f$cvui(8,8,header)
$bits_in_use = 8*2*f$cvui(58*8,8,header)
$name = f$extr(name_offset,20,header)
$write sys$output "id=''id', vbn=''header_vbn', name=''name'"
$
$map_loop:
$fmt = f$cvui(8+6+map_offset,2,header)
$!lbn_bits = f$cvui(map_offset,32,header)
$!show symb lbn_bits
$if fmt.eq.1
$then
$ bits = 32
$ size = 1 + f$cvui(map_offset,8,header) ! 8 bit size
$ lbn = 256*256*f$cvui(8+map_offset,6,header) ! 22 bit map pointer
$ lbn = lbn + f$cvui(16+map_offset,16,header) ! 22 bit map pointer
$endif
$if fmt.eq.2
$then
$ bits = 48
$ size = 1 + f$cvui(map_offset,14,header) ! 14 bits size
$ lbn = lbn + f$cvui(16+map_offset,32,header) ! 32 bit map pointer
$endif
$if fmt.eq.3
$then
$ bits = 64
$ size = 1 + 256*256*f$cvui(map_offset,14,header) ! 14 MSB bits for size
$ size = size +f$cvui(16+map_offset,16,header) ! 16 LSB bits for size
$ lbn = lbn + f$cvui(32+map_offset,32,header) ! 32 bit map pointer
$endif
$
$write sys$output "use=''bits_in_use', fmt=''fmt' lbn=''lbn', size=''size'"
$! size=1
$dump/block=(star='lbn',coun='size') 'dev'
$map_offset = map_offset + bits
$bits_in_use = bits_in_use - bits
$if bits_in_use .ge.32 then goto map_loop
$exit

Re: Read a locked file...

Ian Miller. — Mon, 31 Jul 2006 08:19:46 GMT

Hein,
parhaps you can add some comments and submit it at dcl.openvms.org

Re: Read a locked file...

Hein van den Heuvel — Mon, 31 Jul 2006 22:59:35 GMT

Ian, Sure, I'll try. It should prove an interesting example of the odd things you can do with DCL.

However, I think the solution below is more useful.

Juan, I suggested one could write a program with ACP QIO and NOLOCK. However, I realize that while it is no rocket science, it is a little tricky. So I cobbled up something that seems to do the trick. Minimally tested, but probably good enough.

Program attached, and included below.
Enjoy!
Hein.

/*
** read_locked.c Hein van den Heuvel, July 2006
**
** This program can be used to copy the allocated blocks from a locked file
** Much like BACKUP/IGNORE=INTERLOCK is use the ACP NOLOCK access to over-
** ride file interlock. This requires SYSPRV or CONTROL ACCESS to the file.
** Unlike BACKUP this program simply reads all ALLCOATED blocks as the
** EOF information is likely not to be updated by the active writer (zero).
** The program copies MOST file attributes, but not all as its intended
** use is limited to copy live log files and such.
**
** You may need to use SET FILE/ATTR=(EBK:x,FFB:y) on the output as needed.
**
** If the code format looks a little odd and dated... it is!
** This program is loosly basde on some old example which where floating
** around at Digital Equipment such as SET_EXTENT.
** Suspected authors: Barry Dysert, Guenther Froehlin, me?...
**
** Enjoy!
** Hein, HvdH Perfromance Consulting
**
*/

#define MAXBLOCKS 120
#define MAXBYTES MAXBLOCKS*512

#include atrdef
/*
** libr/extr=fatdef/out=fatdef.h sys$library:sys$lib_c.tlb
*/
#include "fatdef.h"
#include fibdef
#include iodef
#include rms
#include stdio
#include stdlib
#include string

int sys$create(), sys$connect(), sys$write(), sys$close();
int sys$create(), sys$parse(), sys$search();
int sys$assign(), SYS$QIOW(), lib$stop();

main(argc,argv)
int argc;
char *argv[];
{
static char buf[MAXBYTES];
static char *usage = "Usage: $ read_locked name output_name\n";
static char esa[256], rsa[256];
static int status, channel, bytes, vbn=1;
static int file_hbk, file_nbytes, spec_nbytes;
static struct FAB fab, out;
static struct RAB rab;
static struct NAM nam;
static struct
{
short status;
char not_used[6];
} iosb;
static struct
{
short count;
short not_used;
void *address;
} fibdes;
static struct fibdef fib;
static struct atrdef atr[2];
FAT attributes;
struct { int len; char *addr; } devnam_desc;

/******************************************************************************/

/* Verify that we've been properly invoked */

if (argc != 3) printf("%s",usage), exit(1);

/* Use RMS to parse the file so that we get a FID of the QIO */

fab = cc$rms_fab;
fab.fab$l_fna = argv[1];
fab.fab$b_fns = strlen (argv[1]);
fab.fab$l_nam = &nam;

nam = cc$rms_nam;
nam.nam$l_esa = esa;
nam.nam$b_ess = sizeof (esa) - 1;
nam.nam$l_rsa = rsa;
nam.nam$b_rss = sizeof (rsa) - 1;

out = cc$rms_fab;
out.fab$b_fac = FAB$M_BIO | FAB$M_PUT;
out.fab$l_fna = argv[2];
out.fab$b_fns = strlen (argv[2]);

rab = cc$rms_rab;
rab.rab$l_fab = &out;
rab.rab$l_rbf = buf;
rab.rab$w_rsz = sizeof(buf);

if (((status=sys$parse(&fab)) & 1) != 1) lib$stop(status);
if (((status=sys$search(&fab)) & 1) != 1) lib$stop(status);

/* Get a channel for QIO access */

devnam_desc.addr = nam.nam$l_dev;
devnam_desc.len = nam.nam$b_dev;
if (((status=sys$assign(&devnam_desc,&channel,0,0,0)) & 1) != 1) lib$stop(status);

/* Set up the structures required for the ACP interface */

fibdes.count = sizeof(fib);
fibdes.address = &fib;

fib.fib$l_acctl = FIB$M_NOLOCK;
fib.fib$w_fid_num = nam.nam$w_fid[0];
fib.fib$w_fid_seq = nam.nam$w_fid[1];
fib.fib$w_fid_rvn = nam.nam$w_fid[2];

atr[0].atr$w_type = ATR$C_RECATTR;
atr[0].atr$w_size = ATR$S_RECATTR;
atr[0].atr$l_addr = &attributes;
atr[1].atr$w_type = 0;
atr[1].atr$w_size = 0;

/* Get the file's current attributes, such as hi-block, rfm,... */

status = SYS$QIOW(0,channel,IO$_ACCESS|IO$M_ACCESS,&iosb,0,0,&fibdes,0,0,0,&atr,0);
if ((status & 1) != 1) lib$stop(status);
if ((iosb.status & 1) != 1) lib$stop(iosb.status);

/* Validate the specified values (e.g. can't extend the file) */

file_hbk = attributes.fat$w_hiblkl + (attributes.fat$w_hiblkh << 16);
file_nbytes = file_hbk*512;

out.fab$l_alq = file_hbk;
out.fab$b_rat = attributes.fat$b_rattrib;
out.fab$b_rfm = attributes.fat$b_rtype;
out.fab$w_mrs = attributes.fat$w_maxrec; /* LRL=fat$w_rsize ?? */
out.fab$b_fsz = attributes.fat$b_vfcsize;
status = sys$create ( &out ) ;
if (status & 1) status = sys$connect ( &rab );
while ((status & 1) && (file_nbytes > 0)) {

if (file_nbytes >= MAXBYTES) {
bytes = MAXBYTES;
file_nbytes -= MAXBYTES;
} else {
bytes = file_nbytes;
rab.rab$w_rsz = file_nbytes;
file_nbytes = 0;
}

status = SYS$QIOW(0,channel,IO$_READVBLK,&iosb,0,0,&buf,bytes,vbn,0,0,0,0);
vbn += MAXBLOCKS;
if (status & 1) status = iosb.status;
if (status & 1) status = sys$write ( &rab );
}

/* Release the files */

(void) SYS$QIOW(0,channel,IO$_DEACCESS,&iosb,0,0,&fibdes,0,0,0,0,0);
(void) sys$close ( &out );
return status;
}

Re: Read a locked file...

Juan M. Lazaro — Tue, 01 Aug 2006 01:17:07 GMT

Hein, many thanks again, the last dcl you posted runs perfectly, the c program I don't know, I haven't any possibility now to compile it, but when I can I try to test it.

And I agree with Ian, you should post the dcl in dcl.openvms.org.

Thanks again to all the replies.

Juan