topic Re: lstat and file size in Operating System - OpenVMS

lstat and file size

mpradhan — Mon, 28 Mar 2016 00:39:52 GMT

Hello,

I have big file.

$ dir /full dka0:[ssq]PAGEFILE.SYS

Directory DKA0:[SSQ]

PAGEFILE.SYS;1 File ID: (6791,17,0)
Size: 33554432/33554432 Owner: [1,1]
Created: 24-MAR-2016 14:58:22.12
Revised: 24-MAR-2016 15:52:35.58 (2)
Expires: <None specified>
Backup: <No backup recorded>
Effective: <None specified>
Recording: <None specified>
Accessed: 24-MAR-2016 14:58:22.12
Attributes: 24-MAR-2016 15:52:35.58
Modified: 24-MAR-2016 14:58:22.12
Linkcount: 1
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 33554432, Extend: 0, Global buffer count: 0
No version limit, Contiguous best try
Record format: Undefined, maximum 0 bytes, longest 0 bytes
Record attributes: None
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:, World:
Access Cntrl List: None
Client attributes: None

Total of 1 file, 33554432/33554432 blocks.
$

when I use lstat() to stat this file and get the size, I see zero.

#include <stdio.h>
#include <stdlib.h>
#include <string>
#include <time.h>
#include <errno.h>
#include <unixlib.h>

using namespace std;

int
main(int argc, char** argv)
{
string sFilePath;
cout << "Enter Path: ";
cin >> sFilePath;
struct stat st;
int iRet(0);

if((iRet = lstat(sFilePath.c_str(), &st)) < 0)
{
cerr << "lstat(" << sFilePath << ") failed, error=" << strerror(errno) << endl;
exit(-1);
}
cout << "ctime: " << asctime(localtime(&st.st_ctime));
cout << "mtime: " << asctime(localtime(&st.st_mtime));
cout << "atime: " << asctime(localtime(&st.st_atime));
cout << "size: " << st.st_size << endl;
exit(0);
}

$ test_stat
Enter Path: dka0:[ssq]PAGEFILE.SYS
ctime: Thu Mar 24 14:58:22 2016
mtime: Thu Mar 24 15:52:35 2016
atime: Thu Mar 24 15:52:35 2016
size: 0

I think this is a bug in CRTL lstat().

Volume has CLUSTER_SIZE=16.

Any direction is appreciated.

Thanks,

Manoj

Re: lstat and file size

mpradhan — Mon, 28 Mar 2016 00:52:24 GMT

this is the FDL

IDENT FDL_VERSION 02 "27-MAR-2016 20:49:02 OpenVMS ANALYZE/RMS_FILE Utility"

SYSTEM
SOURCE OpenVMS

FILE
ALLOCATION 33554432
BEST_TRY_CONTIGUOUS yes
CLUSTER_SIZE 16
CONTIGUOUS no
EXTENSION 0
FILE_MONITORING no
NAME "DKA0:[SSQ]PAGEFILE.SYS;1"
ORGANIZATION sequential
OWNER [1,1]
PROTECTION (system:RWED, owner:RWED, group:, world:)
GLOBAL_BUFFER_COUNT 0
GLBUFF_CNT_V83 0
GLBUFF_FLAGS_V83 none

RECORD
BLOCK_SPAN yes
CARRIAGE_CONTROL none
FORMAT undefined
SIZE 0
$

Re: lstat and file size

Steven Schweda — Mon, 28 Mar 2016 04:41:05 GMT

My C++ expertise is approximately nil, but...

> size: 0
>
> I think this is a bug in CRTL lstat().

I'd bet against that.

> Size: 33554432/33554432 Owner: [1,1]

Let's see that in bytes:

ALP $ write sys$output 33554432 * 512
0

Look familiar?

Of course, with my weak psychic powers, I can't see how you compiled
your program, but I'd be willing to guess that /DEFINE = _LARGEFILE was
not part of the command. Around here, for example, with a smaller file:

ALP $ dire /size=all DISK$VMS083ALP:[000000]PAGEFILE.SYS

Directory DISK$VMS083ALP:[000000]

PAGEFILE.SYS;6 6400128/6400128

alp $ cxx fsiz
alp $ cxxlink fsiz
alp $ r fsiz
Enter Path: DISK$VMS083ALP:[000000]PAGEFILE.SYS
ctime: Mon Nov 30 17:33:25 2009
mtime: Wed Dec 9 10:31:14 2009
atime: Wed Dec 9 10:31:14 2009
size: -1018101760

Clearly, this is nonsense, but:

ALP $ write sys$output 6400128 * 512
-1018101760

It's consistent nonsense. With 64-bit file sizes (instead of 32):

alp $ cxx fsiz /define = _LARGEFILE /obje = fsizl
alp $ cxxlink fsizl
alp $ r fsizl
Enter Path: DISK$VMS083ALP:[000000]PAGEFILE.SYS
ctime: Mon Nov 30 17:33:25 2009
mtime: Wed Dec 9 10:31:14 2009
atime: Wed Dec 9 10:31:14 2009
size: 3276865536

Using a program with more arithmetic than DCL:

ALP $ pipe write sys$output "6400128 * 512" | bc
3276865536

If you use a language like C or C++ but forget how computers do
arithmetic, then it's easy to become confused.

Re: lstat and file size

Hoff — Mon, 28 Mar 2016 15:56:44 GMT

Okay, the basic question here — lack of 64-bit — appears to have been addressed.

Some general suggestions going forward with this project...

You're apparently working on getting the Commvault client operational on OpenVMS, and are presently getting tangled on some of what will probably be the less-involved problems. As the work on the CommClient backup client progresses, you're particularly going to want to acquire copies of the VMS RMS Data Structures and Internals book (Kirby McCoy) and the most recent copy of the Internals and Data Structures Manual (Ruth Goldenberg) that you can find, and probably also VAXcluster Principles (Roy Davis). Most or all of these are out of print, but used copies are generally available. Not necessarily cheaply. There's also the ODS2 description — those are old-style text files, and are not Microsoft formats — from the OpenVMS Freeware.

These books will give you some background on file locking as well as some of the metadata involved with OpenVMS files.

Why these books? It's very common for third-party clients starting from a UNIX client — as appears to be the case here — to get derailed by the rather substantial differences between the UNIX file system and its metadata and general lack of locking, and that of OpenVMS with RMS and the XQP, and ODS-2 and ODS-5, as well as whatever VSI is cooking up with their new OpenVMS file system, and particularly host-local and cluster file locking that's integrated into OpenVMS applications by default. (Proper management of the file system metadata and of locking are where more than a few of these enterprise backup clients have cratered in their first several versions, unfortunately.)

You'll also definitely want to discuss this project directly with the folks at VSI, and see what sorts of assistance or resources that they might or can offer here.

You'll also want to have a look at the internals of the zip and unzip tools, as this pair of tools is one of the very few open-source and portable packages that deals well with the OpenVMS metadata. Also the vmsbackup tool.

A backup tool is no small project to get right, so hopefully this information can help you deliver what your customers expect.

Re: lstat and file size

mpradhan — Mon, 28 Mar 2016 16:15:26 GMT

thanks a lot, I am not sure why LARGEFILE didnt come to my mind, sorry!

Re: lstat and file size

mpradhan — Mon, 28 Mar 2016 16:18:29 GMT

Thank You!!