topic Re: T4 Help in Operating System - OpenVMS

T4 Help

Joshua Gould — Thu, 02 Aug 2007 08:38:22 GMT

I've installed T4 and things look good. The problem is that the *COM.CSV report that is emailed to me only contains the column headers and no data.

I've looked at the log and I take the same lines in the log, run them and it produces a good CSV file.

(16:39:05) $ Set Command T4$Sys:T4$Mon_Extract
(16:39:05) $ T4Extr T4_ZEUS_01AUG2007_1637_1639_Mon.Dat -
/Csv_File=T4_ZEUS_01AUG2007_1637_1639_Mon.Csv -
/Class=(All,NoScs,NoDisk,NoRms) -
/Process_Name=("*LMD0*","*SHAD*","*PRODOPER*","*BATCH*","*ASSEM*","*T4*")

Re: T4 Help

Volker Halle — Thu, 02 Aug 2007 08:48:54 GMT

Joshua,

there should be a T4$DATA:T4$COLLECT_ZEUS.LOG file. Any errors in there from the extract ?

Volker.

Re: T4 Help

Joshua Gould — Thu, 02 Aug 2007 08:55:27 GMT

No errors in the log. (taken from a later run but they all have failed)

(17:49:05) $ Set Command T4$Sys:T4$Mon_Extract
(17:49:05) $ T4Extr T4_ZEUS_01AUG2007_1748_1749_Mon.Dat -
/Csv_File=T4_ZEUS_01AUG2007_1748_1749_Mon.Csv -
/Class=(All,NoScs,NoDisk,NoRms) -
/Process_Name=("*LMD0*","*SHAD*","*PRODOPER*","*BATCH*"
(17:49:05) $ T4Extr_Status = $Status
(17:49:05) $
(17:49:05) $
(17:49:05) $! Now extract the DISK data as a separate operation.
(17:49:05) $
(17:49:05) $ T4Extr T4_ZEUS_01AUG2007_1748_1749_Mon.Dat -
/Csv_File=T4_ZEUS_01AUG2007_1748_1749_Disk.Csv -
/Class=(NoAll,Disk)
(17:49:05) $ T4Extr_Status = $Status
(17:49:05) $
(17:49:05) $
(17:49:05) $! Now extract the SCS data as a separate operation.
(17:49:05) $
(17:49:05) $ T4Extr T4_ZEUS_01AUG2007_1748_1749_Mon.Dat -
/Csv_File=T4_ZEUS_01AUG2007_1748_1749_Scs.Csv -
/Class=(NoAll,Scs)
(17:49:05) $ T4Extr_Status = $Status
(17:49:05) $

Re: T4 Help

Martin Vorlaender — Thu, 02 Aug 2007 09:10:51 GMT

Joshua,

perhaps you're looking in the wrong places. The snippet you posted isn't the place where the .csv files get glued together. That one starts after a comment

Last, but not least, we glue all the .CSV records together
horizontally. This is done using T4$APRC, which takes two
parameters (P1 and P2), ...

The *_COMP.CSV is generated there (although its name is *_MON.CSV there). The section ends with a line

$ Rename/Log T4_host_date_from_to_Mon.Csv; T4_host_date_from_to_Comp.Csv

HTH,
Martin

Re: T4 Help

Joshua Gould — Thu, 02 Aug 2007 10:09:09 GMT

No errors that I can see.

(17:49:05) $ T4Aprc T4_ZEUS_01AUG2007_1748_1749_AcLg.Csv; T4_ZEUS_01AUG2007_
APRC-I-STATS, 4 record(s) read from input and 4 records written to output ...
(17:49:05) $
(17:49:05) $ If F$Search("T4_ZEUS_01AUG2007_1748_1749_Xfc.Csv", 124124) .Nes
(17:49:05) $ Then
(17:49:05) $ T4Aprc T4_ZEUS_01AUG2007_1748_1749_Xfc.Csv; -
T4_ZEUS_01AUG2007_1748_1749_Mon.Csv
APRC-I-STATS, 4 record(s) read from input and 4 records written to output ...
(17:49:05) $ EndIf
(17:49:05) $
(17:49:05) $ If F$Search("T4_ZEUS_01AUG2007_1748_1749_Lck7.Csv", 124125) .Ne
(17:49:05) $ Then
(17:49:05) $ T4Aprc T4_ZEUS_01AUG2007_1748_1749_Lck7.Csv; -
T4_ZEUS_01AUG2007_1748_1749_Mon.Csv
APRC-I-STATS, 4 record(s) read from input and 4 records written to output ...
(17:49:05) $ EndIf
(17:49:05) $
(17:49:05) $
(17:49:05) $ If F$Search("T4_ZEUS_01AUG2007_1748_1749_Tcp.Csv", 124126) .Nes
(17:49:05) $ EndIf
(17:49:05) $ Net_Device_Index = 0
(17:49:05) $ Aprc_Next_Net_Device:
(17:49:05) $ Net_Device = F$Element(Net_Device_Index, ",", P4) - ":"
(17:49:05) $ Say Net_Device

(17:49:05) $ If (Net_Device .Eqs. "") Then Goto Send_Email
(17:49:05) $ Send_Email:
(17:49:05) $ Rename/Log T4_ZEUS_01AUG2007_1748_1749_Mon.Csv; T4_ZEUS_01AUG20
%RENAME-I-RENAMED, DSA0:[000000.T4$DATA]T4_ZEUS_01AUG2007_1748_1749_MON.CSV;4 r
(17:49:06) $ Delete/Log T4_ZEUS_01AUG2007_1748_1749_Mon.Csv;*
%DELETE-I-FILDEL, DSA0:[000000.T4$DATA]T4_ZEUS_01AUG2007_1748_1749_MON.CSV;3 de
%DELETE-I-FILDEL, DSA0:[000000.T4$DATA]T4_ZEUS_01AUG2007_1748_1749_MON.CSV;2 de
%DELETE-I-FILDEL, DSA0:[000000.T4$DATA]T4_ZEUS_01AUG2007_1748_1749_MON.CSV;1 de
%DELETE-I-TOTAL, 3 files deleted (105 blocks)
(17:49:06) $

Re: T4 Help

Volker Halle — Thu, 02 Aug 2007 10:26:16 GMT

Joshua,

(17:49:05) $ T4Aprc T4_ZEUS_01AUG2007_1748_1749_AcLg.Csv; T4_ZEUS_01AUG2007_
APRC-I-STATS, 4 record(s) read from input and 4 records written to output ...

These are just the 4 header lines, so there are/were no data records in that _AcLg.CSV file and in all the others.

Did you ever run T4 for more than 1-2 minutes ? Same problem then ?

Volker.

Re: T4 Help

Joshua Gould — Thu, 02 Aug 2007 10:28:32 GMT

Yes. I've also ran it a few times for an hour. I did this following the directions in the T4 docs to determine space requirements. I've also tried setting the sample interval down to as low as 10 seconds to produce more data.

The truely odd thing is that if I run the extract manually (in my first post) using the same commands and .DAT file, I get a good CSV.

Re: T4 Help

Volker Halle — Thu, 02 Aug 2007 10:42:46 GMT

Joshua,

there are various data collectors running for T4. Are you saying that none of the files collected by the various collectors have any data, e.g.

T4$DATA:T4_ZEUS_01AUG2007_1748_1749_*.CSV

or does this affect just the _MON.CSV file ?

If you look at the 3 *_xxx.ZIP files (xxx=COMP,DISK,SCS) created for each collection, do they also contain no data records ?

Volker.

Re: T4 Help

Joshua Gould — Thu, 02 Aug 2007 10:47:28 GMT

Disk and SCS have data. Mon and COMP do not.

Re: T4 Help

Volker Halle — Thu, 02 Aug 2007 11:12:52 GMT

Joshua,

*_MON.CSV will be renamed to *_COMP.CSV and the older copies of *_MON.CSV will be deleted.

The *_MON.CSV file will be created by the first T4Extr from *_MON.DAT. Then the data from the other collectors will be appended with T4Aprc, created a new *_MON.CSV file for every append operation.

In your example, T4Aprc has only copied 4 records from the other .CSV files, this will probably happen, if the original input file specified as P2 (*_MON.CSV) only has 4 records.

Is this consistent with a longer run, i.e. only 4 records copied from initial _MON.CSV even if running T4 for an hour ?

So - as shown in your initial question - why does the first T4Extr command fail silently in the context of the T4$COLLECT batch job, but not when executed interactively ?

Take the commands executed interactively and run them in a batch procedure under the same user as T4$COLLECT is running.

Volker.

Re: T4 Help

Volker Halle — Thu, 02 Aug 2007 11:26:00 GMT

Joshua,

you should be able to use $ @T4$SYS:T4$NOW while a T4 collection is running, to create a snapshot of the collected data. T4$NOW runs interactively in your process context - if that is instrumental to the problem.

Volker.

Re: T4 Help

Joshua Gould — Thu, 02 Aug 2007 15:55:26 GMT

>>So - as shown in your initial question - why does the first T4Extr command fail silently in the context of the T4$COLLECT batch job, but not when executed interactively ?
>>
>>Take the commands executed interactively and run them in a batch procedure under the same user as T4$COLLECT is running.

Some insight, I'm still getting 4 records even running an hour.

(16:49:06) $ T4Aprc T4_ZEUS_02AUG2007_1547_1647_AcLg.Csv; T4_ZEUS_02AUG2007_1547_1647_Mon.Csv
APRC-I-STATS, 4 record(s) read from input and 4 records written to output ...

Re: T4 Help

Volker Halle — Fri, 03 Aug 2007 00:06:47 GMT

Joshua,

try to interactively run $ @T4$SYS:T4$NOW while a T4 collection is running.

Also try to run the T4Extr ..._Mon.Dat command in a batch job under the same user as T4$COLLECT is running.

Always use the same account under which the T4$COLLECT batch job is running.

These 2 tests try to determine, if the problem running T4Extr against the *_Mon.Dat file at your node has something to do with process context (batch/interfactive, quotas, symbols, logicals).

T4Extr has - as it's a good tradition in real native OpenVMS tools - a debug logical:

$ define T4EXTR$DEBUG TRUE

Consider to use this logical in your tests, maybe even temporarily add it to T4$COLLECT.COM - at least when running T4Extr against the *_Mon.Dat file - then deassign the logical afterwards.

Volker.

Re: T4 Help

Willem Grooters — Fri, 03 Aug 2007 00:31:51 GMT

First some background:
In certain cases MONITOR introduces time drift (a total of a few seconds in a 24 hour measurement) and this could be highly undesirable. Therefore, a special version of MONITOR is delivered with T4 that prevents this timedrft. In most cases however, this timedrift is not a problem.
This special version could cause your problems.
To prevent it from running, comment out the lines that DEFINE MONITOR ... - depending your hardware and VMS version, to use the VMS-native MONITOR.

Re: T4 Help

Joshua Gould — Mon, 06 Aug 2007 08:04:59 GMT

Hello,

I did further testing on Friday and still baffled.

I guess the oddest thing we found was that I moved the extract for DISK ahead of COM. I ended up with a good Disk.csv and still with an empty COM.CSV.

Re: T4 Help

EdgarZamora_1 — Tue, 07 Aug 2007 09:54:49 GMT

Might you be running VMS 8.3 on Alpha? If so, I recommend waiting for the next version of T4 to come out or requesting a copy of the beta.

Re: T4 Help

Volker Halle — Tue, 07 Aug 2007 12:24:07 GMT

Joshua,

could you try an explicit extract of all the relevant MONITOR classes ? Instead of

/Class=(All,NoScs,NoDisk,NoRms)

try

/Class=(NOALL,DLOCK,FCP,..)

Volker.

Re: T4 Help

Willem Grooters — Wed, 08 Aug 2007 01:18:26 GMT

Joshua,

See thread http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1138290

It might be of help, if you're running 8.3 on alpha.
If so, send a request to the t4-people at HP, they could offer you a new version (in test)

WG

Re: T4 Help

Volker Halle — Wed, 08 Aug 2007 01:38:42 GMT

Willem,

as far as I understand the problem discussed in this thread:

T4Extr of the *_mon.DAT producing _Mon.CSV fails when run in BATCH, but succeeds if run interactively afterwards.

I cannot see, how this could have something to do with the MONITOR not collecting data problem discussed in the other thread.

Volker.