awk, external command and quotes

[labadie_1]

I am struggling with awk and an external command.

I want to do, in a awk script, an external call to a Vms command

If I do
$ gawk :== $ sys$common:[syshlp.examples.tcpip.snmp]gawk

Then if I create a simple file a.awk
BEGIN{}
{ system(" wr sys$output f$time() ") }
Then the command
$ gawk/input=a.awk sys$input
followed by another RETURN
displays correctly the current time.

Now, if I want to issue a command with quotes, like
wr sys$output f$getjpi("20400416","PRCNAM")

I get various errors.

I have tried various tripling of the quotes, to no avail.

Thanks for any hint.

[labadie_1]

Of course, putting the command in a file and doing in the awk @file
works, but does not solve my problem, as I will have a pid varying

:-)

[John Gillings]

Gerard,

Have you tried using PIPE? Perhaps something like:

$ PIPE WRITE SYS$OUTPUT "something" | -
GAWK/input=a.awk SYS$PIPE:

I find this is often a simple way of generating variable input without writing a temporary file. If the input is complex, you can use a command procedure to avoid having multiple continuation lines in the PIPE command, like this:

$ self=F$PARSE(";",F$ENVIRONMENT("PROCEDURE"))
$ IF p1.EQS."INPUT" THEN GOTO Input
$ IF p1.EQS."OUTPUT" THEN GOTO Output
$ PIPE @'self' "INPUT" | @'self' "OUTPUT"
$ EXIT
$
$ Input:
$ ! Generate your input stream here
$ ! using WRITE SYS$OUTPUT
$ EXIT
$
$ Output:
$ ! Execute command to consume the output
$ EXIT

(in your case you probably don't need the output branch, use your AWK command instead).

[labadie_1]

Thanks John for the general method.

I realize I have not correctly explained my problem.

I am still working on the procedure I posted in the thread on "monitor network traffic"
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1120946
(By the way I have fixed a bug)
It displays some interesting data, like

BG270 ,Stream , 705,49153, 127.0.0.1, 127.0.0.1, 280, 0, 15, 0
BG778 ,Stream , 705,49185, 127.0.0.1, 127.0.0.1, 18748, 0, 1853, 0
BG281 ,Stream ,49154, 705, 127.0.0.1, 127.0.0.1, 25704, 0, 1837, 0
BG30531,Stream ,57678, 1800, 193.248.1.155, 10.196.10.30, 62464, 0, 245, 0
BG260 ,Dgram , 161, 0, , , 200310, 0, 5372, 0
BG604 ,Dgram , 3181, 0, 0.0.0.0, , 364228, 0, 35016, 0
BG32505,Stream , 1190,32801, 193.248.1.156, 10.196.11.80, 556708, 0, 59625, 0
BG24149,Stream ,50031, 1501, 193.248.1.155, 10.196.138.225, 2365544, 0, 253419, 0
BG671 ,Stream , 1190,42865, 193.248.1.156, 10.196.138.225, 15110388, 0, 260926, 0
BG710 ,Stream ,49182, 1616, 193.248.1.155, 193.248.120.121, 23229596, 0, 2488443, 0
BG711 ,Stream ,49183, 1515, 193.248.1.155, 193.248.120.120, 195537460, 0, 20934757, 0
BG749 ,Stream , 1190,60648, 193.248.1.156, 193.248.120.121, 826776008, 0, 3051698, 0
BG1201 ,Stream , 1190,64562, 193.248.1.156, 193.248.120.120, 880459944, 0, 22705543, 0
Device | type | ploc | prem | host local | host remote | bytes rec | bytes sent | I/O rec | I/O sen |
____________________________________________________________________________________________________________________


And I thought it could be good to add the pid and processname owning the Bg device.

So it is while I am inside awk that I want to get some info from Vms, typically

f$getdvi("bg123","pid")
and
f$getjpi(f$getdvi("bg123","pid"),"prcnam")

[labadie_1]

In fact, I think the problem is in the syntax: in the awk file, I must have
{ system(" vmscommand ")}
So as long as I have
$ sh system
$ sh time
$ write sys$output f$time()
$ wr sys$output 1

It works fine.

But when I want to issue a command such as
$ wr sys$output "abcdef"
or any other with the character "
the parser wrongly takes the first " of the vms command as the end of the vms command

In fact, if you put as vms command
write sys$output "abcdef"
or
write sys$output

you get the same result
_Expression

:-(

[Steven Schweda · Accepted solution]

You're not thinking UNIX.

alp $ type a.awk
BEGIN{}
{ system(" write sys$output f$time() ") }

alp $ gawk /inp = a.awk sys$input

28-MAY-2007 07:55:48.97


And:

alp $ type b.awk
BEGIN{}
{ system(" write sys$output f$edit( f$time(), "COLLAPSE") ") }

alp $ type b.awk
BEGIN{}
{ system(" write sys$output f$edit( f$time(), "COLLAPSE") ") }
alp $ gawk /inp = b.awk sys$input

%DCL-W-ARGREQ, missing argument - supply all required arguments


But:

alp $ type c.awk
BEGIN{}
{ system(" write sys$output f$edit( f$time(), \"COLLAPSE\") ") }

alp $ gawk /inp = c.awk sys$input

28-MAY-200707:56:54.23

[Hein van den Heuvel]

Free advice...

If you are going to do those getjpi prcnam for several sessions, then you may want to just pre-read some "show system" and stick the results into an array:

---- awk script ----
BEGIN { while ("show system" | getline) {
getjpi_prcnam[$1] = substr($0,10,15)
}
}
---------------------

Now you can just use getjpi_prcnam[pid] and know you are not spawning the system to a crawl.

But was the original input not from "show dev bg"? So you could just pick up process name and pic fron shwo dev/bg/full:

-----------------
BEGIN { while ("show device bg/full" | getline) {
if (/^Device/ ) {split ($2,x,","); bg = x[1];}
if (/UIC/) { split ($0,x,"\""); prcnam[bg] = x[2] }
if (/Owner process ID/) { pid[bg] = $4; print bg, x[2], $4; bg="X" }
}

for (bg in prcnam) {
print bg, pid[bg], prcnam[bg];
}
}


Or in original order:
--------------------------------

BEGIN { while ("show device bg/full" | getline) {
if (/^Device/ ) {units++; split ($2,x,","); bg[units] = x[1];}
if (/UIC/) { split ($0,x,"\""); prcnam[units] = x[2] }
if (/Owner process ID/) { pid[units] = $4; getjpi_prcnam[$4]=x[2] }
}
while (i++ < units) {
print bg[i], pid[i], prcnam[i];
}
}


I suspect the PERL is more suitable for this job though.


Enjoy,
Hein.

[labadie_1]

Steven

Thanks for the tip, I should have known better, as I had struggled with a Linux command on several other Linux nodes with rsh and echo $HOSTNAME
:-)

Hein
Thanks for the valuable info, but as I do
$ ana/sys
tcpip sh dev/fu bg

I do not have the pid and processname owning the bg edvice.

$ sh dev /fu bgxxx
gives it, but not the counters, remote and local port, remote and local host...

And
$ tcpip sh dev/fu bgxxx
gives data with a "bad format", not good for searching

[Hein van den Heuvel]

>> $ ana/sys .... tcpip sh dev/fu bg
>> I do not have the pid and processname owning the bg edvice.

I only see a PID in that output if there is an active IRP.

So I'm suggesting that if you do the regular DCL $SHOW DEV BG/FULL early, in the BEGIN section of the awk, then you can establish a known mapping from BG device name to PID and/or PRCNAM, through a simple bunch of arrays.
Then as you process the ANAL/SYS output, you can latch on to the BG name and look it up in those arrays for pid and/or prcnam.

>>. $ tcpip sh dev/fu bgxxx
gives data with a "bad format", not good for searching

yeah, sort of. It creates a long (815) byte record with embedded CR-LF.

That's what breaks a simple $PIPE tcpip sh dev/fu bgxxx | TYPE/PAGE/

It's not all that hard to process and specificall PERL automatically breaks it down into lines.

If you stick the output into a file, and then $convert/fdl="record; format stream" then the output will show as individual lines as well.

fwiw,
Hein.

[Sebastian Bazley]

You could surely just pass the pid into the script:

$ gawk -v pid=val -f script.awk

[Sheldon Smith]

FYI: With judicious use of quotes, you can do useful things directly from DCL *without* a secondary script file. The trick is understanding when to 'escape' a quotation mark, and when to double it. Or both. 8^)
Here are a couple 'one-liners' from my collection:

Initialize any disks that have *never* been initialized:
------
pipe sho dev dg | gawk "BEGIN {print ""$set noon""}; /Online/ {print ""$mou /over=id /noassi"", $1; print ""$if f$mess( $status, \""ident\"") .eqs. \""%NOHOMEBLK\"" -""; print "" then init/sys /limit"", $1, ""UNUSED""}" sys$pipe: | @sys$pipe:

Show the size of all locally mounted disks (in MB):
------
pipe show dev d/mou | GAWK "/Mounted/ && !/(remote mount)/ {print ""$ write sys$output \"""" $1 ""\t\"", f$getdvi( \"""" $1 ""\"", \""maxblock\""), \"" bl.\t\"", f$getdvi( \"""" $1 ""\"", \""maxblock\"")/2048, \"" MB\""""}" sys$pipe: | @sys$pipe:

[labadie_1]

Sebastian

I am not sure I understand: I have a variable number of BG devices, say 50, so passing a pid will not help

Sheldon

Thanks a bunch for the procedures, I will have a close look

[Sebastian Bazley]

Sorry, missed that part of the thread.

However you could join the pids (e.g. with a comma) if you wanted, and then split them again...

Or of course, put the raw pid details in a file that is read by the awk script.

[labadie_1]

I have attached the last version of my awk script.

It defines some symbols
- statbyr
sorts the BG Devices by Bytes received
- statbys
sorts the BG Devices by Bytes sent
- statior
sorts the Bg devices by I/O received
- statios
sorts the Bg devices by I/O sent

It is highly inefficient, as I do
for (bg in prcnam)
if (bg == toupper(dev))
printf ...

So if I have 200 bg devices, I will be 200x200 times in this loop

I have been surprised to discover that without the line
for (bg in prcnam), I do not find any array member.

Some with a higher knowledge of awk may explain it.

[Hein van den Heuvel]

Wow, that's just horrible.

1)
What's the tolower doing in "if (/^Device/) {tolower" ?
Did you mean:
if (/^Device/) {split (substr($2,1,(length($2)-2)),x,","); bg=tolower(x[1]) ;}

And is that substr stuff just to get of the ":"?
Then just split on that!
if (/^Device/) {split $2,x,":"); bg=tolower(x[1]) ;}


2)
... dev=toupper($2)
:
for (bg in prcnam)
if (bg == toupper(dev))

Is that second 'toupper' 'just in case the first one did not work?'

I think all you need is: bg = dev .. but at a place where all the other data is collected. I don't have a system with 'ucx shwo dev/full' available right not.
But let's assume 'Bytes transferred' is the last line.
In that case the solution show be:

/Bytes transferred/ {byt = 0 ; byt=$3 ; byr= 0 ; byr=$4 ;
bg = dev;
printf(...)
}

3) Should the printting of the header line (print("device... ) not be part of the BEGIN {} block?

4) Why 'awk' 4 times for 4 sort orders?
The data could change?! awk once, sort 4 times! Append to header lines after each sort.


Cheers,
Hein.

[labadie_1]

Hein wrote
>>>Wow, that's just horrible.
Yes, I concur
:-)

>>>1)
What's the tolower doing in "if (/^Device/) {tolower" ?

the bg device in the first list (with the pid) and the second list are one in uppercase and the other in lowercase, so the test failed

>>>And is that substr stuff just to get of the ":"?
yes, because comparing bg227 and bg227: will fail :-)

Then just split on that!
if (/^Device/) {split $2,x,":"); bg=tolower(x[1]) ;}


2)
... dev=toupper($2)
:
for (bg in prcnam)
if (bg == toupper(dev))

>>>Is that second 'toupper' 'just in case the first one did not work?'
I forgot this one.

>>>I think all you need is: bg = dev .. but at a place where all the other data is collected. I don't have a system with 'ucx shwo dev/full' available right not.
But let's assume 'Bytes transferred' is the last line.

Yes it is
In that case the solution show be:

/Bytes transferred/ {byt = 0 ; byt=$3 ; byr= 0 ; byr=$4 ;
bg = dev;
printf(...)
}

3) Should the printting of the header line (print("device... ) not be part of the BEGIN {} block?
May be, but anyway I want it at the back of the display, because I sort with the higher values at the end.

4) Why 'awk' 4 times for 4 sort orders?
The data could change?! awk once, sort 4 times! Append to header lines after each sort.

Yes of course

Thanks for your inputs.

[labadie_1]

I have done as you said, as Bytes transferred is the last motif searched

>>>But let's assume 'Bytes transferred' is the last line.
In that case the solution show be:

/Bytes transferred/ {byt = 0 ; byt=$3 ; byr= 0 ; byr=$4 ;
bg = dev;
printf(...)
}

And I no longer have the pid and prcnam.

There is something I have not understood in the flow of data in awk...

:-(

[Hein van den Heuvel]

I suspect there is still a slight punctuation error on the device name.

Print them out for a debug run, may on 1 device?

Email me a sample SHOW DEV BG/FULL as well as a sample UCX SHOW DEV/FULL and I'll see if I can find time to cehck it.

Flow in AWK is a surprise for first time users.
The main flow is 'all code' for all record in the input.
The /something/ {block} construct is really an 'if ($0 look like something) then {block}'

Most awk programs have simple {} block with no conditional, making that run for every line.

You can add a BEGIN {} and END {} which run jsut once typically to setup, print header fin BEGIN, and to print a total in END.

Cheers,
Hein.

[labadie_1]

Hein

I have emailed you the various files.
Thanks for your time.

I am trying to debug.

[labadie_1]

Thanks to Hein, the procedure now displays the pid and processname for each bg, without looping too much :-)

It is available at
http://dcl.openvms.org/stories.php?story=07/06/04/4359262

[Jess Goodman]

I tried what was posted at that url and it has some major issues. It tries to use the files bbb.awk and a.tmp as input files without first creating these files.

Also you should not use == to create DCL symbols when the symbols are only being used by the command procedure that creates them.

[labadie_1]

Sorry about that, I will fix it.

[Jess Goodman]

I tried running what was posted at that url and it has some major issues. The command procedure tries to use the files bbb.awk and a.tmp as input files without first creating these files.

I recommend using the current process's PID as part of the file name for any temporary files created by a command procedure.

Also you should not use == to create DCL symbols when the symbols are only being used by the command procedure that creates them.

[labadie_1]

>>> I recommend using the current process's PID as part of the file name for any temporary files created by a command procedure.

Yes, like how fntmp is defined in this procedure ?
http://dcl.openvms.org/stories.php?story=03/09/23/7962305

[Jean-François Piéronne]

As Hein has suggested a scriptting language is probably more suitable.

As example, I have attach a Python script doing a similar job (including a sort option on any of the displayed item).

Choice of sort direction can easily be added.

JF