File Transfer

[Jerry Rieman]

My application needs to move large files to a web front end on an as requested basis. We have gigabit ethernet connectivity between the alpha and windows based front end.
Stand alone interactive ftp will move 5 megs in .05 seconds (one twentieth of a second)between the two systems. If I put that same ftp into a subprocess that uses a system service call to sys$creprc to execute the ftp is takes nearly 90 seconds. I suspect a buffer/socket size issue somewhere.
I know the sys$creprc system service call uses the sysgen PQL params for quotas - and I have the PQL values set to the same values as the authorize values used by the interactive user.
What am I missing? Any ideas

[Hoff]

You're probably not specifying a full list of process quotas on the sys$creprc call.

Thomas Pfau (pfau) wrote a callable ftp a while back as another potential approach available here, and that source code has been posted at various sites.

[John Gillings]

Jerry,

As an experiment, what happens if you replace your $CREPRC with a LIB$SPAWN equivalent?

Also be VERY VERY careful about your PQL structure. It's unaligned and can be tricky to convince a compiler to layout correctly. To make absolutely certain it's correct, run your code under DEBUG and STEP/INTO the system service, then EXAMINE/HEX the PQL structure, making sure you can correctly decode the byte/long byte/long records.

[Hoff]

http://www.eight-cubed.com/examples/framework.php?file=sys_creprc.c

[Jerry Rieman]

Thanks guys. I was not using the item list for the quotas - instead expecting to pick up the PQL quotas as defaults. Went ahead and added the item list for the quotas into the call to sys$creprc and performance seems improved. Must have been a low PQL I missed. Yes, the byte/long item list grouping is a tad unusual. Was also going to try lib$spawn as suggested but since my logic was running as a shareable subroutine out of a shareable library, I didn't have the standard CLI easily available.
Appreciate the help. Reps coming.

[John Gillings]

Jerry,

>running as a shareable subroutine out of a
>shareable library, I didn't have the
>standard CLI easily available.

Not sure I understand this. Unless you're running as a "naked" detached process you have a CLI and should be able to call LIB$SPAWN.

Whatever, to diagnose the issue, I'd suggest you extract the FTP operation and execute it independently of whatever else code you have. Your SPAWN test could be a one liner:

status=LIB$SPAWN('@DO_FTP')

Try other ways of executing the same effective command, SUBMIT, SPAWN, interactive, etc... 0.05secs to 90 seconds is a rather serious difference. Not one I'd expect to see from mere tuning.

Relying on default and/or minimum PQL SYSGEN parameters for a $CREPRC is a recipe for a headaches in the future. You WILL forget that you're dependent on them, and when you move the code or upgrade you'll have to diagnose why it mysteriously fails. To save your sanity, make sure all process creations (RUN/DETACHED or $CREPRC) have fully populated PQLs.

[Jerry Rieman]

It is indeed a naked detached process. It is a listener process waiting on input on a specific socket/port - doing database io and returning the results back on the same socket/port to a web front end application - all multi-threaded via calls to shareable subroutines. We frequently use lib$spawn in other areas of our coding, but normally have the default DCL CLI available. The beauty of the sys$creprc service call is that you can load up a temp mailbox with a bunch of commands - and then go execute the whole group in one call to sys$creprc.

I agree that relying on the PQL quotas can be problematic. All the quotas are now explicitly defined in my call to sys$creprc!

Thanks for all the suggestions.