Is SYSUAF merge possible?

[Art Wiens]

I'm bringing a remote cluster to our datacenter and want to make it part of one of our clusters. There are ~300 accounts on the remote system. I have checked and UIC's don't conflict (phew!). Is there an "easy" way to take SYSUAF info from the remote system disk and merge it into the existing SYSUAF? Any freeware solutions out there?

I know there are many aspects to this cluster consolidation, my question is strictly about SYSUAF info.

Remote sys: v7.1-1h1
Local sys: v7.2-2

Thanks in advance,
Art

[Martin P.J. Zinser]

Hello,

given the fact that SYSUAF is just another indexed file, the following should work:

1.) Make backup copies of your SYSUAF files
2.) Create an FDL of SYSUAF using anal/rms/fdl
3.) Make sure nobody access your SYSUAF.
4.) edit/tpu sysuaf.dat
include all the records from the new cluster
you want to keep. Be sure to remove all the
system/tcpip/decnet whatsoever accounts.
5.) exit tpu
6.) convert/fdl of the new sysuaf file
7.) You should be fine concerning sysuaf

Things to keep in mind still:

* Rights Identifiers
* User Directories

No warranties at all!

[Hein van den Heuvel]

Yes you can merge SYSUAF files,
but the 'target' UAF will accessed for
write preventing and east convert/merge.

So I'd use:
$CONVERT/SHARE/STAT/EXCEP=SYSUAF.XXX sysuaf.dat,sysuaf.extra sysuaf.tmp
$ANAL/RMS/FDL sysuaf.tmp
$EDIT/FDL/NOINT sysuaf
$CONVERT/FAST/NOSORT/STAT/FDL=sysuaf sysuaf.tmp sysuaf.new
$!Last chance to bail out.
$RENAM/LOG sysuaf.new sysuaf.dat
$RENAM/LOG sysuaf.dat;-1 sysuaf.old

1) be careful about search lists and logicals
2) yes this process might loose an (password/last-usage) update, so run it at a slow time.
3) while you are at it... give sysuaf 10 - 50 global buffers
4) The XXX file will receive duplicates like DECNET and SYSTEM and such.

No I do not like the idea of using EDIT/TPU
on the file. I'd much prefer DATATRIEVE or even a DCL READ/DELE/KEY=

Yes you need much the same process for rightslist and vmsmail_profile.

good luck,
Hein.

[richard hoop_1]

hi art,

you might find what your looking for at:
www.kesu.be
look at the products / quai
its a tool to extract and insert in the uaf and saves passwords, so you dont have to reset it for all users you moved.

richard

[Martin P.J. Zinser]

Hi,

passwords are not touched by either my or Heins suggestion either.

Greetings, Martin

[Paul Jerrom]

Here's the routine I used earlier this year. Assumes the 2 UAFs are sysuaf_1.dat and sysuaf_2.dat, and corresponding rightslists are rightslist_1.dat and rightslist_2.dat. Order of the sort is important, duplicate records will be taken from the first of the two files specified.

$ anal/rms/fdl/out=sys$login:sysuaf.fdl -
sys$system:sysuaf.dat
$ anal/rms/fdl/out=sys$login:rightslist.fdl -
sys$system:rightslist.dat
$ create/fdl=sys$login:sysuaf.fdl -
sys$login:sysuaf_new.dat
$ create/fdl=sys$login:rightslist.fdl -
sys$login:rightslist_new.dat
$ set noon
$ sort/nodup sysuaf_1.dat,sysuaf_2.dat -
sysuaf_new.dat/overlay/indexed
$ sort/nodup -
rightslist_1.dat,rightslist_2.dat -
rightslist_new.dat/overlay/indexed

Cheers,
PJ

[Art Wiens]

Call me an idiot (everyone shouts - "You're an idiot!") but how can I get a copy of the SYSUAF's in question? Seems I can't COPY nor BACKUP the file:

$ copy SYS$SYSTEM:SYSUAF.DAT sys$login:sysuaf.old
%COPY-E-OPENIN, error opening SYS$COMMON:[SYSEXE]SYSUAF.DAT;1 as input
-RMS-E-FLK, file currently locked by another user

$ backup SYS$SYSTEM:SYSUAF.DAT sys$login:sysuaf.old
%BACKUP-W-NOFILES, no files selected from $1$DGA31:[SYS0.][SYSEXE]SYSUAF.DAT;*
%BACKUP-E-OPENIN, error opening $1$DGA31:[SYS0.SYSCOMMON.][SYSEXE]SYSUAF.DAT;1 as input
-SYSTEM-W-ACCONFLICT, file access conflict
%BACKUP-W-NOFILES, no files selected from $1$DGA31:[SYS0.SYSCOMMON.][SYSEXE]SYSUAF.DAT;*

Kind of makes me wonder what's on my backup tape? :-O

Do I have to come up minimum or such?

Art

[Martin P.J. Zinser]

Hi Art,

seems your system is really busy ;-) Either try while peopole are at home :->> or give
convert/share a go

Greetings, Martin

P.S. If you do this and work with the backup copies obviously all changes to the original done afterwards will be lost

[Hein van den Heuvel]

>Call me an idiot (everyone shouts - "You're >an idiot!") but how can I get a copy of the >SYSUAF's in question? Seems I can't COPY nor >BACKUP the file:


You idiot! You ask a question in public and then you do not even have the common curtousy, or brains, to read the replies that folks are so kind to offer.

:-) :-) :-)

From my earlier reply:
> So I'd use:
> $CONVERT/SHARE/STAT/EXCEP=SYSUAF.XXX
sysuaf.dat,sysuaf.extra sysuaf.tmp

.... /SHARE ...
Do you see now :^)

to re-iterate my warning: /share will garantuee consistency inside records, but not between records. It is possibly that it will pick up an older record for user "A" but a
newer record for user "Z". Both records wil be intact. even if they, or their neighbours are just being update. Shouldn't matter, but it's good to know.
The same can not be said for the backu/ignore=interlock alternative. If someone happens to be updating the file while it is being copied the whole file may end up useless.
Unlikely, but possible. Why risk that.

Cheers,
Hein.

[Art Wiens]

>You idiot! You ask a question in public and then you do not even have the common curtousy, or brains, to read the replies that folks are so kind to offer.

:-) :-) :-) /<

Wow! That's a bit harsh, I asked you to call me an idiot, not make me cry :-(

Many good suggestions/replies have been offered so far. I did read all the replies, even assigned points! (Gave you a 9!) But I must be honest, unfortunately your opening sentence "Yes you can merge SYSUAF files,
but the 'target' UAF will accessed for
write preventing and east convert/merge." was quite difficult to decipher. I also don't have any familiarity with the CONVERT command.

Anyways, being the cautious sort, I always like to do my testing/playing around away from production environment. My initial thoughts were to get copies of the files and try ALL of the methods listed ... see what might work best for me. I tried copying/backing them up at a few different times, hence my latter post about not being able to.

I do appreciate everyone's help ... I have alot of experience w/VMS, but I've been totally away from it for 4 years (got sucked into the Windows World) ... only recently back at it.

Cut me some slack,
Art

[Antoniov.]

Hi Art,
previous answer seem right and i don't answer you because I agree with allbodies.
Now I think there are a lot of LOGIN in your system, so LOGINOUT process keep lock on SYSUAF.
Hein post is the most right but if you can accept a low risk you coul run follow statements:
$! Raise to close files on error
$ ON ERROR THEN GOTO XIT_LOOP
$! Raise to close files on Ctrl-C
$ ON CONTROL_Y THEN GOTO XIT_LOOP
$! Open without lock SYSUAF.DAT
$ OPEN/READ/SHARE=WRITE SRC -SYS$SYSTEM:SYSUAF.DAT
$! Open my sequential file to merge
$ OPEN/WRITE DST SYSUAF.SEQ
$MAIN_LOOP:
$! Read no lock record
$ READ/NOLOCK SRC LINE /END=XIT_LOOP
$ WRITE DST LINE
$ GOTO MAIN_LOOP:
$XIT_LOOP:
$! Close all file
$ CLOSE/NOLOG SRC
$ CLOSE/NOLOG DST
$ EXIT
Naturally you cannot lock SYSUAF because any user can need to login so you aren't sure to read last version of data.
Bye
Antoniov

[Antoniov.]

P.S.
If you will not conver but you wish merge, you can open destination file as follow:
$ OPEN/READ/WRITE/SHARE=WRITE DST -
SYS$SYSTEM:SYSUAF.DAT
Naturally, source file is located on another node and you need modify OPEN of source statement.
Bye
Antoniov

[Andreas Vollmer]

Hi Art,

I understand that you are very careful in this matter.
Maybe if I can recommend to use following approach:

BACKUP/IGNORE=INTERLOCK SYS$SYSTEM:SYSUAF.DAT node_sysuaf.dat

Do this operation on both systems.

After you successfully created a copy of SYSUAF.DAT then use the previous given instructions.
Even if it will fail the original files are still intact!

By the way, the qualifier /IGNORE=INTERLOCK is very helpful. You may know it from the image backup of your system disk etc...

Kind Regards
Andreas

[John Gillings]

Re: last response...

>Maybe if I can recommend to use following approach:

>BACKUP/IGNORE=INTERLOCK SYS$SYSTEM:SYSUAF.DAT node_sysuaf.dat

> By the way, the qualifier /IGNORE=INTERLOCK >is very helpful. You may know it from the >image backup of your system disk etc...

Andreas, sorry to contradict you, but /IGNORE=INTERLOCK is *NOT* very helpful.

The SOLE purpose of /IGNORE=INTERLOCK is to reduce the severity of the BACKUP ACCONFLICT condition from "E" (Error) to "W" (Warning). It is NOT a magic wand to make online backups. Please don't rely on backups of system disks made using it.

Under /IGNORE=INTERLOCK there is no synchronization with other processes or with RMS when reading the file. At best it will give you a "crash consistent" copy of the input file. At worst you'll get random junk. Although it's rare that the output file is completely useless, it's not impossible. Bottom line is any file copied by BACKUP which generates an ACCONFICT should be considered suspect.

If you want to get a clean, guaranteed consistent copy of an open file (opened for shared access), please use CONVERT/SHARE:

$ CONVERT/SHARE SYS$SYSTEM:SYSUAF.DAT node_sysuaf.dat

the output file is guaranteed to be useable. You won't know if record(s) added or modified *during* the CONVERT will be present in the output file.

[Richard W Hunt]

Well, if you really wanted to handle this while your system was running, you might have to do what I did. This is the summary. You might have some code ahead of you.

1. I wrote a little program that used SYS$GETUAI to extract EVERYTHING about a user including the binary password. This program wrote the information to the DCL symbol table with LIB$SET_SYMBOL. Everything comes back as a string symbol when you take this approach.

2. For each user in the incoming SYSUAF, I translated one record using the above GETUAI and created a DCL script that would add the user to my running system with the same exact attributes as they had in the incoming system. This script writer wrote one ADD_xxxxxx.COM script for each user xxxxxx.

3. I wrote a little program that used SYS$SETUAF to handle the binary passwords. If you have the same username and binary password and if the other password-related items are the same, you have preserved the password. So my script writer included a little run of the password-setter based on the binary values in the incoming account's password data.

4. Here's the part that avoided conflicts by name: I didn't just run the scripts. Instead, I wrote another script that located the ADD_xxxxxx.COM files. Using F$PARSE to extract the name and using DCL string subtraction to remove the "ADD_" I found the name of the incoming user. I then did a quick sanity check with F$GETUAI on the incoming user name. If it was OK, I ran the script. I just didn't run the ADD_xxxxxx scripts that had name conflicts. So SYSTEM, DECNET, etc. were never affected.

5. My situation was such that there were no UIC conflicts either, so I never had to deal with that problem.

I don't claim this is elegant. But it works on a running system without too much disruption. Plus it leaves you the files representing the record of what you added.

Before anyone asks why I would need such a bizarre tool, we have migrated our production server twice now. Both times, we took a backup copy of everything but then had to "update" the copies a couple of times when converstion testing ran into snags. Due to logistics issues, it was easier (on the user community) to update the incoming user data this way. And of course, once you have such a tool, you re-use it when the situation comes up again.

[Art Wiens]

I finally got a bit of time on the local cluster. I was able to CONVERT the SYSUAF.DAT files, the VMSMAIL_PROFILE.DATA files, but the RIGHTSLIST.DAT files are giving me a problem:

$ convert/share/stat/exc=RIGHTSLIST.EXC RIGHTSLIST.WPG,RIGHTSLIST.EDM RIGHTSLIST.NEW
%CONVERT-F-LOADIDX, error loading secondary index 2
-RMS-F-DUP, duplicate key detected (DUP not set)

Remember the local system is VMS v7.2-2 and the remote is v7.1-1h1. Were there version differences between rightslist formats?

I've tried pre-creating the output with an FDL from v7.2-2, converting the v7.1 file to v7.2 format... but keep getting the same error.

Any further suggestions or help?

Thanks,
Art

[Hein van den Heuvel]

Use CONVERT/EXCEPTION=rightslist-dups.dat

This is similar to my first reply with the suggestion about SYSUAF itself.

When the convert is done DUMP/RECORD the
exception file and evaluate its contents
to determine whether you can just smile and be happy, (dup on 1,4 - system !) or whether you need to shuffle some records.
You can probably use DCL to READ/DELETE/KEY
and/or WRITE/SYMB to amnipulate a few records.

Hein.

[Art Wiens]

Except I can't CONVERT the RIGHTSLIST files, it just fails, the exception file is empty. I created FDL's from both and they are different...I attached them if you want to have a look. The .FDLWPG is from the local v7.2-2 cluster, the .FDLEDM is from the remote v7.1-1h1 cluster.

Looks like I can only attach one file at a time...here's the local one.

[Art Wiens]

Here's the remote one.

[Hein van den Heuvel · Accepted solution]

Ooops, egg on my face, I did not read right. You were already using /excep and convert failed during an alternate key. Sorry for jumping the gun.

Considering the samll amount of data involved I would suggest to use CONVERT/MERGE instead of a full-fast convert. The /Merge will tell convert to use plain-vanilla RMS $PUTs to insert a record at a time (in primary key order). I suspect (untested) that the failing records will now land in the exception file as there will be an indentifyable failing record.

A DCL loop with reads from secondary rightslist and inserts to the primary would be much similar.

In the case of the convert/fast the primary and first alternate structure were already build, but then convert discovered unrecoverable inconsistency. What could it do but abort?

An other approach would be to pre-process the data files with $SORT to weed out the dups
on alternate. Not pretty, but straightforward.

Good luck,
Hein

[Art Wiens]

That (/MERGE) works! Thank you.

Now, one last question and I think I'm set:

I seem to be "extremely lucky" with minimal "collisions" in IDENTIFIER values (and NO UIC conflicts!!) with a few exceptions. I have a few occurences where there are hex valued ID's - on one side it's a SYS$NODE_xxx system name and one the other it's one that was created probably for file security. What are the ramifications of creating a new SYS$NODE_xxx ID with a different value?

One last push and we're rolling down the hill ;-).

Thanks,
Art

[Art Wiens]

I did some reading ( :-O ) and found that SYS$NODE_xxx identifiers are created upon VMS startup.

10 points for me!! WooHoo!!

Cheers,
Art