VMSMAIL_PROFILE and desynchronized new mail counts.

[Richard W Hunt]

OK, before anyone dismisses this topic as trivial, I know about MAIL's READ/NEW as the preferred way to resynch new mail counts. My problem is not as simple as just executing READ/NEW from MAIL.

Short background: OpenVMS 8.3/Alpha, patched within a month or two at most. Several hundred active users. Dept. of Defense rules apply to user management - which is why this isn't quite so easy as it might first look.

DoD requires me to scan my system for certain security call-outs that they don't like on other systems. It is like talking to a brick wall to convince them that a problem in one o/s is not a security problem in another o/s. They aren't that imaginative or capable of understanding.

So I have certain issues to address based on security scans that call out specific conditions. I get audited so I can't lie (and wouldn't anyway). I get flogged monthly for having such a dirty system as to have users with unread mail building up in their accounts. Even though the flogging should be the users who abandon their accounts and do not read their mail.

I get this flogging because I can read the headers of the various MAIL.MAI files to find mail messages still counted as NEWMAIL, whether internally stored or externally stored in a MAIL$long-hex-time-string.MAI file. When I compare the counts from the MAIL file to the counts in the VMSMAIL_PROFILE file (which I also can read), they don't always (in fact, rarely) match. So I need to somehow reconcile the counts.

The obvious way is to make a system job jump in every so often to synthesize a batch job on behalf of each user to run MAIL, run a READ/NEW command, then file that one message that just got read back as NEWMAIL (to preserve it without disrupting a user file).

Well, the tricky parts comes into play with another DoD wrinkle. When the account becomes abandoned, I am not allowed to take immediate action because I must preserve the account for x months in case the user wants to reactivate it. I cannot delete the MAIL file and start over again. (Besides, I'm not sure that doing so resets the VMSMAIL_PROFILE counts.) Once the account reaches that point in its life cycle, I am required by DoD rulings to flag it as having an expired password - which means that the little synthesized batch job won't work. You can't submit a batch job for USER=xxxx if that user's password is formally expired.

So my question is, has any utility been created or defined that will reconcile mail counts for existing users with some unknown number of NEWMAIL entries in their mail files? Or am I going to have to try to sneak in a program that updates the VMSMAIL_PROFILE file directly?

[Hoff]

Are you up for a game of security chicken? Preemptively declare yourself in non-compliance of applicable standard [whatever], and remove or lock down MAIL from the system. Done.

That's certainly far more secure than "sneaking stuff" onto the system.

It also gets the right folks to yell at the right folks. Or (if nobody yells) it tells you that you shouldn't bother with further effort here.

As for your question, there have been various mail and profile-repairing tools over the years, various of which have been available on the Freeware. But there's nothing integrated that does this, short of rolling your own DCL or your own code.

And if this is a cluster, make sure that MAIL is correctly configured or your NEWMAIL counts will be all over the place.

[Hein van den Heuvel · Accepted solution]

Forgive me but I can not find a good reason to keep the count of actual number of messages in the NEWMAIL folder marked with the newmail flag in sync with the suggested newmail count stored in the profile file. Just leave it alone ?

In my mind actually reading the users Email files, or submitting a job on their behalves, is a bigger security and privacy concern than a mismatch in counts. But I suppose Logic does not apply here. Rules must be followed.

If I was forced to get the numbers in sync then I would opt for the least of evils, which is to just update the Profile record.

Personally I would probably use just DCL script to update the word at byte 35 in the user record, after verifying that the word at byte 31 is 1 (newmail) and the word at byte 33 is 2 (size of a word). It is always item #1 in the TLV list.

But the MAIL$ Utility routines do provide a clean way to set that, so why not call MAIL$USER_SET_INFO with MAIL$_USER_SET_NEW_MESSAGES value?

Back in 1995 Rich Johnson (Harvard Community Health Plan at that time) published a C program to count real newmail and update the profile. I'm sure he does not mind re-sharing. See below.

Also below some DCL code snippets should you not be allowed to run a compiler/program on the box... yeah I know... who are we fooling here?

Hope this helps,
Hein van den Heuvel.


$! ---- DCL to read PROFILE file.
$! ----- To set to a desired add something like...
$! $rec[8*(p+4),16] = val
$! $WRITE/UPDATE value.
$ if p1.eqs."" then p1=f$getjpi("","username")
$ if p2.eqs."" then p2=f$parse("vmsmail_profile","sys$system:.data")
$ open/read/share=write vmsmail 'p2
$ read/key=&p1 vmsmail rec
$ p = 31
$ nam = f$extr(0,p,rec)
$ max = f$len(rec) - 4
$field_loop:
$ if p .gt. max then goto done
$ cod_plus_len = f$extr ( p, 4, rec)
$ cod = f$cvui ( 0, 16, cod_plus_len)
$ len = f$cvui ( 16, 16, cod_plus_len)
$ dat = f$extr ( p + 4, len, rec)
$ p = p + len + 4
$if cod.ne.1 then goto field_loop
$write sys$output "Newmail count for ", -
f$edit(nam,"TRIM"), " is ", f$cvui(0,16,dat), "."
$done:
$close vmsmail
$exit

$--------- DCL to mark each message in Newmail as new ------
$--------- Remove the WRITE/UPDATE to just count

$OPEN/READ/WRITE/SHARE=WRITE file MAIL.MAI ! Verify directory
$READ/ERRO=No_mail_today/KEY="NEWMAIL"/index=1 file record
$new_bit = 48 * 8
$new = 0
$not_new = 0
$newmail = f$extr(8,8,record)
$loop:
$if f$cvsi(new_bit,1,record)
$ then
$ new = new + 1
$ else
$ not_new = not_new + 1
$ record[new_bit,1]=1
$ write/update/symbol file record
$endif
$read/erro=done file record
$if f$extr(8,8,record).eqs.newmail then goto loop
$done:
$write sys$output new, " new and ", not_new, " not so new messages found in NEWMAIL folder"
$no_mail_today:
$close file

---------------- c.o.v. post --------------


Article: 126154
From: Rich_Johnson@hchp.org (Rich Johnson)
Newsgroups: comp.os.vms
Subject: re: mail count
Date: Tue, 1 Aug 1995 14:43:51 -0400
Organization: Info-Vax<==>Comp.Os.Vms Gateway

Here's a C program which sets your newmail count to the number of messages
in your NEWMAIL folder which are flagged as 'new'. It does NOT search folders
other than NEWMAIL for new messages.

Works on OpenVMS 5.5-2.

- Rich

================================================================================
| | "All of us get lost in the darkness; |
| Rich Johnson | Dreamers learn to steer by the stars. |
| Harvard Community Health Plan| All of us do time in the gutter; |
| Rich_Johnson@hchp.org | Dreamers learn to look at the cars." |
| | - Rush, "The Pass" |
================================================================================


/************************************************************************
* File: RESET_NEWMAIL_COUNT.C *
* Author: Richard D. Johnson *
* Created: 3-MAR-1994 *
* *
* Function: This program resets the current user's newmail count to *
* the actual number of messages in the NEWMAIL folder *
* which are flagged as 'new'. It does NOT search other *
* folders for new messages. *
************************************************************************/


#include
#include
#include

typedef struct itmlst { /* item list for MAIL$ calls */
short buffer_length; /* length of input/output buffer*/
short item_code; /* item to get/set */
long buffer_address; /* address of buffer */
long return_length_address; /* pointer to length of buffer */
} ITMLST;

ITMLST nulllist[] = {{0,0,0,0}}; /* empty item list */



main () {
unsigned long mailfile_context = 0; /* context for open mail file */
unsigned long folder_context = 0; /* context for open folder */
unsigned long user_context = 0; /* context for user functions */
unsigned long status; /* status of service calls */
unsigned short new_messages = 0; /* number of new messages */
long messages_selected = 0; /* # of msgs in NEWMAIL folder */
long j; /* lcv/current message */
unsigned short message_flags; /* misc. message flags */
ITMLST folder_inlist[] = { /* open new folder */
{4,MAIL$_MESSAGE_FILE_CTX,&mailfile_context,0},
/* folder context info */
{0,MAIL$_NOSIGNAL,0,0}, /* don't signal errors */
{0,0,0,0}};
ITMLST nosignal[] = { /* generic no signal item list */
{0,MAIL$_NOSIGNAL,0,0}, /* don't signal errors */
{0,0,0,0}};
ITMLST select_inlist[] = { /* select newmail folder */
{7,MAIL$_MESSAGE_FOLDER,"NEWMAIL",0},/* folder name to open */
{0,MAIL$_NOSIGNAL,0,0}, /* don't signal errors */
{0,0,0,0}};
ITMLST select_outlist[] = { /* open folder output */
{4,MAIL$_MESSAGE_SELECTED,&messages_selected,0},
{0,0,0,0}}; /* # msgs selected */
ITMLST info_inlist[] = { /* get message info */
{4,MAIL$_MESSAGE_ID,0,0}, /* message to get info on */
{0,MAIL$_NOSIGNAL,0,0}, /* don't signal errors */
{0,0,0,0}};
ITMLST info_outlist[] = { /* get message info output */
{2,MAIL$_MESSAGE_RETURN_FLAGS,&message_flags,0}, /* message flags */
{0,0,0,0}};
ITMLST setcount_inlist[] = { /* set newmail count */
{4,MAIL$_USER_CREATE_IF,0,0}, /* create user profile */
{2,MAIL$_USER_SET_NEW_MESSAGES,&new_messages,0}, /* set newmail count */
{0,0,0,0}};

if ((status = mail$mailfile_begin(&mailfile_context,nosignal,nulllist)) ==
SS$_NORMAL) {
if ((status = mail$mailfile_open(&mailfile_context,nosignal,nulllist)) ==
SS$_NORMAL) {
if ((status = mail$message_begin(&folder_context,folder_inlist,nulllist))
== SS$_NORMAL) {
if ((status =
mail$message_select(&folder_context,select_inlist,select_outlist)) ==
SS$_NORMAL) {
info_inlist[0].buffer_address = &j;
for (j=1; j<=messages_selected; j++) {
if ((status =
mail$message_info(&folder_context,info_inlist,info_outlist)) == SS$_NORMAL) {
if (message_flags & MAIL$M_NEWMSG)
new_messages++;
}
}
}
mail$message_end(&folder_context,nulllist,nulllist);
}
mail$mailfile_close(&mailfile_context,nulllist,nulllist);
}
mail$mailfile_end(&mailfile_context,nulllist,nulllist);
}
if ((status = mail$user_begin(&user_context,nulllist,nulllist)) == SS$_NORMAL)
{
printf ("Resetting newmail count to %d...\n", new_messages);
status = mail$user_set_info(&user_context,setcount_inlist,nulllist);
mail$user_end(&user_context,nulllist,nulllist);
}
}

[John Gillings]

Richard,

First step I'd suggest is work out why the new mail counts are going bad. The most obvious reason is a cluster where each node has a different VMSMAIL_PROFILE, or a different mail directory. For example, the SYSTEM account on a cluster with multiple system disks will always get drifting new mail counts. You may be able to fix this.

I'd also agree with Hein that an unsynchronized value is no big deal. An alternate to synchronizing the value would be to set it to 1 if you suspect there is new mail or 0 if not. Apart from bragging rights about who gets the most mail, most of the time all you care about is "you have mail" or "you don't have mail". 1 or 0.

(a wise man once explained to me that in computing there are only three numbers you should ever write - 0, 1 and infinity).

[Richard W Hunt]

If this were a commercial site, I would agree with all of you that this falls into the lap of the negligent users, and I would not care that the two counts are out of synch. But in a DoD shop, the military mind expects the low-level users to not have the ability to do anything (or understand anything?) beyond the absolute minimum required for the job. They expect the system administrator to keep a clean system. Which is why in some of my e-mails, my "personalized" signature includes the job title "Senior Systems Janitor."

They also require me to wring every possible nugget of conflicting information out of the system to see whom they can blame for something. But as the first line of defense for user issues, I'm the one who gets flogged when a "finding" occurs. And my problem also stems from the fact that the person who wrote the original test standards knew just enough about OpenVMS to be dangerous (in addition to being aggravating.)

I'll see about the callable MAIL interface routine as noted to set mail values. I had forgotten about it because I haven't touched that interface in (literally) 10 or more years. We have BASIC available on our TEST/DEV boxes, so I can build something there, test it, and migrate it to the production systems. I also could probably write something in DCL to handle the problem of updating the VMSMAIL_PROFILE records if I had to take that approach. That's how I read the records now anyway.

Sadly, the freeware disk is out of bounds most of the time, though perhaps I can copy something from it and re-implement using DoD terminology and event logging up to their standards. Good thing the money is good here because otherwise, I'd tell them where to put their security findings.

As to why the counts get desynch'd - damned if I know why. We used to be a cluster before we split for security purposes. Now we have separate test and production systems. I suppose some of the bad counters could be due to past problems. If I can at least minimize the count mismatches for the active users, maybe I can sneak the "account nearing end of life cycle" cases past the security guys.

I am reminded of Tommy Lee Jones's line from Men In Black. "We're from the FBI. We don't have a sense of humor." I think that adequately describes what I face when dealing with certain members of the network security team.

Thanks for your suggestions, folks. I know my problem is actually more due to a management overconstraint than a true technical issue, but that doesn't stop it from being quite real to me. Your willingness to help in an off-the-wall case is appreciated.