Time got changed

[Cache memory and VMS]

Hi All,
Good morning .

I have a problem with one of our VMS production system. The time on the server has advanced by one hour . I checked the accouting and could locate following entry .

BATCH Process Termination
-------------------------
Username: SYSTEM UIC: [SYSTEM]
Account: SYSTEM Finish time: 21-MAR-2007 15:45:32.86
Process ID: 00000000 Start time: 21-MAR-2007 15:45:32.86
Owner ID: Elapsed time: 0 00:00:00.00
Terminal name: Processor time: 0 00:00:00.00
Remote node addr: Priority: 0
Remote node name: Privilege <31-00>: 00000000
Remote ID: Privilege <63-32>: 00000000
Remote full name:
Posix UID: Posix GID:
Queue entry: 869 Final status code: 000480D4
Queue name: SYS$BATCH_OLTBO2
Job name: TIME_CHANGE
Final status text: %JBC-F-JOBDELETE, job deleted before execution
Page faults: 0 Direct IO: 0
Page fault reads: 0 Buffered IO: 0
Peak working set: 0 Volumes mounted: 0
Peak page file: 0 Images executed: 0
␌
SUBPROCESS Process Termination


can I find put which user executed this job? looks like it was a batch job with /user=system.

Thanks in advance ,

regards
Anup

[Volker Halle]

Anup,

according to the final status code, it looks like that batch job has been deleted, before it got executed.

For the future, you may want to enable TIME auditing: $ SET AUDIT/AUDIT/ENABLE=TIME

Someone may still have executed a set time command manually. Are you running DTSS ?

Volker.

[Cache memory and VMS]

Hi Volker ,,
Thanks a lot for the quick reply . Though it shows that the job got finished before execution , I feel that the script has partially executed and has changed the system time .

the following entry from accoutung logs explains this 21-MAR-2007 15:44:08.36 DELETE OBJ_DELETE OLTBO2 OAT_BATCH 206088C0
21-MAR-2007 15:45:53.20 ACCESS OBJ_ACCESS OLTBO2 SPC_BATCH 20608A36
21-MAR-2007 15:45:53.23 ACCESS OBJ_ACCESS OLTBO2 SPC_BATCH 20608A36
21-MAR-2007 15:45:53.23 ACCESS OBJ_ACCESS OLTBO2 SPC_BATCH 20608A36
21-MAR-2007 16:52:01.69 ACCESS OBJ_ACCESS OLTBO2 SDEV_BATCH 20608A33
21-MAR-2007 16:52:01.71 ACCESS OBJ_ACCESS OLTBO2 SDEV_BATCH 20608A33
21-MAR-2007 16:52:01.72 ACCESS OBJ_ACCESS OLTBO2 SDEV_BATCH 20608A33
21-MAR-2007 16:52:01.81 ACCESS OBJ_ACCESS OLTBO2 STG_BATCH 20608A34
21-MAR-2007 16:52:01.83 ACCESS OBJ_ACCESS OLTBO2 STG_BATCH 20608A34
21-MAR-2007 16:52:01.83 ACCESS OBJ_ACCESS OLTBO2 STG_BATCH 20608A34
21-MAR-2007 16:52:02.41 ACCESS OBJ_ACCESS OLTBO2 OAT_BATCH 20608C02
21-MAR-2007 16:52:02.43 ACCESS OBJ_ACCESS OLTBO2 OAT_BATCH 20608C02
21-MAR-2007 16:52:02.43 ACCESS OBJ_ACCESS OLTBO2 OAT_BATCH 20608C02


we can see that the time got advanced at 15:45 . just trying to figure out how did this happen.

regards
Anup

[Volker Halle]

Anup,

the batch process accounting entry is pretty straightforward: job entry 869 in batch queue SYS$BATCH_OLTBO2 has been deleted at 21-MAR-2007 15:45:32.86 BEFORE being executed. All the counters in that accounting entry are ZERO.

If you want to draw conclusions from the ANAL/AUDIT output, all you can say is:

if the time has jumped ahead by 1 hour, it must have happened between:

21-MAR-2007 15:45:53.23 ...
21-MAR-2007 16:52:01.69 ...

The batch job termination was 31 seconds earlier.

Volker.

[Hoff]

What hardware box?

What OpenVMS version?

What ECO kits have been installed?

There are a couple of ways this sort of thing can happen, depending on the box and the version.

What timezone rules are active?

[Cache memory and VMS]

Hi Hof and Volker ,

Below given is the timecange script which got executed .$ if "''F$MODE()'" .eqs. "BATCH" then set verify
$ set noon
$ set proc /priv=all
$ if "''P1'" .eqs. "SPRING_FORWARD" then goto _SPRING_FORWARD
$ if "''P1'" .eqs. "FALL_BACK" then goto _FALL_BACK
$ write sys$output "an error has occured"
$ goto _EXIT
$ !
$ _SPRING_FORWARD:
$MC SYSMAN
SET ENV/CLUSTER
SET TIMEOUT 00:00:10
SET PROFILE/PRIV=(LOG_IO,syslck)
DO SH TIME
CONFIGURATION SET TIME "+01:00:00"
DO SH TIME
EXIT
$ goto _EXIT
$ !
$ _FALL_BACK:
$MC SYSMAN
SET ENV/CLUSTER
SET TIMEOUT 00:00:10
SET PROFILE/PRIV=(LOG_IO,syslck)
DO SH TIME
CONFIGURATION SET TIME "-01:01:30"
DO SH TIME
EXIT
$ goto _EXIT
$ !
$ _EXIT:
$ exit
$


my assumption is tht, somebody by mistake submited the job , but suddenly realized and deleted the entry. By that time time got chaged .

its COMPAQ AlphaServer es40 , os version is 7.3-2 .

does this help?

thanks i advance ,

regards
anup

[Volker Halle]

Anup,

did the batch job produce a TIME_CHANGE.LOG file - if it really ran ?

The accounting data shows 0 images executed, but the procedure is using SYSMAN, which is an image.

Volker.

[Hein van den Heuvel]

>> my assumption is tht, somebody by mistake submited the job , but suddenly realized and deleted the entry. By that time time got chaged .

Nah, that would require an exquisit sense of time(sic), and the job and trace do not suggest too much smarts / planning.

- One should not set the clock for the timechange

- The timechange should not happen on the first day of spring, but on some Sunday morning 2am as selected by the local authorities.

Maybe a time-zone patch was installed too late?

Somebody made an error.
They may or might not admit to that.
I would send out an Email to active users with privs and check auditing.
If time change auditing was not enabled (as per Volker), then maybe using logs from repeated jobs, or timestamps in long running jobs, you could be able to find
at what time the change happened. With a bit of luck trace that back to overly privved users on the system.

Good luck!
Hein.

[Volker Halle]

Anup,

with running V7.3-2, it's about time to use the OpenVMS AUTO_DLIGHT_SAV=1 mechanism instead of home-grown procedures.

Volker.

[Hoff]

That time change procedure most definitely not one I would use, as it appears it would trigger errors. This sequence looks like it will trigger the off-by-an-hour bug, as it does not adjust the daylight and standard flags required for the correct operation of various applications built directly or indirectly using C timekeeping mechanisms. It also looks like it might well trigger problems with the sequencing and scheduling with the queue manager, too.

There is a provided manual switch-over procedure in SYS$$EXAMPLES:, the DAYLIGHT_SAVINGS.COM [sic] command procedure, that handles this case for a manual switch-over between daylight saving time and standard time.

The AUTO_DLIGHT_SAV parameter is the way to go if your applications can tolerate an automatic change-over. Most can, but there are those around that use the current time as a file index or such.

There have been ECOs that cover this, check that you're current on those. There was a bug in DAYLIGHT_SAVINGS.COM a while back, and the fix was made via ECO.

I've a large collection of information on timekeeping posted over at the new HoffmanLabs website, given the ruckus that was caused by the recent US DST changes. Look for the daylight savings time link in the left navigation of the new web site, and you'll pull up a half-dozen technical articles on this topic.

It's somewhat doubtful that the exact trigger will be found here -- unless somebody owns up to issuing the command "hot", or you find some batch job that executed unexpectedly -- so this is likely going to be a "teaching moment" around enabling various security auditing, and around handling the DST switch-over.

And if I can't convince you to migrate to one of the available and documented mechanisms and you're in a position where you must continue to use the locakl batch-based approach (and you can't call DAYLIGHT_SAVINGS.COM from that), the DCL command SET TIME/CLUSTER is a whole lot shorter than those SYSMAN command sequences you're currently using.

There are timezone and timekeeping ECOs for OpenVMS V7.3-2. The system-level timekeeping matters I was thinking of -- and there are a couple of these -- do not affect the AlphaServer ES40 series boxes.

Stephen Hoffman
HoffmanLabs