Configuration Set Time - Jumped a day

[Zeni B. Schleter]

Two ES47s in a cluster , OpenVMS v7.3-2.
Right after midnight (00:22) , I have been submitting a job to force the cluster time to stay in sync. I used SYSMAN and the commands:
Set Environment/Cluster
Set profile/Priv=(oper,logi_io,syslck)
Configuration Set Time
We have used this procedure on various OpenVMS hosts sinc 1991 without a problem.
This past Friday , it forced the clock ahead a day and synced the cluster to the new time.

This caused a lot of problems as can be imagined. I have removed the procedure. It was supposed to be a safety net if the connection to the Timeserver was ever lost.

This system that is the master had been rebooted the previous weekend. We are fully patched for VMS v7.3-2. I have heard of this happening before but I cannot find any postings now. Does anyone remember any details about this kind of occurrence or can direct me to a site for more information?

[Hoff · Accepted solution]

If SYSMAN let you set the time with a time server active, it would appear you might have hit a bug in either SYSMAN or within whatever software is synchronizing with the time server. I would have expected a TIMENOSET or similar error.

A one-day jump is unusual, and not something I've seen particular reports of.

What is the time server?

Do check the SRM firmware and the MBM time values. The SRM and system time are maintained as a delta of the MBM base time. (There are details on the MBM in the FAQ.) I'd tend to set the MBM to UTC, and leave it alone. From the MBM, the SRM derives the time value, and your scheme loads the time values into OpenVMS.

Do also check for ECOs for whatever software is managing the system time.

I'd encourage choosing NTP or DECnet-Plus DTSS or manual synchronization means; use one, but not a combination of mechanisms. (NTP has checks to ensure the time value is within a specified range, as does DTSS.) DTSS has the ability to drift the time, where the SET TIME, SET TIME/CLUSTER and SYSMAN commands will all chunk the time forward or backward without regard to what the applications might see.

As for a safety net, check for unexpected time values, or -- if you really need a certain time -- look at a WWV or GPS or similar time-based receiver with an NTP server. I'd not try to correct these values without some sort of logging and/or without some sort of manual reporting and manual intervention.

Look in the synchronization logs for whatever product is keeping track of the time, and see if anything looks odd. Also look to see if time-change events are logged. Look at the times displayed within the local manual time synchronization procedure, and determine if the values displayed there look reasonable. If there are no values displayed, I'd add them.

Any power failures or batch restart sequences or multiple parallel jobs or blocked batch queues?

And if you want to maintain your own time, I'd probably use SET TIME/CLUSTER -- the underpinnings are the same SYSMAN pieces, but the command is directly accessible from DCL. The command was undocumented for a while, but is back in the manuals. If I could drift the time, I would -- chunking the time around can cause some real weirdness in applications.

I'd replace the manual synchronization process with NTP or DTSS -- mixing local management and automatic management or such is not something I'd recommend.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1036181

[Zeni B. Schleter]

Multinet's NTP (or XNTP) is in use. The time service is provided by a windows server.

The Configuration Set Time was used to keep the members of a cluster in sync for the job controllers sake. I will try the SET Time/cluster command - interactively.

I check the status of NTP daily (M-F). We do not manually adjust the clocks.

I will print off your response and study it. Lots of good reading.

[Bill Hall]

Zeni,

Follow Hoff's advice, run ntp on each member of your cluster and stop using SYSMAN to synchronize the time. Configure each cluster member to use the Windows server as their time server and each member of the cluster as their peers. With all of the cluster members being peers, they should synchronize amongst themselves if the Windows time server breaks or becomes un-reachable.

I've never seen even one of our clusters fall out of synch with that ntp configuration.

[Hoff]

If you are invoking the cited SYSMAN CONFIGURATION SET TIME command, then you are resetting the running system time using the value read from the SRM console BB_WATCH (TOY) clock.

On the AlphaServer ES47, ES80 and GS1280 series, the SET TIME operation calls back into the SRM console, and the console applies the console-local delta value to the "real" system time from the MBM and ([magic happens here]) you now have a new OpenVMS system time, without regard to what the value was when you issued the time. Forward or backward.

There are some specific processing steps taken with the MBM time, within the system.

There are timekeeping details in the FAQ; the latest edition is over at HoffmanLabs. In particular, there are specific time processing steps taken to manage and maintain the MBM time within the AlphaServer ES47, ES80 and GS1280 series systems.

Do check with Process for any MultiNet NTP ECOs that might exist for that package.

[Zeni B. Schleter]

Thanks for the input and the explanation about the MBM. I vaguely remember that the Configuration Set Time was needed because it set some register that related to the Year of clock that was not set correctly after the first of the year on systems that were up for over a year. I had seen this occur.

I have remove the SYSMAN configuration procedure. I will leave it to NTP to keep the time current.

Again thanks.

[Zia_Ahmad]

This problem surfaces if time is synched via a SYSMAN command on the LAST day of the month [yes, I got bit].
I believe there is firmware fix for it.

[Andy Bustamante]

If you've had time issues with a Windows system, make sure it provides NTP and not SNTP. Check your ntp logs (I'm not familiar with Multinet). If you have an NTPQ command available, use that to verify you're getting an NTP time from the Windows system.

If you don't have a valid NTP response, you can use either a public time service, purchase a valid time (GPS signal) network device, or simply select one of your Alphas as the Master time source.

Andy

[Volker Halle]

Zia,

I have found, diagnosed and reported exactly this problem back in APR-2005. At that time I had been told, that the problem should be fixed in Firmware V7.0.

FWIW,

Volker.