HP UNIX 11: Setting Time...Backwards

[LG Porter]

I have two L2000 servers running HPUX 11.00 in a MCSG two node configuration with a See Beyond application and Oracle 8.17 engine. The time synce on the L-Class servers is out of sync and running 15 minutes faster that the other servers (that is, other N-Class/K-Class servers that the application polls data from in the environment). Is there any none problems associated with moving the time backwards? Is the "date" command the best method for changing the time on a server? Should the appliciation and database engine be shutdown first while the time clock is being adjusted?

[Sridhar Bhaskarla]

Hi,

date is the method to use.

Whether changing the date is going to affect you or not is dependent on your application. Take oracle/application vendor's recommendation on changing the date prior to your decision.

You can use date command with -a option. It will slowly adjust the time which may be the option that suits you. Look at date's man page for more details.

-Sri

[Shannon Petry]

Setting time backwards CAN cause problems with certain applications. If you use "date -a" you can set how fast the clock adjusts itself to your newly entered date.

Most of the HP-UX subsystems will not care about the data going back, but you will obviously note discrepencies in syslog entries.

Time dependand applications like licensing, databases, etc.. are the ones to be most concerned about. Check with your app vendor if your unsure.

Regards,
Shannon

[Michael Tully]

I would suggest you investigate on how to implement NTP. Setting the time backwards *can* cause your system to have problems, and if your using databases it is likely to corrupt them. The safest method is to set the the system time during a reboot in single-user mode.

[James R. Ferguson]

Hi:

You can safely slew the time backwards with:

# date [-a [-]sss[.fff]]

Once adjusted, I urge you to implement NTP. The "Installing and Administering Internet Services" manual, chapter-7 has a very good guide to this implementation:

http://docs.hp.com/hpux/onlinedocs/B2355-90685/B2355-90685.html

I urge you to choose an external source of time to which you can synchronize a local server and in turn, your remaining local ones to it. For a good list of time sources, see:

http://www.eecis.udel.edu/~mills/ntp/clock1.htm

Regards!

...JRF...

[James R. Ferguson]

Hi (again):

Sorry, the url I posted above has mutated. Start here, instead:

http://www.eecis.udel.edu/~ntp/

Regards!

...JRF...

[Shannon Petry]

JRF,

I hate to say it, but even with the -a options I have licensing apps that immediately die thinking that someone is hacking/tampering with licenses.

The -a option is safe for the OS, as is just adjusting the date.

The -a option is not safe for apps that are time dependant.

If I were to take your advice, I would have 200+ customers booted out of applications and loose unknown man hours of work for my company because the license crash means the apps die and no work is salvagable.

Regards,
Shannon

[Brian M Rawlings]

One point of clarification: if you use either the date -a or NTP with a 'drift file' to adjust the time backwards, you should be safe as far as any app or time-critical functions, because you don't actually ever adjust the time backwards.

What you do is slightly slow the rate that time moves forward for that server, so that eventually, actual time catches up.

This is why this method is so widely suggested, rather than dropping the server to single-user to make the change... that method leaves wierd time overlaps and so forth in log files and potentially in other things like Oracle roll-back logs, etc.

Since your server time never actually moves backwards, and never loses a big chunk of time, I consider this method (date -a), or the drift file method (which provides the same basic function) to be the ideal way to move system clocks into sync with each other.

And then, as JRF so adroitly points out, implement NTP and sync up to an atomic clock somewhere. There are scores of excellent posts on this forum about how to do this. Also, for time info and links, check out:

http://www.nist.gov
http://tycho.usno.navy.mil/

Best Regards, --bmr

[Brian M Rawlings]

Shannon -- thanks for the update, I haven't seen apps die because of time drift/correction (if it's slow enough), but if you have, I'll stand corrected.

I just hate all the ugly thing that happen to logs and timestamps...

It may also be that I have always made very slow corrections (15 minutes corrected over the course of a week is a very small change per second or per minute). Presumably, apps that are concerned with this have some threshold that must be hit before they take notice.

Anyway, thanks for the real scoop. --bmr

[James R. Ferguson]

Hi Shannon:

With 'date -a' or a properly configured 'xntpd' or 'ntpdate -B' a *slew*, not a step of the clock occurs. The perceived length of a second will shorten or lengthen but no seconds will be missed.

I respectfully submit that NTP would not be a candidate for use in your environment otherwise. Your comments?

Regards!

...JRF...

[Michael Tully]

Without using the -a option I have seen an Informix database keel over and die a horrible death due to fat finger syndrome by a certain junior system administrator who shall remain nameless. I am not willing to take the risk on something like this, we are lucky enough to have a reasonable maintenance window on Sundays to fix such things. In our organisation Juniors are only allowed to touch certain things. Sorry for the cautious attitude, one bitten twice shy....

[A. Clay Stephenson]

Using date -a (or ntpdate -B) to slew the tiime backwards or forwards is perfectly safe. None of the adjustments, alter the "arrow of time (a physics phrase)" in any way. The system time is always advancing. Slewing backwards is equivalent to somehow moving the server very, very fast and then returning to "normal - earthlike" conditions so that "moving clocks run slow". These relativistic time dilation effects are emulated rather well by the adjtime() system call which date -a or ntpdate -B invokes. I've done this many, many times on boxes running all kinds of license servers and databases and never suffered a hiccup. The application would have no way of knowing that a second is a bit long or a bit short. You should do a man 2 adjtime for a rather good explanation of the underlying process.


On the other hand, if you step adjust the time then it is very possible for transaction number 999 to occur after transaction 1000 - that's rather difficult to explain to a client.

[Bill Hassell · Accepted solution]

15 minutes is a critical problem with related databases, so I would schedule emergency downtime in the near future to reboot everything after ntp has been setup (but NOT started with the start/stop scripts). ntpdate -B will slew the clock and there will still be 86,400 seconds every day. However, ntp will slowdown or speed up each second (a few milliseconds) until the clock is within 128 ms of the server.

Since 15 minutes is a lot of time to slew, it could take 6-10 hours to accomplish and it will be difficult to monitor.

To setup ntp, most large companies have a firewall that is (or can be) synced to NTP servers on the Internet. Setup a simple ntp.conf file that points to the server (hundreds of systems can point to a single firewall/router with no load since the timesync request comes in every 64 secs and then slows to once every 17 minutes when synced). See if the firewall or router is running NTP with:

ntpq -p router_name_or_IP

If it shows the name(s) of it's NTP sources, you're all set. Once all the machines can reach the router, then schedule your reboots and when NTP starts (long before the databases or any other time-critical programs) the date will be step-changed to the right value and then maintained from there on.

[Sridhar Bhaskarla]

The safety is merely dependent on the application. I wouldn't recommend slewing the clock for the following reasons.

1. What if you wrongly specify slewing offset?.

2. If it is a realtime application (say billing for a mobile phone call or for an internet connection), then depending on the offset, the billingtime will be less than the *real time*. It may not matter for one or two connections. How about if there are a thousand connections?

If the above is the case, then take a maintenance window as suggested earlier. Otherwise 'date -a' is a safe command.

-Sri

[Shannon Petry]

Just a follow up for Brian and James, so no points please.

IforLS, NetLS, and FlexLM are network license managers. I can slew time forwards, and it will make no difference to the Licensing server. As soon as I start to move backwards, the license server will kill the licenses.

Logically this makes sense as many served licenses are only valid from DDMMYYYY to DDMMYYYY. If they do no date checking I can keep them forever.

The licensing systems themselves look for time disparity which could allow for expired licenses to be maintained and stop serving when the disparity is detected.

Are there work arounds? Sure there are, but a production environment must be shut down to accomodate them.

Take CATIA for example (CAD/CAM/CAE application used by Daimler Chrysler, Boeing, NASA, and countless others). If I kill the license server CATIA can not perform it's next action. It can not save data. This would cost me everyones hours from the time they last saved till I killed the server by setting the clock at an easy rate of $75.00/hour. In addition I could have simulations and analysis running for days or weeks that die costing hundreds more man hours per job.

Sorry, but I would not want to be the guy who said "just use date -a" and cost someone even $1.00, let alone the potential for this one.

Regards,
Shannon

ps. Catia used to allow for 7 actions to be performed in case of a license outage. I do not know that this is still the case.