time syncing

[himacs]

Hi  Admins,

We have 2 x86 linux servers serves as application and database servers respectively.There are 5 sec of time difference between them and both are synced with ntp clients.

Application users complaining because of this 5 sec difference, they are facing issue.

Now i want both the servers should have same timings.How this can be done.

I planning manual syncing with ntp server using ntpdate command or modify the time in application server same as db server.

Please suggest.

Regards

himacs

[Matti_Kurkela]

If you have been running ntpd on your servers and you have a 5 sec difference between them, you either have broken hardware, or more likely something is wrong with your ntpd configuration.

Can you please show the output of "ntpq -np" command on your servers while ntpd is running (and has been running for at least a few minutes)? Feel free to blank out the IP addresses if you have to; the other values are more important for troubleshooting.

[himacs]

Hi MK,

Thanks for the response.

Now i found out that ntp configured on both the servers,but in rc scripts the same is enabled only on 1 server.

Both are VM machines.

Last month VM team rebooted these machines and ntp started only on 2nd application machine.

So now we are manually syncing both the servers.Requested for a 30 mins  downtime.

But still i didnot get how 1st server is running with current time.(though 5 sec difference with 2nd app server)

Regards

himacs

[Matti_Kurkela]

Ohh... VMs. That was a rather important part of the puzzle.

Virtual Machines have their own set of timekeeping challenges, quite different from physical servers.

First, if the virtualization host provides the VMs a time value as the VM is started. If the host is configured to sync its time with NTP, this time can be quite accurate. However, once the VM's operating system has started, the virtualization host assigns CPU cycles to each VM running on it according to their varying needs: that means the VM's OS-based clock routines that are relying purely on (virtual) CPU cycle counts may be *very* inaccurate, and the inaccuracy may vary according to the workload of the *other* VMs on the same host.

For this reason, it is very important to have the clocks of your VMs either synchronized to some external time source, or paravirtualized (= the VM relies on the host system clock instead of its own virtual one).

Usually, the VM guest tools include a driver or other synchronization mechanism to sync the guest clock to the VM host clock, to offset this problem. But at least VMware recommends turning this mechanism off for Linux hosts and using NTP instead, with some special configuration so that ntpd can be prepared for wild shifts in the system clock (mainly, the "tinker panic 0" line at the beginning of /etc/ntp.conf).

Please see:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427

This is a VMware document, but other virtualization environments should have similar documets, often suggesting the same or similar settings. The first part is about kernel parameters for older Linux distributions, but near the end there are some important points about NTP too.

On a lightly loaded virtualization platform, you may not see any time drift problems. But when the combined workload of the VMs starts really taxing the VM host, the problems may become much more severe than what you're experiencing now (e.g. a time drift of several minutes or more per day, worst case).