In many cases it can be quite difficult to accuracy keep time on a virtualized operating system due to the complex interactions between the hardware, host operating system, virtualization layer and the guest operating system. In my case I found that running Red Hat Linux 5 (CentOS 5) on VMWare Server 1.0.8 resulted in an unstoppable and rapid slowing of the guest clock.
The obvious steps to take include running NTP to control the local clock. This, however, only works when the clock skews very slowly. In my case, as in many, the clock drifts too rapidly for NTP to handle. So we need another solution. VMWare recommends installing VMWare Tools on the guest operating system and subsequently adding the following to your VMX configuration file:
tools.syncTime = true
This does not always work either. You should also try changing you guest system clock type. Most suggestions include adding clock=pit to the kernel options. None of this worked for me. I had to resort to a heavyhanded NTP trick – putting manual ntpdate updates into cron. In my case, I set it to update every two minutes. The clock still drifts heavily within the two minute interval but for me it is an acceptable amount. You should adjust the update interval for your own needs. Every five minutes may easily be enough but more frequently might be necessary.
Using crontab -e under the root user, add the following to your crontab:
*/2 * * * * /usr/sbin/ntpdate 0.centos.pool.ntp.org
For those unfamiliar with the use of */2 in the first column of this cron entry, that designates to run every two minutes. For every five minutes you would use */5. Remember that it takes a few minutes before cron changes take effect. So don’t look for the time to begin syncing for a few minutes.
For me, this worked perfectly. Ntpdate is not subject to the skew and offset issues that ntpd is. So we don’t have to worry about the skew becoming too great and the sync process stopping.
If anyone has additional information on syncing Linux in this situation, please comment. Keep in mind that this is for Red Hat Linux and the kernel with RHEL5 is 2.6.18 which does not include the latest kernel time updates that may be found in some distributions like Ubuntu. Recent releases of Ubuntu likely do not suffer this issue as I expect OpenSuse 11.1 or the latest Fedora would not either.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427
ntpdate puts unnecessary strain on the public time servers. it also causes problems with clock jump which will confuse all programs which do not handle time jumping backwards (i.e. 99.9% of all programs ever written)
> Ntpdate is not subject to the skew and offset issues that ntpd is.
I don’t see where you get this idea from. This is wrong: if you configure ntpd correctly (i.e. you use sync at boot and correct from then on) you will see no such issues with ntpd.
Anon – If you read the article you would see that everything that you state is already stated there with information about why it doesn’t matter. NTPD cannot handle the drift level we are experiencing and even when synced one minute it will lose the ability to maintain sync after less than two minutes. So your obvious recommendation of syncing at boot time is obvious not going to work.
NTPDATE is NOT subject to the skew issues that NTPD is, it simply matches the time without trying to bring the time into line gently. This is required when the drift is too great as it is here.
Please read the article before you make statements that don’t even relate to the subject matter at hand. This entire article was about dealing with time drift when your suggestions (the starting point of the article) don’t work. You’ve apparently missed the entire point.
Hi Scott,
I read your article (I had to read it in order to reply!)
Your article suggests that ntpdate doesn’t handle issues – this is completley right – it doesn’t handle anything all, it just changes the system time by making it leap forwards or backwards. Any big leap in time, particularly backwards will subtely break running applications.
Rather than use ntpdate (which is a workaround) you need to fix your clock skew problem and then use ntpd. Using ntpdate is a poor solution which puts unnecessary strain on the public ntp servers.
All recent versions of VMware and Linux do not have any clock skew issues. Which versions are you running?
The very latest version of both VMWare Server and CentOS/RHEL (5.4). VMWare Server 2.0.2.
Started having the issue with 1.0.8 on 5.2. We updated to the 2 series (big mistake as almost everyone has noted) as it just isn’t as robust or reliable as the 1.0.x series.
We have migrated all that we can to Xen and will be leaving VMWare’s silly toys behind to go to enterprise level from here on out. We started with VMWare when Xen wasn’t enterprise ready yet years ago so we don’t regret the decision but they just aren’t in the game anymore. They are driven by marketing, not engineering, and it really shows in their products.
Hi all,
Its right to say KVM or XEN is much much better than most VMware products. I would never ever recommend any Virtualization software over KVM or XEN. But for this problem of VMware server and host/guest time synchronization I can tell you that disabling any guest site time synchronization like ntp or chrony (although chrony could be the better choice when it comes to big time drifts) could solve your problem. You have also to make sure that “vmware-tools” are compiled against the actual guest kernel and running correctly (just check dmesg) and that “cat /proc/cmdline” should give you something like “clock=pit”. Further “tools.syncTime = true” should also be set in the vmx guest file like mentioned above.