System time not syncing via internet

[Colin Law]

On a 12.04 system, in System Settings > Time & Date I have selected to
set time automatically from the internet.  When I reboot it syncs but
it then drifts a couple of seconds each day so whatever mechanism
there should be to keep it in sync is not working.  The machine is
permanently on so after a few weeks it gets noticably out.

Any suggestions what might be going wrong?  I know I could add a cron
task to sort it but I would like to find out why it is not working.

Colin