If you moved the site from one server to a new, faster one, you probably have DNS propagation issues. Anyone who hit your server just before the site got moved got the old DNS settings, which are cached for [usually] 24-48 hours. Everyone else who has no problems either did not have the DNS cached, or has had the cached DNS expire and got the new settings. If the server didn't get changed (as someone said it did), then I have no idea what the problem is.