I use a DNS provider with fail-over functionality. If their DNS servers find the main site unresponsive, the DNS records are automatically switched to a backup server and I receive alert emails on two different email addresses.

How does that work with user-generated content? All of my data is stored in MySQL, so if the main server is unresponsive then so would the database.

Is there some way to sync MySQL across servers in real time without a major performance hit?

I run a multi-master database cluster with MariaDB and Galera Cluster. All select queries are to the nearest master and have the same performance as with a single MySQL node. Inserts, updates and deletes have to be confirmed by all nodes, but for most sites these operations are rare compared to the select queries. Therefore the practical performance is about the same as with a local database in my experience.

I run a live chat service for support on all sites. if that fails to appear then I k now something is up. I used to have it making a sound on error, but we like to sleep when we are supposed to :-)

In my case, I really need something to wake me up in case of emergency :-( The server provider claims that they monitor for problems 24/7, but obviously that was BS.

Monitoring for problems and fixing them are two different things. They may monitor, but it may take time to fix it. I'm not defending the host.

There are various independent monitoring services with differing levels of alert, but they don't fix the problem.

There's good advice in this thread.

I have a local solution running that pings my server and sounds a warning if it times out.

But the Internet flashes like a starry night and it was going off too often, especially here in a rural area where they are constantly upgrading.

@engine, I've forgiven my host company. They began using WHMCS for their ticket center this year, and apparently it didn't alert them when I submitted a high priority ticket. So it reeeeeally sucked for me, but it's not really fair to blame them when it's not their fault.

@Kendo, I have the same problem here, unfortunately. Rural area with slow internet, I've just gotten accustomed to streaming TV and movies that are blurry and blocky :-/ The best speed I get is 3MB, but it regularly drops to < 100k (especially late at night). And I have to reboot the router at least once a week.

@lammert's advice is gold, but (a) I have no clue how to set it up, and (b) at this point I can't afford a second server. My income has been going down steadily for a few years, and this crash will cause a long term dip, too. So honestly, a $7 option like UptimeRobot or Pingdom are closer to my alley :-(

I found Pingdom.com, but the explanation of the company is kinda vague; like, their site has a ton of information but doesn't actually say anything. It's $15 /month (or $120 /year if you pay annually) and I think it does what I need? But there's no explanation of the difference between "Synthetic Monitoring" and "Real User Monitoring", and I have no idea what "10 Uptime, 1 Advanced, 50 SMS" means.

I have been using Pingdom for several years with a managed VPS. It works well, I sometimes get a text message in the middle of the night (good thing my host doesn't have issues too often). What you need are the "Synthetics" (URL checks every minute or less often - this is the uptime report, and also they have a newer Page Speed feature checking your URL every 30 minutes). The Real User Monitoring seems to be more like a metrics solution - I haven't used it.

Although I'm not quite ready to stop using server monitoring, I must say that in most situations when there's been an issue I had to wait for the host to solve it anyway.

I've used Hyperspin for many years.

They have servers all over the world, so when one of their servers detects an outage, a very different location and server does the second check.
This really helps sort out false alarms (is it my site or the internet).

They also have a feature that will look for content on a page.
So stick a weird sentence into a database, make the test page pull the data from the database, and so if Hyperspin does not see your db-loaded word, you know your systems (server, mysql, web server, etc) are broken somewhere.
[hyperspin.com...]

But there's no explanation of the difference between "Synthetic Monitoring" and "Real User Monitoring", and I have no idea what "10 Uptime, 1 Advanced, 50 SMS" means.

Could be communicated more clearly, but it's easy enough to find: 10 uptime monitors (different endpoints), 1 Page Speed or Transaction check ("advanced"), and a quota of 50 SMS messages (per month, presumably). You'd need a Zapier connection to another service for phone call alerts though.

Real User Monitoring is for monitoring performance on the server and/or client side.

I prefer UptimeRobot myself (and they do have voice alerts if you're on a plan, which I'm not), but I only use it for non-critical endpoints. The critical ones are managed by my DNS provider, much like lammert's set-up (except I went for master-slave replication; not a lot of writes and too much cross-continent latency), so when a server goes down it only affects some users, and only for a couple of minutes as traffic is automatically rerouted.

Alternatively, maybe there's a smartphone app that can ping your server and buzz you when it's down?

The more I think about it... I'm about to pay for a separate back up and pay a monthly fee for this uptime monitor, I could probably pay slightly more and have a (slower) VPS set up to mirror the main server and act as a fallback, then set up a cron to ping my main server every 5-10 minutes and email me if the ping fails twice in a row. I think that would give me a lot more options for close to the same price.

A ping can succeed when the web server or database server is down.

I have a small script www.example.com/health-check.php which does a few database queries to check if the backend is working. It then sends either the text "ERROR" or "OK" back. If the result is OK, both the front-end and back-end of the site is working. The checking server needs some logic to process the result of this query. You could probably implement this with curl and a few lines of bash script.

In my case the result is interpreted by the DNS provider remotely and by haproxy locally which has this kind of site check logic built-in.

Replication !== backup. After all, if you make a mistake on the main server, it's instantly copied to the fallback. You'll still need backups.

And the health check (and alert) would ideally be connected to the fallback mechanism, so you wouldn't need the cron job.

Hello,

If you have more than one server, you can create a cron job (for example), to make them ping each others ,(or to fetch a given page, a server can respond to ping, but still be frozen), then if the script can't reach the other server, it sends you an email, and you can certainly configure your phone to warms when a new email comes (choose a dedicated email box).

I use PRTG Network Monitor from Paessler (free version) that monitors all my web servers and send me texts and e-mail if it's unable to connect to them (server down) or expected string not displayed (issue with CMS or databases). A little more difficult, but you can also have it trigger events to restart services or even reboot servers.

That's all fine and wonderful, but what if the data center is down? According to PRTG, everything is working, so no alert to send. Or if the data center's power is down, the server it's on isn't running, so it can't send an alert. Or if the DC's internet connections are down, any alerts can't go out.

So for those cases, I use Uptime Robot, also free for up to 50 monitoring actions (web sites, ports, heartbeat monitoring) with e-mail notifications. They have a PRO plan that's based on the number of monitors you want, which also includes text or phone alerts. (The free version checks every 5 minutes, while the PRO version checks every minute.)

The combo of internal and external monitoring works great. Besides alerting me to when something goes down, they've also saved my butt more than once when I made an update that screwed things up and brought sites down.