Redundant systems

UPDATE: This is a two year old article but for some reason feedburner moved it to the top of my rss feed after I tagged it. Still, it's an interesting post so please read on. :-)

Our entire ASP-site went down tonight as our ISP had routine maintenance on the power supply that went completely wrong. That's the kind of stuff that makes my day, knowing that a few thousand students/pupils preparing for their exams right now probably would have taken the opportunity to kick my *** if given.

The whole thing made me think about how difficult it really is to make a completely redundant system. Sure, all of our servers have two power supplies, all of our disk are RAID enabled and all of the web-servers are load balanced. But there is always one SPOF (Single point of failure) that will bite you in the *** when you're least expecting it.

It's possible to create enterprise solutions that comes very near to 100% uptime ("six nines", 99.9999%). But in order to manage this you will probably find yourself in a situation where you will need very expensive solutions such as:
- Mirrored sites with realtime mirroring of data.
- Very expensive enterprise storage solutions (SAN).
- Database clusters.
- Hot swap hardware where components can be replaced without interruption.
- Onsite support 24/7.
- Very clever and competent staff-members.
- Last but not at least; robust software architecture built around enterprise services such as transactions, data consitency, distributed applications and security.

These solutions, often referred to as enterprise solutions, are usually found in business critical systems. And when a system is considered critical for business there is also the will of investing into redundant IT solutions.

But what happens when industries such as ours (education) starts demanding the same level of service from our solution? Education is after all as "business" critical as it get for schools and universities? They certainly cannot come up with the same type of money as a big private company. But yet I can already today see users more upset with our service, it's learning, being unavailable then their internet banking service.

Here's my suggestion; introduce differential pricing on hardware, just as you can find on certain software products. Educational institutions (at least in Norway) have a completely different pricing on, say, Exchange Server then a private company. This effectively lets microsoft sell their products in a market that might not have been able to afford it without large discounts.

Do you think it sounds stupid? Wait and see.