Some years ago I had the opportunity to work alongside a veteran software developer. That was a treat for me and also a way to learn big lessons fast.

I remember being overconfident in my abilities, fresh out of school, and making silly mistakes when all that knowledge had to be put into practice.

I wanted to be quick, and agile, and free! I wanted to get in, fix the problem and move on!

But there was an incident that taught me a valuable lesson.

The server we were managing got hacked and crashed.

Working alongside the Veteran we managed to identify the security vulnerability, fix it and then restore the website within 6 hours. This was a big and popular forum. 6 hours recovery time was much shorter than the couple of days that this usually takes.

Shortly after restoring access, I heard from one of the members saying: “The way you recovered from this and the speed at which you did it is nothing short of impressive. In my career, I have worked for big software companies and none of them have in place such a good recovery plan.”

I could not take much credit for that, so I decided to pay attention to “the Old Veteran” because it was clear now he knew was he was doing :).

The Importance of Backups

We were able to bounce back so quickly because we had backups. Now only that, but we had versioned backups. Meaning we could go “back in time” to before the problem, see what changed and fix it. And then restore almost all of the user data, with minimal loss. Without versioned backups, this process would have been long and tedious and I do not know if we would have been able to spot the point of entry.

This is a happy ending story and here is what I have learned:

1. You always do backups – even if you think you don’t need them.

2. You test your backups – an untested backup is no backup. I have a story here where a client was paying their hosting company for a remote backup system and when the time came to use it, the backups were corrupted and so not usable.

3. You never delete things – you rename them and then archive them – this way you can always retrace your steps back to something that was working

4. When writing software you always, always use source control – which is basically a system that does smart backups of your work that allow you to “go back in time” and fix problems.

A beginner’s mistake- “I am too good for Backups”

As I have said, fresh out of school, I had bright ideas and I wanted to move very fast, but I did not ever have to deliver work that was used by real people, in a real situation, facing potential attacks from real online threats.

When you are prototyping and testing out an idea, it is OK to be quick, because if the idea is bad or not useful, you need to find out fast. But once you have something that you want to build out for the long term, then you need to switch gears and sacrifice reaction speed for being more organized.

I confess that this did not make sense to me for a long time. But as I worked in bigger and bigger projects it became obvious how the “slow work” of thinking of a structure to organize your code, setting up source control and doing backups was actually the fast lane. Why? Because it reduces risk and allows you to easily maintain the project as you move forward.

The opposite of this is working at neck-breaking speed, not “wasting time” with backups or source control, in order to put something on the market quickly. All the projects that I managed or I was a part of, that did not put in the time to be organized, eventually ground to a halt and had to be abandoned or rewritten.

I have done this mistake enough times to learn my lesson: for quality and sustainable work always do backups and use source control.

Client’s point of view – Do backups make business sense?

It is now obvious for me that backups are not just a good idea. But why should you care about them?

It depends on how well you can manage risk and how important is your data and your customers’ data to you and your business.

If you can afford to lose it all, then you don’t need backups.

If you can afford the downtime of having to rebuild your application from scratch, then you don’t need backups.

But in my opinion, good backups are a cost-effective way to mitigate the security and data loss risks associated with running an online business.

Do you have a backup policy in place? And if you do, have you tested your backups lately to make sure that you will find in there what you expect to find?