Time for a Quick Rant

This is an actual quote from what we can only assume is a functional human being:

The database is very big so we stopped taking backup’s.

Eight lords a leaping are you kidding me? Seriously! Seriously? By the Great Gu and all the Valkyries in Valhalla, you stopped taking backups of your PRODUCTION database because it was “very big.” And I’ll put down Brobdingnagian stacks of cash that “very big” in this case is probably 200-500gb or at worst 1-2tb. People, assuming you have enough brain stem intact to regulate breathing, you must know, you must by all the sparkly vampires in Twighlight KNOW that you need to have backups. Right? I mean, nothing ever goes wrong on this shiny marble we call Dirt, does it? No one would EVER just run a DELETE statement in production without a WHERE clause, would they you hairy bottomed tree climbing mouth breather? And I’m sure that, if it happened, you could just blow magic unicorn powder at the server in order to get the company’s billing list back, right? Because without that backup, you’re relying on the undivided attention, and total positive intent, of Odin and Freya to ensure that you never, oh, I don’t know, lose power causing the rocker arm on the disk to bash down repeatedly on the platter like Thor’s Hammer on an Ice Giant’s head, with similar results for your database. Until you have so much data that EMC hosts your company’s holiday party, for free, there is enough disk space, somewhere, to take a backup of your database.

Now, get out there and get it done. Don’t make me travel to each and every one of your places of work with the lead-weighted hickory learning bat to lay some education up side your beanie holder. Please, just take a backup.

I really mean it this time. Backup your database
I really mean it this time. Backup your database

35 thoughts on “Time for a Quick Rant

  • In the 1-2 TB range this is crazy, BUT I have made the recommendation to stop taking backups before. At a former employer, we had a 55 TB database with 3 TB of transactions a day (that could be recreated from source systems). It probably shouldn’t have been in a RDBMS, but it made no sense to take backups, because there was no way they were ever going to restore. But this is a .01% exception.

  • Hey Joey, I’d agree. 55TB is way too large to realistically backup, at least using standard mechanisms. But, as you say, it’s an edge case. People far too often use edge cases to justify an action when they’re no where near being in the same situation.

  • I’d like to know 2 things:

    1. How much do you charge for lead-weighted hickory learning bat sessions. I know some people that need that experience 🙂

    2. Where are you ordering your magic unicorn powder? I’m all out

  • byron

    not one, but two of the best things i’ve read this year:

    “you must know, you must by all the sparkly vampires in Twighlight KNOW that you need to have backups…”

    “Until you have so much data that EMC hosts your company’s holiday party, for free, there is enough disk space, somewhere, to take a backup of your database.”

  • Great post, Grant.

    One thing I always point out to people who say things like “my database is too big to backup”, is that backups are not the same thing as business continuity.

    So while they may not run SQL Server backups, they might be able to better ensure business continuity at some other level, such as simple SAN duplexing. Again, that’d be really expensive for such a large database. But serious, do you really want ZERO recoverability?

    And as Joey mentions, you might have to use non-standard approaches, such as partitioning the database, then backing up only the most recent partition.

    There are lots of ways to skin this cat, but to say “No, I just can’t do it” seems like the laziest of all answers.

  • last time I had a database that was too big to back up on the box it was on, I striped it to multiple boxes (keeping track of where things were, of course) Interestingly, it was in a DR situation. I didn’t see the original post, but as Kevin said earlier, there are many ways to skin this cat. This would be one of them.

  • I can’t believe that quote would come from a DBA, unless he’s been tenured (can that even happen?). Business continuity aside, at least discuss with the business what data is critical (we will die without), discover what data is static (config tables, translation tables?), and try to come up with some sort of backup. I’d really like to know how big “very big” is?

  • I’ve heard this as well over the years. While large is always relative, as has been pointed out by Joey, unless you’re in some egde case (and in his case, the data could be reconstructed – but I’m sure painfully), you need backups.

    No one thinks about the “oops” factor. What happens if your genius storage or Windows guys installed a new HBA driver that corrupted your data? It’s not just the DELETE or UPDATE without a qualifying WHERE clause (and even then …).

    The problem is people rarely think about nearline data – that is, the stuff most frequently used/updated vs. archival or read only data. Out of a 55TB database, is 1% active? 10%? 50%? You devise its backup scheme and underlying disk structures accordingly. All of a sudden you’re not dealing with Planet Earth, but maybe Manhattan.

    The best excuse I’ve heard to date is, “We have clusters, we don’t need backups.” No, really.

  • But what if I have clusters and you, then I really don’t need backups. Ha!

    I’ve seen people say that they don’t need backups because they have RAID storage. I’ve also had people just flat out say, “How often do you need backups.” Which is frightening as hell. I’ve only really needed them a few times in serious crunch times, but, oh my, did we really need them.

  • Nice rant Grant. It’s amazing how many companies out there don’t have backups let alone tested them. I wonder if there is a list of articles about companies that went out of business because they didn’t have a database backup to restore.

    I cringe every time someone tells me they haven’t tested their disaster recovery plan.

  • Dave Schutz

    As I think Alan Hirt was alluding to, file group backups. At least backup up the important parts of the data. One way to get a skinny cat.
    And test those backups and restores.

  • Scott

    Yep, out of the two dozen or so clients I’ve had it’s the same issue I have had to deal with, ie – database becomes so large, and too critical to the business it becomes a “monster” that people are too scared to do anything with; including any form of backup.

    Just be glad that us DBA’s exist and can do something about it. ;o)

    S.

  • Mark Walters

    My usual here is no one ever lost their job for failing to take a backup. But you will lose your job if you cannot get the business back up and running (restore).

  • Jim McCarthy

    If he is so worried about the disk space and how big the DB is, I have his solution. Stop SQL services, delete the large DB, and move on..voila..disk space recovered. This also gives him plenty pf room top store his resume, because it better be up to date.

Please let me know what you think about this article or any questions:

This site uses Akismet to reduce spam. Learn how your comment data is processed.