I’ve always said if you’re running a script for the first time in a production environment, you’re doing it wrong. Testing is fundamental to technology, yet it is one of the single most frequently skipped processes. Let’s talk about this a moment.
Developing in Production
Through my work, I travel quite a bit. That means I get to meet a lot of different people with varying circumstances on their systems. I’ve even met someone who did all their development directly in production.
They ran a software that ran an assembly line. There wasn’t a test assembly line. Oh yeah, they had checks and validations that all their inputs and outputs were valid prior to ever putting the code into production. However, they had no mechanisms, of any kind, for ensuring that everything worked, except that assembly line.
That’s about the only exception to this rule that I’ve ever heard of.
RULE: You don’t code in production.
Yet, you see it all the time. Especially from DBAs. What? The DBA is the biggest culprit? Yeah. It’s a very common practice for a DBA to just start typing T-SQL in their production environment. There’s a problem. They’re going to fix it. In short, coding, right there, in production. Oh sure, the smart ones will put a BEGIN TRAN and ROLLBACK transaction in place prior to running to ensure that they can recover. However, and I’ve done this, you have to remember to highlight the BEGIN TRAN when you run the code or…
I’m just getting going with my ham radio hobby. I’ve had a Technician license for a year. I’m studying for my General. Anyway, my wife said I could get a new radio, so I’ve set up a new rig. I won’t bore you with the details.
Anyway, I’ve got a High Frequency radio, but, due to being a technician, I’m very limited on the frequencies where I’m allowed to transmit. I’ve tried contacting people, but nothing seemed to be working.
In short, my tests were failing.
So, I went back through the process of validating all connections and settings (in data speak, I checked my code and server). I did limited tests using my car radio (which can pick up HF, although only FM, trying not to bore you here) which enabled me to validate that the core set up was working (I did unit tests). However, I couldn’t get a full contact to work.
Until this weekend. There was a contest running to make contacts through 6 meters (where I have permission to go). So, a whole bunch of hams, who are normally hanging out on 20 & 40 meters, were suddenly transmitting on 6 meters.
I could hear them (one test). So, when I heard one call CQ (ready to talk to any station), I responded (another test). THEY HEARD ME!!!
So I finally had a complete test of my set up. From Massachusetts I was able to speak with Kentucky, North Carolina and all the way to Palm Beach in Florida.
Test passed at long last.
For those interested: KC1KCE
What’s the Point?
Great question. I have two key points.
First, you have to test prior to production. In my ham radio example, I ensured everything I could before I started transmitting. Power was set up. Cables were connected. Grounds were in place. I was on the right channels. I had the system configured correctly. All before I started transmitting. All before I went to production.
Second, your tests have to be done in a way that mirrors production. Now, the ham radio example breaks down here, because, without tons of specialized equipment (and I just don’t have all that yet), I can only test by transmitting, meaning, in production. However, you don’t have to do that (unless you’re the person working on the assembly line). You can set up a production-like environment and run your code, at least once, before you get to the production environment.
Appropriate testing is one of the single best ways you can protect your production environment and your business. Make testing a central part of everything that you do.
I have some opportunities to share with you how to do more and better testing for your database environments. I’m teaching some all-day classes on DevOps. Testing will be a fundamental aspect of everything we do. Here are the two events: