Instead of working, I blog.

One of the biggest problems DBAs face when trying to cleanup an environment is fighting the user base. We're quite often forced to limit access they've grown used to and even to institute policies the limit what they can and can't do within their code.

Now, here I'm talking about SQL code mainly, but it also stretches to things like deploying new reports and the like.

I call this post the speed of stupidity because most of the problems that need to be cleaned up happen because users get an idea and before they even consider whether it's a good idea or not, or whether the way they thought of to do it is the way it should be done, or whether somebody's even already doing it, they bang out the code real quick and push it to the server... where it proceeds to do its particular brand of damage. And this is really the hardest thing to control in any company where people have had full reign to do whatever they want whenever they want. And it's really hard to explain to them and to demonstrate to them that they're doing more harm than good. And what's worse is that they typically have the FULL backing of upper management.

And management is where it gets tricky because quite often it's the directors and VPs who are pushing their people to produce this code right away. They honestly have lost their ability to reason the logistics of their business. They never even stop to think of performance on the server so it never crosses their mind that there may be a reason to wait. They're so concerned with the fact that they had what they consider to be a good idea and they're now tunnel-visioned on that goal. Not only that, but at this point they've actually convinced themselves that their business can't last another day without this code being in place. This is where that reasoning breakdown comes in. Do you honestly mean to tell me that your company has grown and made money and done solid business for 15yrs, but it's all going to crash down because you don't get this new report by noon? Do you really think this report is that important? Is it important enough to bring the other reports to a halt?

And that's at the heart of this post. It takes a while to architect a good report and get the code right and test it and tweak it, but you can be stupid in an instant. This is what DBAs have to fight. We have to find a way to convince upper management that they can push code into production just because it crossed their minds. They have to learn to slow things down and think them through. It's still ok to give their people assignments to write reports, and other SQL code, but they have to understand that there has to be a process to not only make sure that it's going to be done well, but also that it hasn't already been done by someone else.

And I'm telling you, that's a hard attitude to change. So the next question is, how do you go about changing that attitude? Well, it happens very slowly. You first have to determine the problem. Then you have to determine a fix. Then you have to gather stats from the server. And sometimes depending on the nature of the problem, you could need to gather the data for just a day or 2, or a couple months even.
If it's SQL code, you can code your fix and run the 2 blocks of code next to each other and log the performance differences. Then you can pull stats on how often that code gets run and show a definite improvement over a block of time. If that code is causing deadlocking, or blocking, or just other general contention you'll also want to log the amount of time you spend dealing with these issues. This is what can take a month or 2 because you'll need a large enough sample to be conclusive. Then when you've got the performance gains charted out, you can also list the time you spent dealing with the issues and present that to management. It's really hard to ignore the issue when the numbers are right in front of them. Managers respond really well to hard numbers and if you present them with the hard numbers that prove how much time you spend on issues that could have been avoided by following a process, then they're much more likely to listen. And there could be some other factors as well. These issues tend to stack up on each other and can effect SLAs and application uptime. So these are other factors that should be presented if they apply.

When you take these numbers to the manager you'll want to present it in a way that more helpful than offensive. You know he's the one who's been allowing this to go on, and he does to. So there's no need to rub his nose in it. Just present it as a way to solve a problem that's costing the company money. Look, here's all the time I've spent working on these issues in production that could have been avoided had there been a process to review the code beforehand. A manager would be a fool to ignore this level of proof that things need to change.

Look guys, I know having that new idea gets you excited and you want to see it materialized as soon as possible. But that doesn't mean it should be. Take a minute and think whether the company will really fold if your code isn't pushed into prod right away. What's the real business value to what you want to do? Will it really help you make decisions or help you do something you really need, or is it just nifty to be able to do? Don't waste time and resources on something that's just nifty to know. It has to actually provide something useful to your business.

OK, I'm bordering on rambling here but you get the idea. Let's slow down and start doing things right.