Monday, January 26, 2009

The courage to say the impossible

I was interviewing a guy the other day for a DBA position I have open and I like to ask questions that help me determine how much someone's actually worked with SQL. And one of the ways I'll do that is to ask them how to perform an impossible task and see how they deal with it. In this case, I asked how he would restore a single table from a backup.

He thought about it and went through different scenarios, all of which I kept narrowing down until he was forced to deal just with the backup file itself with no other parameters. He kept trying to think of the flag that he would use to just pull a single table out of that file and he just couldn't think of it. He finally said that he didn't know of a way to do it, but he was willing to learn.

He failed that question in a couple of ways. First and foremost, he had to thing too long and hard about something he should've known for as long as he's been a DBA. So there's that. 2nd, he failed because he didn't have the courage to tell me that it wasn't possible and that as a user, I'm screwed. The data can't be recovered that way. Now, this is a very valid question and tells me a lot about a DBA because we're quite often called on to deliver bad news and if you're not able to do that, then you may not be as strong as you thought.

What he needs is the courage to stand up and tell me that what I'm asking for isn't possible, but he was too caught up in the fact that he's in an interivew and I wouldn't be asking a question like that if I didn't know of a way to do it. I actually explained all this to him and he agreed and said that he thought it was a trick question, but didn't want to say anything.

I've been in that position before where I was thrown by a T-SQL question like that. And initially I reacted the same way because there are so many pitfalls and neat little tricks in the sql lang that I could easily have missed something really big. Or there could have been some obscure legacy trick that still worked. So I get where he's coming from. I really do. However, backup syntax isn't like that at all. It all fits on one page so it's a lot easier to lookup.

And not to go off on too big of a tangent, sales guys are the same way. They hate to say no to a client. They feel it's the worst thing they could do, but it's not. The worst thing they can do is to make promises they can't keep. I saw this at a gig I had about 4yrs ago a lot. The sales guys were always making promises and then expecting the devs to deliver on time. It's messy, and it promotes nothing but crap code. So while they always made their deadline the software was so buggy and so poor with concurrency, it spent more time down than anything. So while they met the letter of the contract, they've greatly harmed their reputation and pissed off the client because now they've gotta take a couple more weeks to fix issues. And that could really harm the client's data as well. Because you could have to make changes to the underlying schema and now you're in the middle of a conversion right after you go live. So I realize you don't want to have to tell a customer no, it's best to tell him the truth. As much as I'd like to get this entire app architected and fully written by this time next month, that's just not realistic. Why don't I get back to you on when we could have a complete product ready, or we can work out a release schedule and we can get you the most critical functions right away and then bring others online as they become available?

So while nobody wants to say no, or that's not possible, or your data's gone for good, it's necessary sometimes and your customers (whoever they are) will appreciate your honesty. And they'll appreciate that what you give them will actually work when it's released.

And by all means, if you're interviewing and you're put in an impossible situation, don't be afraid to say it's an impossible situation. Acknowledge to the interviewer that you realize your choices are limited and that your actions will be factored by that limitation. Then present them with the possible solutions and leave it at that. In this exact case I gave above, the answer would have been something like this:

Well, you can't restore a single table from a native sql backup. If you had some kind of 3rd party product like LiteSpeed or Red-Gate or Hyperbac I could do something for you. But as it stands, you have to restore the entire DB. If you were lucky enough to at least have filegroup backups We might even be able to get away with a partial restore if the table's in the right filegroup. But the situation as you've outlined is pretty bleak.

That answer tells me a couple things. It tells me that you know right away that the task is impossible. So you've worked with sql backup before. It also tells me that you know the situations where what I'm asking would become possible and you've done it enough to know those other solutions immediately. And it shows me that you're willing to be honest with me about my situation. That kind of answer will win a lot of points because if I actually had this problem, you would be the one I wanted to clean up my backups going forward because you've clearly been there before and are quite capable of putting me in a better position. And as an interviewer I may or may not realize that's what's going on, but I'm digging you pretty well right now.

And don't be afraid to tease a little. I had an interview once where they were having problems with SQL memory. They wanted to access more than 4GB, and had the RAM installed, but couldn't get SQL Server to recognize it. I gave him some guidance on how he would see that memory used in SQL and he said that it was perfect for his scenario. Then he asked what they did wrong and how to fix it. I said, well sounds like you need a DBA and if you bring me aboard I'll be happy to help you with that.

So there ya go. There's a little how and why on answering interview questions. There are ways to answer questions that tell them you have experience and ways to answer that don't tell them much about you at all.

Good luck.
Friday, January 23, 2009

MSSQL is not Oracle

We're having a problem with BOE right now and our BOE admin has been on with their support tech for weeks now. The guy keeps insisting that the problem is our DB backups. He says that taking offline backups takes the DB offline so BOE loses connection during that time. The problem is that we don't take offline backups and I don't know anybody in the mssql world who does. That's a fairly common occurance in the Oracle world, but we just don't do it. Now you take that and combine it with the fact that the backup only lasts a few secs and is over 40mins before the problem occurs every day and you've got a nice recipe for moving on and trying something else. But does he do that? Nooooo... of course not. Instead he keeps insisting that the DB is going offline somehow.

So I wrote a quick powershell ping script that pings every 60secs and writes the result to a file. And I ran it from 2 separate servers, including the BOE server. And the DB never faltered once. And again, you've got the pefect ingredients for moving on, but is that the route we're taking? Of course not. So I told our BOE admin to have the issue escalated and to most importantly, never mention the DB as a problem with this issue again. I'm not going to say it again... it's not the DB.

Let's hope the next tech can see past the DB and actually try to solve the issue.

About Me

My Photo
Sean McCown
I am a Contributing Editor for InfoWorld Magazine, and a frequent contributor to SQLServerCentral.com as well as SSWUG.org. I live with my wife and 3 kids, and have practiced and taught Kenpo for 22yrs now.
View my complete profile

Labels

Blogumulus by Roy Tanck and Amanda Fazani

Page Views