The Getting Better Moment

Jun 15, 2015 career suggest edit

The beads of sweat gathered on my forehead were oddly juxtaposed against the cool temperature of the air conditioned room. But there they were, caused by the heat of the CTO’s anger. I made a sloppy mistake and now sat in his office wondering if I was about to lose my job. My first full-time job. I recently found some archival footage of this moment.

I wore headphones everywhere back then

So why do I write about this? Unless you’ve been passed out drunk in a gutter for the last week (which is much more believable than living under a rock), you’ve heard about this amazing opus by Paul Ford entitled “What is Code?”

If you haven’t read it yet, cancel all your appointments, grab a beer, find a nice shady spot, and soak it all in. The whole piece is great, but there was one paragraph in particular that I zeroed in on. In the intro, Paul talks about his programming start.

I began to program nearly 20 years ago, learning via oraperl, a special version of the Perl language modified to work with the Oracle database. A month into the work, I damaged the accounts of 30,000 fantasy basketball players. They sent some angry e-mails. After that, I decided to get better.

This was his “getting better moment” and like many such moments, it was the result of a coding mistake early in his career. It caused me to reminisce about the moment I decided to get better.

When I graduated from college, websites were still in black and white and connected to the net by string and cans. They pretty much worked like this.

The Internet circa 1997 - image from Wikipedia - Public Domain

As a fresh graduate, I was confident that I would go on to grad school and continue my studies in Mathematics. But deep in debt, I decided to get a job long enough to pay down this debt a bit before I returned to the warm comfort of academia. After all, companies were keen to hire people to work on this “Web” thing. It wouldn’t hurt to dabble.

Despite my lack of experience, a small custom software shop named Sequoia Softworks hired me. It was located in the quaint beach town of Seal Beach, California. You know it’s a beach town because it’s right there in the name. The company is still around under the name Solien and now is located in Santa Monica, California.

My first few weeks were a nervous affair as my degree in Math was pretty much useless for the work I was about to engage in. Sure, it prepared me to think logically, but I didn’t know a database from a VBScript, and my new job was to build database driven websites with this hot new technology called Active Server Pages (pre .NET, we’d now call this “Classic ASP” if we call it anything).

Fortunately, the president of the company assigned a nice contractor to mentor me. She taught me VBScript, ADODB, and how to access a SQL Server database. Perhaps the most valuable lesson I learned was this:

Dim conn, rs
Set conn = Server.CreateObject("ADODB.Connection")
conn.Open("Driver={SQL Server};Server=XXX;database=XXX;uid=XXX;pwd=XXX")
Set rs = conn.Execute("SELECT * FROM SomeTable")
Do Until rs.EOF
  ' ...

  rs.MoveNext ' NEVER EVER EVER EVER FORGET THIS CALL!
Loop
conn.Close

As the comment states, never ever ever ever forget to call rs.MoveNext. Ever.

A benefit of working at a tiny company, it wasn’t long before I got to work on important and interesting projects. One of these projects was a music website called myLaunch. The website was a companion to the multi-media Launch CD-ROM magazine. Cutting edge stuff, you can still find them on Amazon. I wish I had kept the Radiohead issue, it sells for $25!

Launch magazine

It wasn’t long before the CD-ROM magazine was discontinued and the website became the main product. Launch later was bought by and incorporated into Yahoo Music.

One of my tasks was to make some changes to the Forgot Password flow. I dove in and cranked out the improvements. This was before Extreme Programming popularized the idea of test driven development, so I didn’t write any automated tests. I was so green, it hadn’t even occurred to me yet that such a thing was possible.

So I manually tested my changes. At least, I’m pretty sure I did. I probably tried it a couple times, saw the record in the database, might have seen the email or not. I don’t recall. You know, rigorous testing.

And that brings me to the beginning of this post. Not long after the change was deployed the CTO (and co-founder) called me into his office. Turns out that a Vice President at our client company had a daughter who used the website to read about her favorite bands and she had forgotten her password. She went to reset her password, but never got the email with the new generated password and was completely locked out. And we had no way of knowing how many people had run into this problem and were currently locked out, never to return.

When I returned to my desk and sprinkled the code with Response.Write statements (the sophisticated debugging technique I had at my disposal), I discovered that sure enough, the code to email the new password never ran due to a logic bug.

I soon learned there’s a pecking order to finding bugs. It’s better to

… have the computer find the bug (compiler, static analysis, unit tests) than to find it at runtime.
… find a bug at runtime yourself (or have a co-worker find it) before a user runs into it.
… have a user find a bug (and report it to you) before the daughter of your client’s Vice President does.

I wasn’t fired right then, but it was made clear to me that wouldn’t hold true if I made another mistake like that. Gulp! And by “Gulp!” I don’t mean a JavaScript build system.

Inspired by a fear of losing my job, this was my Getting Better Moment. Like Paul Ford, I decided right then to get better. Problem was, I wasn’t sure exactly how to go about it. Fortunately, a co-worker had the answer. He lent me his copy of Code Complete, and my eyes were opened.

Reading this book changed the arc of my career for the better. Programming went from a dalliance to pay off some of my student loan bills, to a profession I wanted to make a career out of. I fell in love with the practice and craft of writing code. This was for me.

The good news is I never was fired from my first job. I ended up staying there seven years, grew into a lead and then a manager of all the developers, before deciding to leave when my interests lead elsewhere. During that time, I certainly deployed more bugs, but I was much more rigorous and the impact of those bugs were small.

So there you go, that’s my Getting Better Moment. What was yours like?

Found a typo or mistake in the post? suggest edit

Comments

14 responses

Matthew Jones • June 15th, 2015
My getting better moment occurred when I thought I could handle a task my boss gave to me in X time, but it ended up taking Y time (much longer) to figure out that what we wanted to do was possible but not feasible in our timeframe. My naivete caused a bunch of wasted time all around. I wrote about it here: http://www.exceptionnotfoun...

Great post!
Chris Mccown • June 15th, 2015
Mine came right after I graduated college. I had to write a tool that would email the admins of our customers - around 2500 people. So I created the mail object, email then looped through the customer emails, clearing out the To field after every send. Except what got deployed to production had the clear statement commented out. So the first loop iteration sent the email to the first customer. The second sent it to the first and second. The third to the first, second, and third-party and so on. By the time dev ops unplugged the server from the wall, it had sent out thousands of emails, with customers getting them thousands of times. I got chewed out pretty got for that, but I really started learning code design and testing principles after that.
Steve Desmond • June 15th, 2015
Joined my first real team (always solo work before that) was way out of my league, had to up my game or be left in the dust!
nmerrigan • June 16th, 2015
Mine came from a mistake where we sent out a URL with the admin session key in the URL and had a lot of admin log in at the same time. It was around 2002 and I learnt the hard way about session hijacking but it was my moment of right, I better learn how to secure these things. For some reason, we all have that oh crap, I am never making that sort of mistake again. .
Mark Hinshaw • June 16th, 2015
While not software related, it was a getting better moment. I was in the Marine Corps after high school, as a crew chief on the UH-1N. This means I was a mechanic as well as an aircrewman. As a young crew chief, I took a tool from one aircraft, that was currently being tested (lots of start up shutdown, go flying stuff), and moved to another aircraft to do some work. I didn't tell the crew chief doing the testing that I took the tool. Because of that I wasted his time, and imposed some stress on him, because he thought he had lost the tool in question. And in case you don't know, a lost tool is a BIG deal, because if it is "lost" next to a drive shaft, that helicopter may not come home. Needless to say, I was "taught" my lesson. But most importantly, I had a Staff Sergeant who told me, "You can do one of two things; you can come to work tomorrow, pouting because you made a mistake and be lesser for it, or you can learn from your mistake, and come to work tomorrow with your head held high and apply that knowledge to a new day." I resigned myself to that latter, and because of it, I became a better crew chief. Now I get to apply the same notion to the software that I write.
Christos Matskas • June 16th, 2015
Thanks for sharing Phil! Fresh out of university and straight into my first proper dev role, I was working in a small dev team littered with bad practices (not their fault). Unfortunately, since I was as green as can be, I didn't know better and my tenure there was filled with many "Oh sh&t" moments. An epic one that I'll always remember is doing development and testing against the live, yes the LIVE, database. It wasn't long before i run my first DELETE command without a WHERE clause, wiping out over 45,000 records from the DB. Needless to say that this was the moment that made me realize that a small action can have a massive impact. We were able to restore most of the data using a backup lying on another dev's machine (yes, no backup process or infrastructure) and I decided to get my act together. 11 years on, I still remember that moment that almost cost me my job and made me think hard on how to become a better developer.
tsaylor • June 16th, 2015
I'll just chime in to say I totally remember, and enjoyed, those Launch CD-ROM magazines! Rather innovative for their time.
Dan Moyer • June 16th, 2015
Late 90's before the dotcom bubble burst. Luckily happened in days before much media coverage of hacking and security infiltration. Was doing testing on a web site and using real email addresses from the database. The test should never had caused a 'send' But mistakes happen. The mistake caused a test message email send to a couple thousand users. Spent the next couple weeks replying to pissed off customers wandering what the bogus email was. Quickly learned to never, ever, use live customer email addresses from the database for testing.
Georgi Vajarov • June 17th, 2015
Phil - thank you for sharing. And, here is my Getting Better Moment.
While working at a customer's site (a bank) I had the simple task of fixing a mismatched ID so the appropriate loan can be assigned to a specific customer. Well, I wrote my fancy TSQL statement and (without wrapping it in a transaction) hit the "run" button. A small problem: I forgot to include a "where" clause. As a result, ALL loans were assigned to this one customer. Yeah, it was that bad. What made matters worse is that they did not have a backup. So I spend a good portion of the week importing data from their mainframe system.
Another way to look at it - the bank had only one very pissed customer, but many were very happy. Call me Robin Hood if you'd like :).
Manikandan Ramachandran • June 17th, 2015
Well, back in the (same old Classic ASP) days, we had a very basic set of environments.(A Dev/Test and a production) and all the code was deployed manually to the production box (by myself). :-)..

Due to my booboo in deployment, I decided to restore an older version of production database on to the SQL server during an online day. It started restoring and then the progress bar stopped moving after a while. :-).. and stood there for ever. By then users started phoning.. and you could guess what would have happened after that.

Didn't get fired that day.. but that surely was the getting better moment for me...
Alex Dresko • June 18th, 2015
Man.. all these comments like, "I screwed up really bad back when I was just a wee lad". Well, yeah, I screwed up back when I was a wee lad.. I also screwed up REALLY bad last week. And, yeah, I had the CTO in my office, like, pronto.. from three hours away. Still got my tail tucked between my legs.

Please, someone, tell me I'm not the only person who has done this nearly 20 years into their career...
haacked • June 18th, 2015
We all still do it from time to time. But experience has taught us to hide the bodies well. :P
Andrew • August 16th, 2015
Last week I was knee deep in code I had written that I knew was going bad but I was already 20 days over the deadline (meant to be a 10 day task). I've began reading The Pragmatic Programmer and consuming as much as I can to help me improve. At a wall at the minute but I will get better.
Andrew • August 16th, 2015
In my defence I have only been a developer for 8 months but I still need to get better.