Tape Measure

Tape_Measure by pat00139 used under Creative Commons

I went to the Milwaukee ALT.NET (@AltDotNetMke) group’s Holiday hangout a couple weeks ago. It was a fun night of drinking a couple beers, talking shop and some non-shop items. One of the things that we talked about was something that is very near and dear to my heart: how fast can you get a new member of your team up to speed and productive and what the answer says about how mature your development processes are. I heard a pretty surprising response from a couple of the folks at the Holiday Hang out when I asked the question.

You may have heard of the Software Engineering Institute at Carnegie Mellon University and their tools for measuring Information Technology / Software Development teams. You almost certainly have heard of the Joel Test that pokes a little fun at the SEI and offers a simple 12 question “test” to rate the quality of your software development team. I have quoted the Joel test to numerous developers and architects over the years myself. I have a bit of a twist on both of these approaches it is a relatively simple test:

  1. Bring a brand new developer onto your team. Needs to be a developer with an average level of experience, but a “rock star” developer or a novice will not skew the results too far either way.
  2. Give him/her a laptop that has access to the domain and has the basic developer tools installed (Visual Studio as an example). But the laptop cannot have been setup by anyone on your team.
  3. Give him/her basic instructions for accessing your environment (source control repository, tools you use, development servers and databases).
  4. Start timing to see how long it takes for the new developer to be able to get the latest version of your source code and successfully have it running on their machine.

The new developer is allowed to ask follow-up questions (and you should take notes on every question that they ask), but you are not allowed to actually do anything for them. Each time that you have to look at their machine in order to answer a question, you should add an hour to the clock time. Once the developer has the code running, take note of that time. I like to call this the “zero to configured” metric. If you are like most development teams, this number probably makes you want to weep uncontrollably. Most development teams probably think this should take them a “few minutes”, but the actual number is probably located between several hours and never with most answers hovering in the neighborhood of a couple of days. Seriously – try this test and see for yourself.

I wish I had actual data on this metric to share, but I only have anecdotal information. But if you work in software, you probably have a lot of anecdotal stories of your own. Here are a couple tidbits:

  • The project that could not build or debug unless the files were located in C:\projects\erin
  • The search for the .h file that you only have the .obj for
  • The class library that only worked from C:\Program Files (did you know that is not a universal in Windows, it can be installed on other drives and I know that older versions of Italian installed used C:\Programi)
  • The application whose production build only worked on one developer’s laptop (and the 4 hours of terror as chkdsk and several other tools were run trying to get it passed a boot error)
  • The application that required you to be running as a local administrator (*)
  • Brining on an extra developer to help finish a release on time only to have him leave 2 weeks later never having written a line of code (a shining example of the Mythical Man Month)
  • The unit tests that work perfectly until you deployed them on the server and then could only run if you spent an hour in regedit32

I was not personally involved in every one of these and I have tried to keep the details vague to protect the guilty (unless you name is Erin – sorry dude).

Does a high “zero to configured” metric mean you are a bad group of developers? No. It could be that you are a good group of developers and you don’t have a lot of new people on boarding. We tend to work on problems that we encounter more frequently and if you don’t bring on new developer, you probably won’t address it. It does mean that you will probably have a painful time when you get a new machine.

Does a low “zero to configured” metric mean you are an awesome group of developers? No. It could be you have a very simple application or spend too much time worrying about on-boarding new developers. I will say that there is a high correlation between a good development team and a low “zero to configured” metric.

That was why I was so impressed by the answer that one of the guys gave when I asked the question at the alt.Net hangout. He said that he rolled onto a project a few weeks before and was up and running in about 2 hours. The impressive part was that the 2 hours included installing the local version of SQL Server. Kudos to that team!

** Seriously – quit running as an administrator. You will write better code, spend less time debugging issues and be much more secure as a result.