The Code Dojo

This morning Mark and I attended a great talk by Eric Smith, CTO and Co-founder of Control4, at the UTC CTO P2P Forum, titled “Outsourcing: What not to do.”  The short version of the story is that a few years ago Control4 decided they should try outsourcing. They spent about $700,000 setting up a nice 50-man shop in Bangalore, with a good Indian HR manager and a guy from here that moved over there as a technical manager for the shop. In about a year and a half, they spent about $3,000,000 on the facility, and ran into a bunch of problems. They couldn’t hire and keep the top notch developers because they weren’t a big name company, and they had a lot of churn and turnover due to the 20-25% annual growth in average salaries. They had a hard time being clear enough and specific enough in their specs and task lists to get it built right the first time. The 12.5 hour time difference made things very hard for communication. On some projects, they had to go back and forth 17 times with changes, bugs, clarifications, etc. before it was done right.

When he talked about the results they got from this $3M, 1.5 year investment, he said that only about 30% of what the team produced was able to be salvaged and used. The other 70% had to be rebuilt from scratch. One of the most telling things he said was that by the time they got all the kinks worked out and the team there was working at full speed, the production they saw from the Bangalore office was about what they would have had if they had kept their technical manager here in the States and hired one more guy like him. Two top notch guys (for argument sake, say you’re paying them each $150K per year, a total of $300K/year plus their office space and any other administrative overhead) would have yielded the same output as a 50 person team costing $2M per year. He didn’t say, but if that doesn’t already factor in the “30% usable output” then the difference is even more drastic. Don’t forget that another cost besides the money is the time it takes to get to the right solution. If 1.5 years is what it would take to do it right, then 1.5 years to get it 30% right means you still have a lot of work to do and a lot more time (1-5 years) to really get it done, which in a competitive market can leave you in the dust.

One really sad thing is that this story is hardly unique. Many people we’ve talked to who have tried outsourcing projects (to India, Russia, or anywhere) have run into similar problems and worse. Most of the outsourced web development projects we hear about didn’t get in that deep of course, but the principle is the same.

So what is the moral of the outsourcing story? It really is the same lesson many others have learned: Cost, or hourly rate, isn’t the only thing that matters. Sure your outsourced developers are cheap, but what do you get for it? What’s the quality and quantity of their output?

Mark tells a great story about a guy who has a brain tumor. He needs brain surgery. For the same price, he can hire a team of 10 nurses, or one brain surgeon. In this case it is obvious who he should hire – the quality of the brain surgery done by the team of nurses wouldn’t be the same. Now, say you fell of your bike and skinned your knees and elbows. You probably don’t even need a nurse to put on a bandage for you, but if you hired one, he or she would do a great job I’m sure. Say you need stitches though… you’d probably want at least a nurse for that. Most doctors or paramedics could probably handle broken bones and could stop some pretty serious bleeding. It’s all about finding the right skill level for the job. There’s usually a minimum skill level to get something done right. Then there’s a range of skill levels where the solutions range from “right” to “better” to “awesome”, and a point where higher skill doesn’t give you higher quality. This spectrum is different in almost every project, and for every client. What quality is acceptible to you may depend on a lot of things: your budget, how concerned you are with the perceptions it gives your audience, how much time you have, how concerned you are with short-term needs and costs compared to long-term benefits and savings.

That brings me to Mark’s next metaphor: finding the right balance on the spectrum of quality and cost is all about “the 9′s”. A solution that is 90% of the way to perfect, that’s one 9. If you get it to 99%, that’s two 9′s. Some applications need 99.999% of perfect, or the well-known five 9′s. The problem is that not all 9′s are created equal. The first 9 is the least expensive, fastest, and easiest. Almost any web developer can get you to one 9 in most projects they start. At that point, some developers will hit a brick wall, and can’t really get you past that point, no matter how much time and money you throw at it. They simply aren’t equipped for the task. A good example of one 9 is the really cheap website, like the $499 web site special you can find sometimes, for a template-based “custom” design and a static web site. Even though it feels like 90% of perfect is really good, it doesn’t mean that you’re better than 90% of the other websites out there. You’re probably in the bottom 5-10% with a site like this, maybe lower.

The second 9 may increase your costs by 20%, or it may double the cost, but it is usually still in a reasonable price range. This might be the $750-1000 web site, with an inexpensive web designer who can design and build a static site without starting from a template. This developer has a higher skill level and can build something that has much higher quality than the cookie cutter template site. The 99% site is probably getting a lot closer to average, but I’d say it is probably still in the bottom half of web sites.

The third 9 takes another step up in both cost and quality, and for a lot of people, it is well worth it. 99.9% is a lot better than 90%, and here you might be an average or above average website for a small to medium company that wants to look respectable. Continuing with the website analogy, this might be your $1500-2000 web site that has a pretty nice custom design, well built HTML/CSS code, and might have some content management functionality. For example, this could be a custom design implemented into a good WordPress theme, so you can manage the pages in your site, and maybe do some blogging or enable some cool plugins. It might not be a lot of custom stuff, but it will get you something reasonable.

The fourth 9 might be where you run into the knee of the curve, so to speak. Here you might see that it multiplies your cost by more than double, but the quality of the result also goes up a lot. Sometimes 99.99% is overkill for what you need, so the cost is a lot higher without having a much higher value to you. But when that extra quality has extra value for you, it can be a much better decision than taking the less expensive route.  This might upgrade your site to a really sweet custom design from a good marketing firm or graphic design shop, and could include a shopping cart for example, so you can sell products online. This could be anything from a custom theme for something like OSCommerce or ZenCart or some other out-of-the-box ready-made shopping cart, to a pretty nice custom shopping cart solution that works just the way you want it. Price ranges for something like this might be anywhere from $2000-3000 to $6000-10,000 depending on exactly what you get and from whom.

As you probably guessed, the fifth 9 is where you get a really awesome solution. Most of the time in my experience, this is overkill for 97% of businesses, and the extra investment doesn’t give them enough extra value to be worth it. But if you’re one of that 3%, this can be a total dream come true. The five 9′s solution for you might be an awesome custom web site designed by a great advertising agency with a highly-customized cart and fulfillment system underneath. This site could include content management of everything from FAQs, employee directories, and photo galleries to custom planning and quoting tools, integrated CRM functions, and integration with your warehouse or manufacturing line. You may start out with a $20,000=50,000  jump start on the project, then over a few years spend $100,000 to $300,000 on enhancements and upgrades. This is the kind of site that becomes the core of your business operations, tying together your accounting, employee management, inventory, ordering, and customer service in ways that save you a lot of man-hours and a lot of money. For the right company, this is a solution that is worth its weight in gold, but for the wrong company, it can be the millstone around its neck, dragging it down because the costs weren’t really justified and didn’t provide the return.

So how do you know which site to choose? How many nines are right for you? Often the best way to evaluate it is as a simple business decision. Try to assess the value of a feature, and weigh that against the cost of that feature. If it makes sense, do it, and if not, don’t, or do it later when the scale tips in favor of that feature. Everyone wants an awesome web site, but not everyone can afford a really awesome web site, and not everyone will get a lot of return on investment (ROI) for a five nines website. Try your best not to get your heart set on features that just don’t make any business sense, unless you’re willing to pay for them and not see any benefit from them other than that you got what you wanted. Just because you think a particular feature is awesome doesn’t necessarily mean your audience will.

I guess the bottom line to all of this is that the cheapest option, like outsourcing, isn’t always the best option, and that quality really matters. Buying cheap can easily give you quality that is too low, causing expensive repair or replacement of the faulty parts, and potentially costing you a lot of headaches and lost time that you’ll never be able to get back again. On the other hand, don’t jump to the most expensive solution if the ROI isn’t going to justify the costs. Find the right place on the spectrum for what you need, and stick to it. Have a plan for growth and the future, and be patient enough to stick to it when it means you have to wait a while longer to get your dream system. There are plenty of people who can share their stories with you. Learn from their mistakes so you don’t have to make as many mistakes of your own.

Tags: brain surgery, outsourcing, ROI, Web Development