Software without Borders: Measure the Success of Your Outsourcing with Metrics

“If you can’t measure it, you can’t manage it.” – Peter Drucker

Are you afraid that outsourced software development means having little or no control over the development process? After all, if you work together with the programmers in the same room, then there is no need to closely measure their progress.

Or do you?

When I worked as a programmer in the 1980s, my boss used to joke that he was going to hire a guy with a kettle drum and put him in the corner of the room. Every time the kettle drum was hit, we had to have written a line of code!

Today, outsourcing promises huge cost savings and executives are less concerned with lines per minute than with dollars per hour. But in the end, it’s important to know the money you spend is fueling real progress in the development of your software.

How can this be done? It’s called metrics.

Business process outsourcing (BPO) provides an example of how outsourcing can be successfully measured. Business processes such as accounts receivable and outbound sales calls can be so well defined that you can accurately measure how efficiently and effectively they’re implemented. New software tools not only help you detect problems and inefficiencies, but can predict and fix the problems before they even arise.

To measure new software development you track how many new features are added over time. Some metrics split the programming required into work units and then track how many units are completed over time. It’s best, in my experience, to measure these results daily and at least weekly.

Engineers are notoriously optimistic about their ability to create working software. So another metric measures how accurate their estimates are for the time required to finish the software development. Initially, their ability to estimate will likely be poor. You can set a goal for the engineers to improve this skill as your development continues so you improve the predictability of your process.

For maintenance programming you need to track the work units or bugs fixed over time. In addition, you should measure the amount of rework required for bugs that fail the QA step after bug fix attempts.

Your service provider team should commit to a schedule for completing the programming work. As part of this their commitment, they must also agree to the definition of work units and the productivity level they believe they can achieve. Their commitment requires them to work independently and without specific instructions for daily work activities. This is much more effective and frees up your time too.

You typically measure the throughput of your outsourced team as a whole. A team is typically a combination of junior and senior members. Junior engineers will need guidance and mentoring from the senior engineers. This is normal and should be expected and encouraged. But it should also be measured over time. A senior engineer can be expected to spend from 5% to 25% of his or her time with junior engineers, depending on the complexity of the project and prior experience of the junior engineer.

Today most people use simple software tools like spreadsheets and Microsoft Project to track the metrics of their outsourcing. More sophisticated tools are also available but are expensive and best applied when you have a large portfolio of software development projects. New tools are being developed to automatically compute your metrics as your software is developed. For example the amount of time source files are checked out of your source code control system can be used to help measure the productivity of your engineers.

You can use metrics as the basis for a service level agreement (SLA) struck with your service provider. But remember: the purpose of an SLA is to help guide your software development to success and to detect and correct problems as they arise. It’s not to support micro management, set up a blame game or create an adversarial relationship with the vendor’s team.

Will software development become as predictable as BPO and enable you to fix problems before they occur? I doubt we’ll ever have this much control over the creative software development process. But who knows? That guy with the kettle drum may not be far off!