Agile Outsourcing: Software Design and Agile Methodologies

In an ideal world, software project management is a well-oiled machine where a set of business analysts write excellent requirements documents. Next, a group of super programmers takes these and converts them into an outstanding software design — including user interfaces. They pass them on to the programmers, who perform an outstanding job on turning this into highly tuned code that follows all coding guidelines. Then, the testing and quality assurance group does its part, the bugs get fixed, the software gets retested and the final delivery of the software is made to the sheer delight of the end users.

This is one end of the idealized vision of how software development works. And, of course, this is a rare case with any in-house software development project — and much rarer in outsourced software development.

The other end of the spectrum is where the programmers themselves are the software designers also, and they make up the software design as they go along. There are no software design documents around to speak of when it is handed over to another group of people for on-going maintenance and support.

The most successful projects are the ones that keep their eyes on the prize, a working software solution that solves the business problem of the ultimate customer and also comes with short, reasonably current requirements documents or design documents.

In this month’s column I explore some of the typical problems and observations about software design that happen in real-life and how agile methodologies address them very effectively.

Software Design Documents Are a Necessary Evil

Reasonably updated software design documents are a necessary part and even a requirement for agile outsourcing. Why? If you want changes done quickly by different people in the same group or geographically dispersed groups of people working on the same software development project, it’s always helpful to have reasonably accurate design documents available. If you don’t have them, your software development agility suffers because you’ll be wasting time trying to figure out what someone else did.

Software Design Groups Do Only the Initial Design

Ask any group of programmers that has been successful in delivering software that effectively addressed business problems, “How much of the original design is still left in the solution?” They’ll typically tell you that it’s only a fraction. This is actually good, not bad — in most cases. It means that the solution has adapted to changes needed as the project progressed.

Unless your software development project is a fat, multi-billion-dollar contract that covers all contingencies or a highly sensitive military application that requires all software design to be fully up to date at any time, the final software design that emerges is different from the initial design. This is just reality.

So how do you balance this reality with having just enough documentation that enables on-going maintenance and support easy? No more, no less.

Changes in Requirements Necessitate Changes in Design

Design documents become obsolete not because of bad application of software design techniques, but changes in requirements brought about after the requirements phase is well over. Design documents may become dated because of changes upstream from the design process. Programmers may respond to these changes in requirements well after the initial design is done. This is where it becomes difficult to keep design documents updated.

Continuous Improvement in Design Is Always a Necessity

Changes in requirements may bring about changes in software design, and this in turn, may make programmers realize that if they change certain aspects of the initial design, they may end up with some re-usable components that make further enhancements easy. This kind of flexibility in the software design may not mean much immediately, but may save person-months or person-years of effort down the road. This is especially true of software product development. You always want to design a generalized feature that is customizable rather than a specific feature that one of your customers has asked for. In real life, a second customer always needs a feature that is only slightly different from the first, and customizable aspects help achieve this quickly and efficiently.

Formalized Software Design Is Usually Not All Good or All Bad

Writing detailed software design documents for fairly commonplace applications may be a waste of time. However, there are always tricky parts of the application like complex workflows or detailed state transitions that need to be documented fully for programmers or future maintainers of the application to understand what should be going on in a software module. Comments and in-line documentation in programs left by programmers are often inadequate for these purposes.

How do Agile Methodologies Address These Problems?

Agile methodologies address many of the software design issues that come up in the design phase in an effective way. Since agile methodologies respond to changes in requirements faster, the design that follows also adapts and becomes the right design quickly. However, it needs to be noted that periodic, radical, total review and redesign may be necessary so that the design is internally consistent, efficient and effective.

Let me provide an example. In responding to new features in the last few agile releases, programmers have added a whole new set of tables to the database. Periodic and thorough review of the design may be necessary so that at different stages you rationalize the database design itself, as you go along.

Agile methodologies, with their frequent releases and interactions with end users, allow the software developers to match the software design, particularly the user interface design to the precise needs of the end users. In many cases, these kinds of design changes are very hard to foresee at the initial design stage. They may have to be unearthed only in an iterative way. A good example may be some of the changes online companies like eBay or Amazon may have had to make to their Web sites. They have unearthed totally unexpected and new sets of uses for their software, such as third-party stores. Their initial design plans may not have foreseen these kinds of changes.

Short design notes or mini-design documents are a very effective way in agile methods to document those parts of the software design that are undergoing changes or represent complex pieces of logic or code. These documents — no more than a few pages — are easier to create and keep updated than volumes of detailed design documents. They are much more realistic and practical. (A sample of one from my own company is available for review in a link under “Useful Links” below.)

Even in non-agile methods, in practice, a lot of the redesign of the software is done on the fly by the programmers themselves. Rare is the case where the original design is adhered to, precisely. There are always exceptions, such as military applications or applications that involve complex rules or logic that need to be documented properly.

Making the programmers themselves responsible for documenting parts of the software design that are important, can make sure that future developers or maintainers of the application are not left high and dry.

Implications for Outsourcing Clients and Service Providers

Outsourcing clients often insist on following a software development methodology strictly and a strict fixed price contract. The service provider writes out a detailed fixed price contract and allows exceptions or deviations from the requirements and design only with additional change requests and only with additional money, outside the fixed price contract.

When the software is delivered, they complain about the service provider being inflexible about changes in requirements. Of course, a one-line change in a requirement may mean months of backend effort, or a few hours of work, depending upon the complexity of the backend design. From their point of view, the scope of work may be small or large — and, understandably, they want to protect their revenues and margins.

Use of agile methodologies may dictate that the pricing model be tailored appropriately. A time and materials model with a cap on the total price may be a viable alternative, preserving the interests of both the client and the service provider. This is especially beneficial when it comes to changes in requirements and resulting changes in the design.

If all parties agree that the goal of the software solution is to solve business problems efficiently and effectively, agile methodologies provide a good way to address even the problems that typically arise in the design stage of the software development process. By keeping only essential design tasks and performing them well, they ensure that future development teams and maintainers of the application have enough documentation around so that they can do their own agile development in turn, without wasting time figuring out how a piece of software worked.

Useful Links


Mini-design document sample