Monday, February 05, 2007
Software Development and the Worship of False Gods
Making a perfect plan is actually quite easy: you wait until the project is done, and then you write down what happened.
Sadly, that defeats the purpose of the plan. Every software project has a budget. Sometimes the budget is expressed in months, sometimes it’s in dollars, but it’s always there, and it represents the company’s Investment in the project. At the outset of a project, the plan gives some confidence that a reasonable Return on Investment (ROI) can be expected.
During execution, the plan is used to track progress against the budget. This includes the need to coordinate all the different teams that contribute to the project, since a lack of coordination wastes budgeted resources and involves increased Investment and lower ROI.
So, how do we compromise on the ideal of writing a perfect plan at the end of the project? The typical answer is: with predictability; set the plan up front, and then stick to it. Deviations from the plan represent increased Investment, and lower ROI and so should be avoided.
Predictability has become the primary goal of most software organizations. In the name of predictability, we have the SDLC . Given that change is to be avoided, we invest time and effort at the beginning of the project to make the best guess possible. Numerous mathematical models and techniques have been devised to improve the estimation of software engineering costs (COCOMO, Function Points, etc.). This is vital since, once the estimates have been given, any change to the time or resources required represents higher Investment and lower ROI.
Now here’s something interesting: the above logic assumes the connection between scope, time and resources – that a change in scope must be associated with a change in time or resources. For the estimates to hold and the predictions true, neither scope, nor time, nor resources can vary. Software Developers are notoriously hard to come by, so fixing resources is reasonable. Customers (including internal marketing teams) make commitments based on the expected delivery date, so time is definitely fixed – missing a delivery date results in costs across departments, business units or even companies!
But who said the scope is fixed? So, rather than saying “we have to deliver this functionality by that date,” perhaps we should say “when that date comes, we’ll give the customer something that they are happy to accept.”
That would radically change the focus of software development. We would be forced into constant dialog with customers, to discover what they want and to validate that we’ll be able to give it to them. We would be forced into transparent operation, where our customers would be engaged in a partnership with our design teams; they would have a direct say in what got built, so when the date arrived, they’d know we’d made the best possible use of our limited time. We would be forced to consider new information as it came to light, and to work together with our customer to determine an appropriate response.
We’d be forced to focus on delivering software, instead of desperately trying to make the emerging reality fit the plans we made at the beginning of the project.
By enslaving ourselves to Predictability, we have made the success of our projects contingent on the guesses that we make in the planning stage. Furthermore, we have become averse to changing our plans, even though changes are inevitably required to improve on our initial vision of the product.
What would this approach do for ROI? By fixing the time and resources for the project, we’ve fixed the Investment. By attending to the customer and delivering to them the product that best meets their needs in the time provided, it would seem reasonable that we have increased our Return. Return on Investment goes up.
All that said, you still need to have a plan. Given that time is fixed, it’s absolutely vital to know if your project is on time. The instant there’s a deviation from the plan of record, you need to be on the phone with the customer, letting them know what’s happening and getting their input on what reaction would suit them best.
"In preparing for battle I have always found that plans are useless, but planning is indispensable." - Dwight D. Eisenhower
February 13 2007 addendum: this same argument is made more eloquently by David Anderson on pgs 168 -169 of "Agile Management for Software Engineering"
Sadly, that defeats the purpose of the plan. Every software project has a budget. Sometimes the budget is expressed in months, sometimes it’s in dollars, but it’s always there, and it represents the company’s Investment in the project. At the outset of a project, the plan gives some confidence that a reasonable Return on Investment (ROI) can be expected.
During execution, the plan is used to track progress against the budget. This includes the need to coordinate all the different teams that contribute to the project, since a lack of coordination wastes budgeted resources and involves increased Investment and lower ROI.
So, how do we compromise on the ideal of writing a perfect plan at the end of the project? The typical answer is: with predictability; set the plan up front, and then stick to it. Deviations from the plan represent increased Investment, and lower ROI and so should be avoided.
Predictability has become the primary goal of most software organizations. In the name of predictability, we have the SDLC . Given that change is to be avoided, we invest time and effort at the beginning of the project to make the best guess possible. Numerous mathematical models and techniques have been devised to improve the estimation of software engineering costs (COCOMO, Function Points, etc.). This is vital since, once the estimates have been given, any change to the time or resources required represents higher Investment and lower ROI.
Now here’s something interesting: the above logic assumes the connection between scope, time and resources – that a change in scope must be associated with a change in time or resources. For the estimates to hold and the predictions true, neither scope, nor time, nor resources can vary. Software Developers are notoriously hard to come by, so fixing resources is reasonable. Customers (including internal marketing teams) make commitments based on the expected delivery date, so time is definitely fixed – missing a delivery date results in costs across departments, business units or even companies!
But who said the scope is fixed? So, rather than saying “we have to deliver this functionality by that date,” perhaps we should say “when that date comes, we’ll give the customer something that they are happy to accept.”
That would radically change the focus of software development. We would be forced into constant dialog with customers, to discover what they want and to validate that we’ll be able to give it to them. We would be forced into transparent operation, where our customers would be engaged in a partnership with our design teams; they would have a direct say in what got built, so when the date arrived, they’d know we’d made the best possible use of our limited time. We would be forced to consider new information as it came to light, and to work together with our customer to determine an appropriate response.
We’d be forced to focus on delivering software, instead of desperately trying to make the emerging reality fit the plans we made at the beginning of the project.
By enslaving ourselves to Predictability, we have made the success of our projects contingent on the guesses that we make in the planning stage. Furthermore, we have become averse to changing our plans, even though changes are inevitably required to improve on our initial vision of the product.
What would this approach do for ROI? By fixing the time and resources for the project, we’ve fixed the Investment. By attending to the customer and delivering to them the product that best meets their needs in the time provided, it would seem reasonable that we have increased our Return. Return on Investment goes up.
All that said, you still need to have a plan. Given that time is fixed, it’s absolutely vital to know if your project is on time. The instant there’s a deviation from the plan of record, you need to be on the phone with the customer, letting them know what’s happening and getting their input on what reaction would suit them best.
"In preparing for battle I have always found that plans are useless, but planning is indispensable." - Dwight D. Eisenhower
February 13 2007 addendum: this same argument is made more eloquently by David Anderson on pgs 168 -169 of "Agile Management for Software Engineering"
# posted by Dana @ 8:25 AM 
