Monday, June 18, 2007
The Eye of the Beholder
No - I'm not talking about the Dungeons and Dragons game...
This article felt like a challenge to me. It would seem that Agile Methods are a little slow making inroads to embedded development (here's a short article). Ganssle's article was a throwback to my early Agile days, and it brought into stark clarity some of argument in favour of proper unit testing. I posted the following at the bottom of the New Code article:
Beauty is in the eye of the beholder. Every individual and team has its own unique style of coding. Anyone introduced to that code will find it ugly in some fashion or another - unless they can take on the culture and context of the original authors.
In other words, I agree that we'll spend most of our time working on legacy code, and that we'll invariably find it ugly.
What we really need is to capture the intent of the code - the reasoning behind it - in the unit tests. Subsequent generations of designers will have a better appreciation of what was done. They'll have the courage to refactor whenever necessary because they've got the tests to prove that their new code behaves exactly like the old code.
Does this apply only as we write new code? Absolutely not! Any adjustments to legacy code should be preceeded by the creation of a firewall of unit tests around the effected code, articulating and asserting its behaviour. The subsequent refactoring is then less prone to error, and ends with a fully tested section of code.
This article felt like a challenge to me. It would seem that Agile Methods are a little slow making inroads to embedded development (here's a short article). Ganssle's article was a throwback to my early Agile days, and it brought into stark clarity some of argument in favour of proper unit testing. I posted the following at the bottom of the New Code article:
Beauty is in the eye of the beholder. Every individual and team has its own unique style of coding. Anyone introduced to that code will find it ugly in some fashion or another - unless they can take on the culture and context of the original authors.
In other words, I agree that we'll spend most of our time working on legacy code, and that we'll invariably find it ugly.
What we really need is to capture the intent of the code - the reasoning behind it - in the unit tests. Subsequent generations of designers will have a better appreciation of what was done. They'll have the courage to refactor whenever necessary because they've got the tests to prove that their new code behaves exactly like the old code.
Does this apply only as we write new code? Absolutely not! Any adjustments to legacy code should be preceeded by the creation of a firewall of unit tests around the effected code, articulating and asserting its behaviour. The subsequent refactoring is then less prone to error, and ends with a fully tested section of code.
Monday, February 05, 2007
Software Development and the Worship of False Gods
Making a perfect plan is actually quite easy: you wait until the project is done, and then you write down what happened.
Sadly, that defeats the purpose of the plan. Every software project has a budget. Sometimes the budget is expressed in months, sometimes it’s in dollars, but it’s always there, and it represents the company’s Investment in the project. At the outset of a project, the plan gives some confidence that a reasonable Return on Investment (ROI) can be expected.
During execution, the plan is used to track progress against the budget. This includes the need to coordinate all the different teams that contribute to the project, since a lack of coordination wastes budgeted resources and involves increased Investment and lower ROI.
So, how do we compromise on the ideal of writing a perfect plan at the end of the project? The typical answer is: with predictability; set the plan up front, and then stick to it. Deviations from the plan represent increased Investment, and lower ROI and so should be avoided.
Predictability has become the primary goal of most software organizations. In the name of predictability, we have the SDLC . Given that change is to be avoided, we invest time and effort at the beginning of the project to make the best guess possible. Numerous mathematical models and techniques have been devised to improve the estimation of software engineering costs (COCOMO, Function Points, etc.). This is vital since, once the estimates have been given, any change to the time or resources required represents higher Investment and lower ROI.
Now here’s something interesting: the above logic assumes the connection between scope, time and resources – that a change in scope must be associated with a change in time or resources. For the estimates to hold and the predictions true, neither scope, nor time, nor resources can vary. Software Developers are notoriously hard to come by, so fixing resources is reasonable. Customers (including internal marketing teams) make commitments based on the expected delivery date, so time is definitely fixed – missing a delivery date results in costs across departments, business units or even companies!
But who said the scope is fixed? So, rather than saying “we have to deliver this functionality by that date,” perhaps we should say “when that date comes, we’ll give the customer something that they are happy to accept.”
That would radically change the focus of software development. We would be forced into constant dialog with customers, to discover what they want and to validate that we’ll be able to give it to them. We would be forced into transparent operation, where our customers would be engaged in a partnership with our design teams; they would have a direct say in what got built, so when the date arrived, they’d know we’d made the best possible use of our limited time. We would be forced to consider new information as it came to light, and to work together with our customer to determine an appropriate response.
We’d be forced to focus on delivering software, instead of desperately trying to make the emerging reality fit the plans we made at the beginning of the project.
By enslaving ourselves to Predictability, we have made the success of our projects contingent on the guesses that we make in the planning stage. Furthermore, we have become averse to changing our plans, even though changes are inevitably required to improve on our initial vision of the product.
What would this approach do for ROI? By fixing the time and resources for the project, we’ve fixed the Investment. By attending to the customer and delivering to them the product that best meets their needs in the time provided, it would seem reasonable that we have increased our Return. Return on Investment goes up.
All that said, you still need to have a plan. Given that time is fixed, it’s absolutely vital to know if your project is on time. The instant there’s a deviation from the plan of record, you need to be on the phone with the customer, letting them know what’s happening and getting their input on what reaction would suit them best.
"In preparing for battle I have always found that plans are useless, but planning is indispensable." - Dwight D. Eisenhower
February 13 2007 addendum: this same argument is made more eloquently by David Anderson on pgs 168 -169 of "Agile Management for Software Engineering"
Sadly, that defeats the purpose of the plan. Every software project has a budget. Sometimes the budget is expressed in months, sometimes it’s in dollars, but it’s always there, and it represents the company’s Investment in the project. At the outset of a project, the plan gives some confidence that a reasonable Return on Investment (ROI) can be expected.
During execution, the plan is used to track progress against the budget. This includes the need to coordinate all the different teams that contribute to the project, since a lack of coordination wastes budgeted resources and involves increased Investment and lower ROI.
So, how do we compromise on the ideal of writing a perfect plan at the end of the project? The typical answer is: with predictability; set the plan up front, and then stick to it. Deviations from the plan represent increased Investment, and lower ROI and so should be avoided.
Predictability has become the primary goal of most software organizations. In the name of predictability, we have the SDLC . Given that change is to be avoided, we invest time and effort at the beginning of the project to make the best guess possible. Numerous mathematical models and techniques have been devised to improve the estimation of software engineering costs (COCOMO, Function Points, etc.). This is vital since, once the estimates have been given, any change to the time or resources required represents higher Investment and lower ROI.
Now here’s something interesting: the above logic assumes the connection between scope, time and resources – that a change in scope must be associated with a change in time or resources. For the estimates to hold and the predictions true, neither scope, nor time, nor resources can vary. Software Developers are notoriously hard to come by, so fixing resources is reasonable. Customers (including internal marketing teams) make commitments based on the expected delivery date, so time is definitely fixed – missing a delivery date results in costs across departments, business units or even companies!
But who said the scope is fixed? So, rather than saying “we have to deliver this functionality by that date,” perhaps we should say “when that date comes, we’ll give the customer something that they are happy to accept.”
That would radically change the focus of software development. We would be forced into constant dialog with customers, to discover what they want and to validate that we’ll be able to give it to them. We would be forced into transparent operation, where our customers would be engaged in a partnership with our design teams; they would have a direct say in what got built, so when the date arrived, they’d know we’d made the best possible use of our limited time. We would be forced to consider new information as it came to light, and to work together with our customer to determine an appropriate response.
We’d be forced to focus on delivering software, instead of desperately trying to make the emerging reality fit the plans we made at the beginning of the project.
By enslaving ourselves to Predictability, we have made the success of our projects contingent on the guesses that we make in the planning stage. Furthermore, we have become averse to changing our plans, even though changes are inevitably required to improve on our initial vision of the product.
What would this approach do for ROI? By fixing the time and resources for the project, we’ve fixed the Investment. By attending to the customer and delivering to them the product that best meets their needs in the time provided, it would seem reasonable that we have increased our Return. Return on Investment goes up.
All that said, you still need to have a plan. Given that time is fixed, it’s absolutely vital to know if your project is on time. The instant there’s a deviation from the plan of record, you need to be on the phone with the customer, letting them know what’s happening and getting their input on what reaction would suit them best.
"In preparing for battle I have always found that plans are useless, but planning is indispensable." - Dwight D. Eisenhower
February 13 2007 addendum: this same argument is made more eloquently by David Anderson on pgs 168 -169 of "Agile Management for Software Engineering"
Sunday, December 10, 2006
What is Software Design?
Conversations at work led me to a question: what does it mean to design software? The software world attaches significant importance design documents. The following Wikipedia quote from their Software design section sums it up:
I went back to my University texts for a definition of design. I read through some of Norman's "The Design of Everyday Things" and scanned a few chapters in Papanek's "Design for the Real World." These got me thinking about the meaning of design outside the software world. I started thinking about a lamp.
What does it mean to design a lamp? You start by understanding the intended use of the new product. You make a few prototypes and test them with typical users to validate that your design will meet their needs. You work within the constraints of the materials and manufacturing processes you have, and you make a plan for how the lamp can be mass produced.
The design process ends with instructions for how an object can be made - that's the design.
What's the equivalent in software development? After a few conversations with my peers, we happened on something interesting: we started with the end and worked backward.
The end of lamp production has a customer pulling a lamp out of a box plugging it in and turning it on. The end of software production then, has a customer installing the software on their computer and using it. That seems reasonable.
Going backward one step, the lamp was built in a factory. The software passed through a factory too, where it was burned onto discs. Again, this lines up well.
Another step back: the factory got the blueprints and manufacturing instructions from the designer and set up their machines accordingly. The software...well, the software arrived at the factory as compiled binaries on a disc.
"Hold the phone," we said.
These binaries are just a direct translation of the code: essentially, we could have handed the code over to the factory. That means the code is the design.
Woah.
The more we talked about this, the more sense it made to me and the more excited I got. In my experience, we learn most about what we're developing while we're developing it. There is clearly a need to have a reasonable starting point, but it's not possible to get it right on paper. That's like thinking that you can fly a 747 because you read the manual.
Saying the code is the design makes sense out of this. From this way of thinking, the 'on paper' design that you start with is the initial design. As the code is written, the design evolves along the path of least resistance, until the requirements are met.
Of course, a quick web search showed this concept isn't new. At all. Here's a seminal 1992 article by Jack Reeves. Read this! It does complete justice to the ideas expressed above.
It is possible to design software in the process of programming, without a planMy organization is fairly typical, and design documents are required. It's important to me, professionally, that the design documents on my projects are as good as they can be. Once I started questioning the nature of design, I discovered that I really didn't have any idea what 'good' means in this context.
or requirement analysis, but [that] would not be considered a professional approach. A separate design prior to programming allows for multidisciplinary
designers and subject matter experts to collaborate with highly-skilled programmers for software that is both useful and technically sound.
I went back to my University texts for a definition of design. I read through some of Norman's "The Design of Everyday Things" and scanned a few chapters in Papanek's "Design for the Real World." These got me thinking about the meaning of design outside the software world. I started thinking about a lamp.
What does it mean to design a lamp? You start by understanding the intended use of the new product. You make a few prototypes and test them with typical users to validate that your design will meet their needs. You work within the constraints of the materials and manufacturing processes you have, and you make a plan for how the lamp can be mass produced.
The design process ends with instructions for how an object can be made - that's the design.
What's the equivalent in software development? After a few conversations with my peers, we happened on something interesting: we started with the end and worked backward.
The end of lamp production has a customer pulling a lamp out of a box plugging it in and turning it on. The end of software production then, has a customer installing the software on their computer and using it. That seems reasonable.
Going backward one step, the lamp was built in a factory. The software passed through a factory too, where it was burned onto discs. Again, this lines up well.
Another step back: the factory got the blueprints and manufacturing instructions from the designer and set up their machines accordingly. The software...well, the software arrived at the factory as compiled binaries on a disc.
"Hold the phone," we said.
These binaries are just a direct translation of the code: essentially, we could have handed the code over to the factory. That means the code is the design.
Woah.
The more we talked about this, the more sense it made to me and the more excited I got. In my experience, we learn most about what we're developing while we're developing it. There is clearly a need to have a reasonable starting point, but it's not possible to get it right on paper. That's like thinking that you can fly a 747 because you read the manual.
Saying the code is the design makes sense out of this. From this way of thinking, the 'on paper' design that you start with is the initial design. As the code is written, the design evolves along the path of least resistance, until the requirements are met.
Of course, a quick web search showed this concept isn't new. At all. Here's a seminal 1992 article by Jack Reeves. Read this! It does complete justice to the ideas expressed above.
Thursday, September 28, 2006
Confusion over "Agile"
I read something recently that finally crystalized something that's been pissing me off for a long time. It seemed like a good place to start this blog.
While reading Steve Yegge's post "Good Agile, Bad Agile" a useful distinction occurred to me: there's a philosophy, as put forward in the Agile Manifesto, and there are executions of that philosophy: the various Agile Methods that people are so quick to rail against are examples.
I read the vitriolic derision aimed at the agile community. Steve's post goes on at length about 'Bad Agile.' Articles at Hacknot are harsh. A quick web search uncovered various other instances of developers bashing Agile. These pieces are backed with statements about how Extreme Programming is broken, or why Scrum can't work. That's fine. Everyone's entitled to their opinion.
The authors of these articles often go on in self aggrandizing fashion, to describe how they develop software, and how that's sooo much better than 'Agile' methods. Their contempt is palpable; I wonder if they run off to wash their hands after typing that word, 'Agile,' the way Sicilians expectorate after speaking the name of familia disgraziata.
The comments typically boil down to: Agile is for unthinking fanatics.
And this is where I find myself frustrated. They've tossed the entire Agile movement on the trash heap, but their own software modi operandi? The ones they're so proud of? Almost invariably, they are executions of the Agile philosophy!
While reading Steve Yegge's post "Good Agile, Bad Agile" a useful distinction occurred to me: there's a philosophy, as put forward in the Agile Manifesto, and there are executions of that philosophy: the various Agile Methods that people are so quick to rail against are examples.
I read the vitriolic derision aimed at the agile community. Steve's post goes on at length about 'Bad Agile.' Articles at Hacknot are harsh. A quick web search uncovered various other instances of developers bashing Agile. These pieces are backed with statements about how Extreme Programming is broken, or why Scrum can't work. That's fine. Everyone's entitled to their opinion.
The authors of these articles often go on in self aggrandizing fashion, to describe how they develop software, and how that's sooo much better than 'Agile' methods. Their contempt is palpable; I wonder if they run off to wash their hands after typing that word, 'Agile,' the way Sicilians expectorate after speaking the name of familia disgraziata.
The comments typically boil down to: Agile is for unthinking fanatics.
And this is where I find myself frustrated. They've tossed the entire Agile movement on the trash heap, but their own software modi operandi? The ones they're so proud of? Almost invariably, they are executions of the Agile philosophy!
I'm not the first to have noticed this. One of the earliest comments on this post gives a decent comparison between Scrum and Google's development practices, so I won't add any detail here.
Steve talks about a lot of other things too - things that don't fall into the category of Agile development, good or bad. He talks about managers who motivate with incentive rather than fear. At Google, they limit meetings to core hours, they hire good, disciplined developers, some of whom work on tools to help the others. Everyone gets time to work on side projects, and they're strongly urged to take advantage of that.
Guess what folks - that's just good management. These things lead to a productive environment, but they don't make it any easier to change the direction of your product while you're writing the code. And that's what Agile is all about.
What I'm saying is this: as an Agilist, I agree with everything Steve calls 'Good Agile.' Some of it is simply an interpretation of the Agile Manifesto; it's not Scrum, or XP, or anything - it's a version of Agile that exists within Google. The rest just describes a productive work environment.
This leads me to a few interesting conclusions. First, most people know Agile through the practices and procedures that follow from the Agile philosophy; few are aware of the philosophy itself. Further, the mapping between the philosophy and the practices is not straightforward. Without awareness and understanding of the philosophy, the practices are easily misinterpreted. The unfortunate consequence is developers who judge Agile on its face and reject it, even though they may actually be Agile at heart.
My second conclusion is that having a group of Agile developers isn't enough; for a project to be successful, you also need the right environment. If the culture surrounding the development team is toxic to Agile precepts, it will be difficult if not impossible to mount a successful Agile project.
