Wednesday, November 25, 2009

The Agile missing point and the Waterfall Illusion.

What if everybody is wrong? Well, not everybody, but majority.

Reading Kelly Water’s Post about “Agile Project Management: Avoiding The Waterfall” I came to think about this. Almost nobody claims today that they are following waterfall, because of the very bad connotation that word now has, but not actually because they understand what are they doing. Same thing for Agile followers.

From the post, I rescue that any process that has phases reassembles a waterfall. From actual presentations, blogs and discussions, Agile often means just following methodologies like Scrum, Lean or XP. Now I question: are we right about this? Let me see if we are, or not.

Waterfall is a term, it seems nobody knows who coined it, related to a phased methodology. Dr. Winston W. Royce wrote about it (without given that name), describing an ideal form that was not so good . The phases were the requirements gathering, analysis on those requirements, system design, coding, testing and operation. The ideal was each step may iteratively cycle with the next down and the next up, in little steps. The flaw was it was usually the case that testing will not just go back to coding, but it may go up till design, and from there, jump back to requirements gathering again. That flaw (the large back jump) was caused not because the phases, but because the testing phase is where the actual product is finally validated against design and requirements.

The problem is, there is no solution to this. We can dilute or minimize the impact of the testing findings, using several techniques, but we cannot avoid it. At any level, team or individual, the same steps are repeated: I see what it is needed, I analyze what is possible, I think on how to do it, I do it and finally I check if it works. That can describe the process of developer creating a simple three line method! That is, phased development is there, everywhere, and that is not the problem. The problem is when that phased process is done in a way that the impact of facing reality with needs is too big that multiplies the work and the cost of the back jump.

So, the waterfall illusion is when you think that rain is not a waterfall, simply because the raindrops are so tiny you don’t see them fall individually. But they fall anyway. It means, people that think eliminating the analysis and design steps and jump into coding breaks the phased process, I’m sorry, it doesn’t, it may avoid a large body of water fall into one place, but will make millions of little bodies fall everywhere. See the point?

Ok, there is another idea into this, and that is to make the water flow up. You start coding, with no idea of what you need, and then the phases are completed backwards. (Hey! If you want to start really backwards, then start by testing! Oh, yes, we have TDD). The code will make a design “emerge”. The next logical steps if to create requirements that match the design. Well, I guess that does not work. In that case, the requirements are still presented before coding. So the new phases are: requirements, coding, design, testing. Or, requirements, testing, coding, design, testing, coding, design, etc. Anyway, we always have phases.

Now, Agile. In the Agile Manifesto the idea was to value common sense upon theory. That is why “individuals and interactions were valued over processes and tools, working software over comprehensive documentation, Customer collaboration over contract negotiation and Responding to change over following a plan.”. Nowhere on those lines there is a mention of chaos process over phased process. Where is the missing point? Well, actually in three places.

1. Comprehensive documentation was assumed to be the design’s product. So, design must go, correct? Well, that is not true. The idea behind this is to prefer creating working software rather that documenting. Actually, creating working software requires you to think first on what do I need to create software for, how can I accomplish that, think how will I do it, do it and then check if it works. You can do all that without opening a word processor, right? Working software is not any software, but the software that delivers the business value, and that is not produced just by hitting keys.

2. Customer collaboration over contract negotiation seems to define the negotiation to be a non collaborative task. Actually, any customer involvement implies a contract (or mini contracts), which is the definition of what to do, and who will do what. Collaboration does not mean the client will bring coffee, or will help writing the code. It means active communication to shape the bush with little cuts. The client says the overall figure, then supports indicating the little branches you may cut further. Contract and mini-contracts.

3. Responding to change over following a plan does not mean going into chaos. The following a plan refers to follow a line without looking at changes in time. That is flawed thinking, the plan may perfectly be to respond to change in an organized way, validating each time to improve the next one. The plan is simple: we decide as we go, following these principles.

All the above is in the principles.
“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
“Continuous attention to technical excellence and good design enhances agility.”
“Business people and developers must work together daily throughout the project”
And
“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Now, let’s talk about project managers and architects.
I understand many agilistas think they need to disappear. Yes, YAGNI. Architects are document creators and thus have no business in Agile. Project Managers are plan followers thus they are not welcomed in Agile world.

Of course, that is not correct. I mean, the stereotype many people has about architects and project managers. In a team, you need different roles. If you have only coders, and all people just code, you have an incomplete team. Roles are not fixed either, you can have four, ten, twenty different roles in a team. Each person performs a type of work, needed to complete the main goal.

So, does that mean Architect and PM are roles? No, they are professions, and each one plays a role in the team. And no, an architect is not the one to write design diagrams, and the PM is not the one with MS Project under his arm telling everyone how to type its code.

An architect is the one that owns the architecture, that is, knows what the structures and organization of the functional elements and other architecture issues, are. Not only that, he is responsible of assuring the business goal is met, either changing the code that builds the architecture or changing the architecture to match the code.

The PM is the one in charge of measuring everything, keeping an eye on evolution, times, resources needs, organization of meetings, pending issues, team problems, process glitches, and a long etc, so coders wont need to worry about those things.

Closing this long post, I can summarize:
1. No, there is not only waterfall and the good guys. Water is falling in all projects, we need focus in minimizing the bad effects.
2. No, agile does not mean following a methodology and getting rid of good guys like Architects and PMs, but understanding the common sense of the manifesto and apply it to improve development.


Cheers!

Quick Post Edit: Reading at this, you may think I believe all Agile people thinks the same. Well, no. I was practicing agile principles even before the name was coined, and find out those are misunderstood by a lot of people. Fortunately, not all of them! So, Agile is the way to go.

No comments:

Post a Comment

Post a Comment