The cooperation of agile software development and project management is getting more and more spotlight recently. To address the challenge, PMI announced the new PMI Agile certificate from second half of 2011. What is the link between project manager and software developer, how can they be agile using best practice project management methods?

The cooperation of agile software development and project management is getting more and more spotlight recently. To address the challenge, PMI announced the new PMI Agile certificate from second half of 2011. What is the link between project manager and software developer, how can they be agile using best practice project management methods?

Original article (Hungarian): http://www.hiradastechnika.hu/data/upload/file/2011/HT2011_4_10.pdf

Authors: Zoltán Csutorás CSM CSP, Agilis project management consultant, Adaptive Consulting Kft.,

and Árpád Kocsis, Section Manager, Nissan Europe Information Systems

1. Introduction

Both agile software development and project management is a field with lot of books, studies and articles available. Both fields have their own training structure and own certificate. We would like to take a look how these two fields will work together in real life, in practice: develop software agile way within the framework of a project.

We are looking for the answer: what is the real agile software development in business environment, i.e. in a corporation where IT’s role is to support business strategy or to be part of the support process. We look at software development from the view of the corporation.

Developing software in IT or ICT industry (e.g. Nokia, Microsoft, Apple) – whereas the software is the product itself – is different, however some statements might still be valid. Also we set aside R&D. We primarily focus on the situation which is typical for an IT supplier, for a small business: business software development in corporate environment. No difference between internal, external or outsourced development – our statements will be valid for all 3 cases.

Our goal is to show the connections, the causes and the effects, the constrains, under which agile software development and project management must go together.

2. The corporate environment

The environment can be described using the modified version of the MOST pyramid, see Drawing-1.

The corporation moves because of the interest of the Owner – the Owner wants profit. The Executives decides on objectives and strategy. To implement the strategy, there must be projects on tactical level, which are led by the Project Manager according to the project management methodology. The actual work will be carried out by engineers, they develop the software. (Of course there are projects without engineers, but those are not interesting for this article)

An example from auto industry:

The power, the interest of authority works top-down: the intention of the owner becomes task for the executives, the executives dictate objective for the project manager, and the project manager delegates tasks to engineers. The objectives and tasks of the project team is defined from the top.

The expectation of the owner, the executives and the project manager for projects is predictability, go according to plans and stay within the frames (time-budget-scope-quality).

The developer is at the bottom of the pyramid and must adapt to the plans and goals of the above. Or if he doesn’t, there will be other developer.

The IT project must exist in this medium and the IT engineer must adapt to the corporate environment. There is no other way.

3. Project Management

For corporations, project is the tool of change – for every change, for every industry. Every big development goes as project. Drawing-2 shows the project environment.

The projects are led by using methodology. Big corporations have their own methodology based on international standards, such as implementing PMI or PRINCE2.

The IT tasks are executed as part of or within the boundaries of a business project. However the project and the software development aren’t the same! The project begins way before the software developers start working, and not ending when the software is ready, see Drawing-3.

As you can see, before the software is actually ordered from the developers, the project must be structured. Or at the end, when the software is ready, that is not enough: it must be handed over to daily operations and stabilize the maintenance.

Going back to the auto industry example: the project isn’t ready when the software update is handed over. It is ready when the new car model can be sold without problems.

The scope of work is much wider than the software development itself, and its boundaries already defined when the programming starts. The traditional standards and methods of system development (e.g ISO 12207) use this wider scope too.

4. Methods and approaches

After the business environment and project manager now let’s talk about the software development. When using the term “waterfall development” or “agile development”, it is really not a specific methodology, but an approach to software development.

The waterfall or PPP approach splits the workflow into phases (PPP stands for Phased Product Planning). We use waterfall for this approach and meaning every methodology fit to this definition.

On the other way, the agile approach is based on trust in individual and the team, accepts the uncertainty of the software development and uses iterative development.

The waterfall model can be easily linked with project management methodology because of its age and its origin. However no project management study tells you that software can be developed only using waterfall.

They don’t, because project management and software development are on different levels of the project, as it can be easily spotted on Drawing-1.

5. The agile software development

The agile software development is primarily a set of values. The agile manifesto and the 12 agile principles define short and clear values. The core of the value system is the speed, reacting to change, trust in individuals and in the team, and using working software as the only measurement to evaluate success. The agile approach is nothing more than strong shift in values comparing to waterfall. Waterfall assumes if the team follows best practice, rules and organization structure, it will be successful. But the agile approach assumes if the team of right people gets a clear objective and clear boundaries, they will create efficient methods, rules and organization structure. The difference is between the order of creating culture and team, not about the need for the rules. The agile approach defines a framework to willingly and continuously review the team very own way of working, and improve their methods parallel with their product. The framework is not about developing the product, it is about to customize methods for unique problems.

There are many methods created using agile approach. Most known is the Scrum, but there are others, e.g. eXtreme Programming, DSDM, or Kanban System for Lean Software Development. They will never become rigid methodologies because it is their very essence to adapt and continuously change.

The agile approach is the best if the project’s goal is to develop a new (or not yet existing) product. In such case there nowhere to look back, no patterns; therefore no method exists to solve the new problem. Nature of these kinds of projects are the unclear requirements, not enough information to specify, no design pattern for planning and tasks cannot be identified in advance.

In the article „The New New Product Development Game” (which was one of the inspiration for Scrum) looked at specific projects and teams making outstanding performance under these circumstances. These teams solved problems like this: “Fuji-Xerox's top management asked for a radically different copier and gave the FX-3500 project team two years to come up with a machine that could be produced at half the cost of its high-end line and still perform as well.” Their so called “rugby method” provided the basis for the Scrum.

Important to highlight 2 facts, which are not mentioned enough in the agile approach. First, originally the Scrum teams had deadline, cost, quality and objective (not requirement/scope) set in stone. Their freedom was in the way of reaching the objective. Second, these teams were used in product development requiring high level of innovation. In these projects the way to achieve success was unknown and the specifics of the final product were unknown.

If the agility is interpreted as the empowerment of the team to specify product parameters and define their own work methods, than the need for agility depends on the level of innovation, namely the knowledge of the future product. The most we know what to do, the less agility needed. In this case don’t spend energy on inventing new ways of working (but still keep improving processes). On the opposite, if the project has lot of innovation, there are no patterns in the product concept and no experience in defining the result, than surely the PPP will fail – and agile approach should be used.

6. The apparent contradiction

What is going to happen when the Scrum Master (let’s assume the software development follows Scrum) meets with the project manager at the kick-off?

There will be complete confusion! The Scrum Master is prepared to effectively lead an agile software development, but not to work within a traditional project. The Scrum Master training does not say anything about the project environment.

On the other hand, the project manager is not told that agile approach exists, what do they mean and what is his role in Scrum. The agile approach will be very strange.

The confusion is even more because of the different terminology. For example the word “planning” used by both of them with different meaning. In software development, planning means the technical planning – architecting the software. But this is part of the “execution” phase according the project managers, and not part of the “planning” phase.

The reason of the misunderstanding is that the Scrum Master and the Project Manager has different viewpoints, with different terminology, different tools and different processes.

Drawing-4 shows the exact place and role of methodologies. Why is that? Because on the level of developer and the chief developer they have their own methodology – software development methodology -, and in the same time the project manager has project management methodology. All of that works under the umbrella of Project Portfolio Management – with its own methodology.

Furthermore: The usual understanding of the agile development is in contradiction with the project management methodologies.

One of the examples is the importance of planning: for QA reasons the existence of a project plan is essential, while according to the Agile Manifesto it is not. Other great examples are the need for written contract, the need for detailed documentation or knowing estimations in advance.

The differences grown so large that they become blockade between engineers and managers – both sides try to convince the other on their own truth.

The fight is pointless, because the nature of the project is a given fact, should be accepted. Business objectives must be reached – and the engineer can be replaced.

7. The waterfall approach

If agile approach makes such a conflict, shouldn’t be easier just to use waterfall approach? This is the approach familiar to managers and executives, provides high degree of predictability and goes according to plans.

Looking at traditional “waterfall” software developers today, in the 21^st century, turns out that they don’t work according to the “waterfall” definition of the 20^th century.

The biggest reason: there is no project without change, there is no software without change request. The business life became fast, IT must follow. Every project management methodology includes change management. Therefore it is going to be exactly the Project Manager who forces waterfall to be flexible. The result: no specification written in stone.

The second biggest reason is efficiency: the large corporations have realized that small teams are more flexible and more efficient than large teams. Nowadays you cannot find 100 developers sitting in the same room developing based on a 1000 page specification. That world is over. The outsourcing has changed everything. This waterfall is not that waterfall.

As a conclusion, the rigid, process-based software development methods are getting more and more flexibly, more and more agile. The need for being agile comes from top, from the direction of executives, who want a software working according to the quickly changing business needs, a fully functioning good software – and everything for tomorrow.

8. The meeting of the two approach

The software development models and approaches are just tools of the good project manager. They aren’t opposite, they are just good for different reasons.

Doesn’t matter which approach to use, at the end in the hand of a successful manager the practice will be the same. In waterfall change is possible. If change is frequent, if the planning is based on experience and not wrong assumptions, it is going to work and surprisingly many “agile” practice will be recognizable.

Teams starting with agile approach will find their own rules, getting more organized and more standardized. After a while the truly good agile team will have so much rules that a strict waterfall team would envy.

Drawing-5 shows the meeting of the two models. The waterfall models start with high bureaucracy, but by time they are adapting to business needs they will be flexible.

Starting from the other side, the agile approach may seem chaotic in its original form, but as the team defines more and more rules, they will be more bureaucratic.

The two approach will approximate!

What a surprise! Well, thinking over again, it makes sense:

- The goal is the same everywhere: working software, satisfied customer.

- The corporate environment described in section-2 is the same, meaning that the practice must be similar, independently if it is agile or not.

- The project management environment described in section-3 is a given fact. It is the same independently from the software development (agile or not).

- Section-4 describes that all software development methods created with the same purpose, the difference is in the approach and the tools.

- Turned out that the contradiction described in section-6 appears only below the level of project management. None of the approaches are actually opposition with the project management.

As a result, the application of methodologies in practice cannot be too different.

Because the approaches are actually going towards each other, there is an optimum point between: the “ideal” software development model. The “working software” is the area where the project is going confidently, according to the plan, driven by managers, and the software development will succeed. Because this point can be reached using process-optimization, any similarity with CMMI Level-5 is not a coincidence.

The role of the project manager is to find this point using the tools and elements of the methodology. Nothing is new here, since PMI itself is not more than offering of set of practices – provides building blocks (processes) to build up the project.

9. Summary

We have started at the corporate environment, checked the software development and the project management, taken a look at software development approaches, especially the agile approach.

Turned out that agile model is differ from traditions, but this is not a problem since none of the models supposed to be followed blindly, and the objective is the same everywhere. Starting from any approach, the project management and the practice will be the same.

For every approach you need a project manager who is clear on expectations and able to build up the project using the elements (practices and processes) of the appropriate methodology.