This paper describes the practices of agile methods from the viewpoint of project management.
The project management techniques are complex processes that require the understanding and coordination of several domains of knowledge.
As more and more software projects engage Agile Methods, there are emerging patterns of success and failure. With growing adoption of Agile Methods, project managers increasingly need to understand the applicability to their projects and factors that drive key project performance characteristics.
Agile Methods have advantages, especially in accommodating change due to volatile requirements. However, they also present concomitant risks with managing the many dependent pieces of work distributed across a large project.
The paper is divided into four parts. In the first part an overview of the project management and its processes and knowledge areas discussed. after that the agile methods discussed following with a short history of RAD(We should mention that just three most used and famous methodologies are discussed).
In the second part the project management approaches and a brief definition of each approach are given.
In the third part we looked at the agile methodologies from project management areas view such as cost, time, quality and risk management and we compared agile methodologies and we explained their advantages and disadvantages.
In the fourth part we discussed about combination of agile methodologies and their utilization in large and complex projects.
And finally we propose our own idea about the future of project management in agile methods.
Keywords Project Management, Rapid Development Methodologies, Agile Project Management, History of RAD, Project management approaches, Agile Performance Measurement, Investment and Risk, Agile Enterprise Framework, Agile Methodology Fit
What is Project?
- A human activity that achieves a clear objective against a time scale
- "A project is a one-shot, time-limited, goal-directed, major undertaking, requiring the commitment of varied skills and resources".
A project is a temporary endeavor undertaken to create a unique product or service. A project is temporary in that there is a defined start (the decision to proceed) and a defined end (the achievement of the goals and objectives). Ongoing business or maintenance operations are not projects. Energy conservation projects and process improvement efforts that result in better business processes or more efficient operations can be defined as projects. Projects usually include constraints and risks regarding cost, schedule or performance outcome.
What is Project Management?
Many have attempted to define project management. One example, Oisen,3 referencing views from the 1950's, may have been one of the early attempts. Project Management is the application of a collection of tools and techniques (such as the CPM and matrix organization) to direct the use of diverse resources toward the accomplishment of a unique, complex, one-time task within time, cost and quality constraints. Each task requires a particular mix of theses tools and techniques structured to fit the task environment and life cycle (from conception to completion) of the task.
Notice in the definition are included some the success criteria, The Iron triangle. Those criteria for measuring success included in the description used by Oisen3 continue to be used to describe project management today. The British Standard for project management BS60794 1996 defined project management as:
The planning, monitoring and control of all aspects of a project and the motivation of all those involved in it to achieve the project objectives on time and to the specified cost, quality and performance. The UK Association of project Management (APM) have produced a UK Body of Knowledge UK (BoK)5
which also provides a definition for project management as:
The planning, organization, monitoring and control of all aspects of a project and the motivation of all involved to achieve the project objectives safely and within agreed time, cost and performance criteria. The project manager is the single point of responsibility for achieving this. Other definitions have been offered, Reiss6 suggests a project is a human activity that achieves a clear objective against a time scale, and to achieve this while pointing out that a simple description is not possible, suggests project management is a combination of management and planning and the management of change.
Lock's7 view was that project management had evolved in order to plan, co-ordinate and control the complex and diverse activities of modern industrial and commercial projects, while Burke8 considers project management to be a specialized management technique, to plan and control projects under a strong single point of responsibility.
While some different suggestions about what is project management have been made, the criteria for success, namely cost, time and quality remain and are included in the actual description. Could this mean that the example given to define project management Oisen3 was either correct, or as a discipline, project management has not really changed or developed the success measurement criteria in almost 50 years.
Project management is a learning profession. Based upon past mistakes and believed best practice, standards such as BS 60794 and the UK Body of Knowledge5 continue to be developed. But defining project management is difficult, Wirth,9 indicated the differences in content between six countries own versions of BoK's. Turner10 provided a consolidated matrix to help understand and moderate different attempts to describe project management, including the assessment. Turner10 further suggested that project management could be described as: the art and science of converting vision into reality. Note the criteria against which project management is measured is not included in that description. Is there a paradox however in even attempting to define project management? Can a subject which deals with a unique, one-off complex task as suggested as early as Oisen3 be defined? Perhaps project management is simply an evolving phenomena, which will remain vague enough to be non-definable, a flexible attribute which could be a strength. The significant point is that while the factors have developed and been adopted, changes to the success criteria have been suggested but remain unchanged. Could the link be, that project management continues to fail because, included in the definition are a limited set of criteria for measuring success, cost, time and quality, which even if these criteria are achieved simply demonstrate the chance of matching two best guesses and a phenomena correctly. Prior to some undergraduate lectures and workshops about project management, the students were asked to locate some secondary literature describing project management and produce their own definition. While there were some innovative ideas, the overriding responses included the success criteria of cost, time and quality within the definition. If this is the perception about project management we wish those about to work in the profession to have, the rhetoric over the years has worked. Has this however been the problem to realizing more successful projects? To date, project management has had the success criteria focused upon the delivery stage, up to implementation. Reinforced by the very description we have continued to use to define the profession. The focus has been to judge whether the project was done right. Doing something right may result in a project which was implemented on time, within cost and to some quality parameters requested, but which is not used by the customers, not liked by the sponsors and does not seem to provide either improved effectiveness or efficiency for the organization, is this successful project management?
Project Management Life Cycle
The process flow of Project management processes is shown below. The various elements of project management life cycle are
- Need identification
- Closing out
a) Need Identification
The first step in the project development cycle is to identify components of the project. Projects may be identified both internally and externally:
- Internal identification takes place when the energy manager identifies a package of energy saving opportunities during the day-to-day energy management activities, or from facility audits.
- External identification of energy savings can occur through systematic energy audits undertaken by a reputable energy auditor or energy service company.
- In screening projects, the following criteria should be used to rank-order project opportunities.
- Cost-effectiveness of energy savings of complete package of measures (Internal rate of return, net present value, cash flow, average payback)
- Sustainability of the savings over the life of the equipment.
- Ease of quantifying, monitoring, and verifying electricity and fuel savings.
- Availability of technology, and ease of adaptability of the technology to Indian conditions.
- Other environmental and social cost benefits (such as reduction in local pollutants, e.g. SOx)
Initiating is the basic processes that should be performed to get the project started. This starting point is critical because those who will deliver the project, those who will use the Bureau of Energy Efficiency project, and those who will have a stake in the project need to reach an agreement on its initiation. Involving all stakeholders in the project phases generally improves the probability of satisfying customer requirements by shared ownership of the project by the stakeholders. The success of the project team depends upon starting with complete and accurate information, management support, and the authorization necessary to manage the project.
The initiation stage should include a plan that encompasses the following areas:
- Analyzing the business needs/requirements in measurable goals
- Reviewing of the current operations
- Financial analysis of the costs and benefits including a budget
- Stakeholder analysis, including users, and support personnel for the project
- Project charter including costs, tasks, deliverables, and schedule
The planning phase is considered the most important phase in project management. Project planning defines project activities that will be performed; the products that will be produced, and describes how these activities will be accomplished and managed. Project planning defines each major task, estimates the time, resources and cost required, and provides a framework for management review and control. Planning involves identifying and documenting scope, tasks, schedules, cost, risk, quality, and staffing needs.
The result of the project planning, the project plan, will be an approved, comprehensive document that allows a project team to begin and complete the work necessary to achieve the project goals and objectives. The project plan will address how the project team will manage the project elements. It will provide a high level of confidence in the organization's ability to meet the scope, timing, cost, and quality requirements by addressing all aspects of the project.
Project planning generally consists of
- determining how to plan (e.g. by level of detail or rolling wave);
- developing the scope statement;
- selecting the planning team;
- identifying deliverables and creating the work breakdown structure;
- identifying the activities needed to complete those deliverables and networking the activities in their logical sequence;
- estimating the resource requirements for the activities;
- estimating time and cost for activities;
- developing the schedule;
- developing the budget;
- risk planning;
- gaining formal approval to begin work.
Additional processes, such as planning for communications and for scope management, identifying roles and responsibilities, determining what to purchase for the project and holding a kick-off meeting are also generally advisable.
For new product development projects, conceptual design of the operation of the final product may be performed concurrent with the project planning activities, and may help to inform the planning team when identifying deliverables and planning activities
Once a project moves into the execution phase, the project team and all necessary resources to carry out the project should be in place and ready to perform project activities. The project plan is completed and base lined by this time as well. The project team and the project manager's focus now shifts from planning the project efforts to participating, observing, and analyzing the work being done.
The execution phase is when the work activities of the project plan are executed, resulting in the completion of the project deliverables and achievement of the project objective(s). This phase brings together all of the project management disciplines, resulting in a product or service that will meet the project deliverable requirements and the customers need. During this phase, elements completed in the planning phase are implemented, time is expended, and money is spent.
In short, it means coordinating and managing the project resources while executing the project plan, performing the planned project activities, and ensuring they are completed efficiently.
e) Monitoring and Controlling
Project Control function that involves comparing actual performance with planned performance and taking corrective action to get the desired outcome when there are significant differences. By monitoring and measuring progress regularly, identifying Bureau of Energy Efficiency variances from plan, and taking corrective action if required, project control ensures that project objectives are met.
Monitoring and Controlling includes:
- Measuring the ongoing project activities (where we are);
- Monitoring the project variables (cost, effort, scope, etc.) against the project management plan and the project performance baseline (where we should be);
- Identify corrective actions to address issues and risks properly (How can we get on track again);
- Influencing the factors that could circumvent integrated change control so only approved changes are implemented
- In multi-phase projects,process also provides feedback between project phases, in order to implement corrective or preventive actions to bring the project into compliance with the project management plan.
- Continuing support of end users
- Correction of errors
- Updates of the software over time
Project Maintenance is an ongoing process, and it includes:
Monitoring and Controlling cycle
In this stage, auditors should pay attention to how effectively and quickly user problems are resolved.
Over the course of any construction project, the work scope may change. Change is a normal and expected part of the construction process. Changes can be the result of necessary design modifications, differing site conditions, material availability, contractor-requested changes, value engineering and impacts from third parties, to name a few. Beyond executing the change in the field, the change normally needs to be documented to show what was actually constructed. This is referred to as Change Management. Hence, the owner usually requires a final record to show all changes or, more specifically, any change that modifies the tangible portions of the finished work. The record is made on the contract documents - usually, but not necessarily limited to, the design drawings. The end product of this effort is what the industry terms as-built drawings, or more simply, "as built." The requirement for providing them is a norm in construction contracts.
When changes are introduced to the project, the viability of the project has to be re-assessed. It is important not to lose sight of the initial goals and targets of the projects. When the changes accumulate, the forecasted result may not justify the original proposed investment in the project.
f) Closing out
Project closeout is performed after all defined project objectives have been met and the customer has formally accepted the project's deliverables and end product or, in some instances, when a project has been cancelled or terminated early. Although, project closeout is a routine process, it is an important one. By properly completing the project closeout, organizations can benefit from lessons learned and information compiled. The project closeout phase is comprised of contract closeout and administrative closure.
This phase consists of:
- Project close: Finalize all activities across all of the process groups to formally close the project or a project phase
- Contract closure: Complete and settle each contract (including the resolution of any open items) and close each contract applicable to the project or project phase
Project Management Knowledge Areas with the Related Processes
Each of the nine knowledge areas contains the processes that need to be accomplished within its discipline in order to achieve an effective project management program. Each of these processes also falls into one of the five basic process groups, creating a matrix structure such that every process can be related to one knowledge area and one process group.
Software development projects represent an investment of resources by the project's sponsor, an investment that often yields little or no return. The Standish Group's Chaos Report 1994 states that fewer than 10% of software projects in large companies were successful. Medium sized companies do better with 16% of their software projects being successful, and small companies succeed on 28% of their software projects (Standish 1994). Given these statistics it is worthwhile to invest significant effort in Risk Management for software projects. "Research at The Standish Group also indicates that smaller time frames, with delivery of software components early and often, will increase the success rate." (Standish 1994).
Extreme Programming offers nothing to help integrate the efforts of non-software developers. Unfortunately, some advocates of Extreme Programming suggest that the efforts of technical writers, database managers, and quality assurance specialist are not required. In reality, while Extreme Programming does not explicitly describe how to integrate the work of others, the practices do not preclude the ability to integrate with other efforts. Small Releases make Integration Management a more continuous process in contrast to processes that place deployment, documentation, and testing at the end of the schedule.
At a more tactical level, the Extreme Programming practice of Continuous Integration requires that the work of software developers be integrated on a daily basis. While this practice can cause additional overhead for individual developers, it allows the team to identify problems daily that would otherwise become undiscovered rework accumulating until all developers integrate their individual work products.
Scope Management & Time Management
Ask most software development teams for a copy of their project plan and you will receive an activity list formatted as a Gantt chart. Many times these activity lists will describe several phases of activities such as Analysis, Design, Construction, and Testing. Areas of functionality will be broken out under these headings in order to assign them to specific programmers, but seldom are the assignments identified in the Gantt chart clearly traceable back to a Requirement or other specification documents. All too often, the missing item that would help a team improve their planning practices is a well-constructed Work Breakdown Structure. Extreme Programming focuses almost all of its planning efforts on building a thoughtful Work Breakdown Structure and its constituent Work Packages.
Extreme Programming does not teach Work Breakdown Structures and Work Packages explicitly, however, careful study of the Story Cards used in Extreme Programming reveals that they are almost identical to Work Packages in their key attributes.
Human Resources Management
Often one of the most challenging aspects of project management is managing human resources. For software development projects in particular this includes the complex juggling of technical tasks between individual software developers who have different individual skills, effectively treating each developer's assigned tasks as an independent subproject. This type of project plan often suffers from key resource bottlenecks and status meetings reduced to determining which individuals are falling furthest behind. Extreme Programming addresses this head-on by eliminating the dependency on individual developers. Work Packages are scheduled and authorized based on the needs of the business and the users not the needs of the software developers. All developers are cross-trained to work in all areas of the code base. Developers broaden their skills, and project managers stop worrying about keeping individual software developers for the entire duration of the project. The process maintains knowledge of the full code base in the team, not in individuals.
As programmers move from work authorization to work authorization, and often from one area of the code to another, it is easy to see that maintaining quality in the work product could be challenging. Extreme Programming requires a very disciplined design approach to allow freedom in assigning resources while maintaining high quality.
When a project manager mentions the need for improved communications on a project, software developers often immediately envision an increased number of meetings and documents. While formal meetings and written documents have their place in a communication plan there are many other tools for facilitation of communication between project participants. The Extreme Programming practices include several simple practices intended to enhance communications.
Often a Project Manager is evaluated on his or her ability to complete a project within budget. The costs include estimated cost, actual cost and variability. Contingency cost takes into account influence of weather, suppliers and design allowances.
How the 80/20 Rule can help a project manager?
The 80/20 Rule means that in anything a few (20 percent) are vital and many (80 percent) are trivial. Successful Project Managers know that 20 percent of the work (the first 10 percent and the last 10 percent) consumes 80 percent of your time and resources.
The History of RAD
Traditional lifecycles devised in the 1970s, and still widely used today, are based upon a structured step-by-step approach to developing systems. This rigid sequence of steps forces a user to "sign-off" after the completion of each specification before development can proceed to the next step. The requirements and design are then frozen and the system is coded, tested, and implemented. With such conventional methods, there is a long delay before the customer gets to see any results and the development process can take so long that the customer's business could fundamentally change before the system is even ready for use. In response to these rigid, cascading, one-way steps of Stagewise or Waterfall Models of development, Barry Boehm, Chief SW Engineer at TRW, introduced his Spiral
Model. The Spiral Model is a risk-driven, as opposed to code-driven, approach that uses process modeling rather than methodology phases. Through his model, Boehm first implemented software prototyping as a way of reducing risk. The development process of the Spiral Model separates the product into critical parts or levels while performing risk analyses, prototyping, and the same steps at each of these levels. Similarly, Tom Gilb's Evolutionary Life Cycle is based on an evolutionary prototyping rationale where the prototype is grown and refined into the final product.
The work of Boehm and Gilb paved the way for the formulation of the methodology called Rapid Iterative Production Prototyping (RIPP) at DuPont in the mid-to-late 1980s. James Martin then extended the work done at DuPont and elsewhere into a larger, more formalized process, which has become known as Rapid Application Development (RAD). RAD compresses the step-by-step development of conventional methods into an iterative process. The RAD approach thus includes developing and refining the data models, process models, and prototype in parallel using an iterative process. User requirements are refined, a solution is designed, the solution is prototyped, the prototype is reviewed, user input is provided, and the process begins again.
What is Agility?
There is no Agility for Dummies. Agility isn't a silver bullet. You don't achieve it in five easy steps. So what is it? From one view agility characterized in two statements:
"Agility is the ability to both create and respond to change in order to profit in a turbulent business environment.
Agility is the ability to balance flexibility and stability" (Highsmith 2002).
In an uncertain and turbulent world, success belongs to companies that have the capacity to create change, and maybe even chaos, for their competitors. Creating change disrupts competitors (and the entire market ecosystem); responding to change guards against competitive thrusts. Creating change requires innovation: developing new products, creating new sales channels, reducing product development time, customizing products for increasingly smaller market segments. In addition, your company must be able to respond quickly to both anticipated and unanticipated changes created by your competitors and customers.
An example of a product development effort in which all the aspects of agility come into play is that of small, portable DNA analyzers. These instruments can be used for analyzing suspected bio-terror agents (e.g., anthrax), performing quick medical diagnoses, or undertaking environmental bacterial analysis. These instruments must be accurate, easy to use, and reliable under wide-ranging conditions, and their development depends on breakthroughs in nanotechnology, genome research, and micro-fluidics. Developing these leading-edge products requires blending flexibility and structure, exploring various new technologies, and creating change for competitors by reducing delivery time. These are not projects that can be managed by traditional, prescriptive project management methodologies.
Some people mistakenly assume that agility connotes a lack of structure, but the absence of structure, or stability, generates chaos. Conversely, too much structure generates rigidity. Complexity theory tells us that innovation—creating something new in ways that we can't fully anticipate (an emergent result) occurs most readily at the balance point between chaos and order, between flexibility and stability. Scientists believe that emergence, the creation of novelty from agent interaction, happens most readily at this "edge of chaos." The idea of enough structure, but not too much, drives agile managers to continually ask the question, "How little structure can I get away with?" Too much structure stifles creativity. Too little structure breeds inefficiency.
This need to balance at the edge of chaos to foster innovation is one reason process-centric methodologies often fail. They push organizations into over-optimization at the expense of innovation. Agile organizations don't get lost in some gray middle ground; they understand which factors require stabilization and which ones encourage exploration. For example, in a high-change product development environment, rigorous configuration management stabilizes and facilitates flexibility just as a focus on technical excellence stabilizes the development effort.
Overview and definitions
The "Agile Movement" in software industry saw the light of day with the Agile
Software Development Manifesto4 published by a group of software practitioners and consultants in 2001 (Beck et al. 2001; Cockburn 2002a). The focal values honored by the agilists are presented in the following:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
These central values that the agile community adheres to are:
First, the agile movement emphasizes the relationship and communality of software developers and the human role reflected in the contracts, as opposed to institutionalized processes and development tools. In the existing agile practices, this manifests itself in close team relationships, close working environment arrangements, and other procedures boosting team spirit.
Second, the vital objective of the software team is to continuously turn out tested working software. New releases are produced at frequent intervals, in some approaches even hourly or daily, but more usually bi-monthly or monthly. The developers are urged to keep the code simple, straightforward, and technically as advanced as possible, thus lessening the documentation burden to an appropriate level.
Third, the relationship and cooperation between the developers and the clients is given the preference over strict contracts, although the importance of well drafted contracts does grow at the same pace as the size of the software project. The negotiation process itself should be seen as a means of achieving and maintaining a viable relationship. From a business point of view, agile development is focused on delivering business value immediately as the project starts, thus reducing the risks of non-fulfillment regarding the contract.
Fourth, the development group, comprising both software developers and customer representatives, should be well-informed, competent and authorized to consider possible adjustment needs emerging during the development process life-cycle. This means that the participants are prepared to make changes and that also the existing contracts are formed with tools that support and allow these enhancements to be made.
According to Highsmith and Cockburn (2001, p. 122), "what is new about agile methods is not the practices they use, but their recognition of people as the primary drivers of project success, coupled with an intense focus on effectiveness and maneuverability. This yields a new combination of values and principles that define an agile world view." Boehm (2002) illustrates the spectrum of different planning methods with Figure 1, in which hackers are placed at one end and the so called inch-pebble ironbound contractual approach at the opposite end:
Hawrysh and Ruprecht (2000) state that a single methodology can not work for the whole spectrum of different projects, but instead the project management should identify the specific nature of the project at hand and then select the best applicable development methodology. To stress the point further, according to McCauley (2001) there is a need for both agile and process-oriented methods, as there is no one-size-fits-all software development model that suits all imaginable purposes. This opinion is shared by several experts in the field (Glass 2001).
Cockburn (2002a, p. xxii) defines the core of agile software development methods as the use of light-but-sufficient rules of project behavior and the use of human- and communication-oriented rules. The agile process is both light and sufficient. Lightness is a means of remaining maneuverable. Sufficiency is a matter of staying in the game (Cockburn 2002a). He proposes the following "sweet spots" the presence of which in software development work enhances the prospects for a successful project outcome:
- Two to eight people in one room
- Communication and community
- Onsite usage experts
- Short and continuous feedback cycles
- Short increments
- One to three months, allows quick testing and repairing
- Fully automated regression tests
- Unit and functional tests stabilize code and allow continuous improvement
- Experienced developers
- Experience speeds up the development time from 2 to 10 times compared to slower team members.
In this section, the current state of agile software development methods is reviewed. The selection of methods is based on the definition proposed in "overview and definition section". As a result the following methods are included in this analysis: Extreme Programming (Beck 1999b), Scrum (Schwaber 1995; Schwaber and Beedle 2002), Crystal family of methodologies (Cockburn 2002a), Feature Driven Development (Palmer and Felsing 2002), the Rational Unified Process (Kruchten 1996; Kruchten 2000), Dynamic Systems Development Method (Stapleton 1997), Adaptive Software Development (Highsmith 2000) and Open Source Software development (e.g., O'Reilly 1999). Methods will be introduced and reviewed systematically by using a defined structure where process, roles and responsibilities, practices, adoption and experiences, scope of use and current research regarding each method are identified. Process refers to the description of phases in the product life-cycle through which the software is being produced. Roles and responsibilities refer to the allocation of specific roles through which the software production in a development team is carried out. Practices are concrete activities and work products that a method defines to be used in the process. Adoption and experiences refer primarily to existing experience reports regarding the use of method in practice and method developers' considerations how the method should be introduced in an organization. Scope of use identifies the limitations regarding each method, i.e. if such have been documented. Finally, the current research and publications are surveyed in order to get an overview of the scientific and practical status of the method.
Extreme Programming (XP) has evolved from the problems caused by the long development cycles of traditional development models (Beck 1999a). It first started as 'simply an opportunity to get the job done' (Haungs 2001) with practices that had been found effective in software development processes during the preceding decades (Beck 1999b). After a number of successful trials in practice (Anderson et al. 1998), the XP methodology was "theorized" on the key principles and practices used (Beck 1999b). Even though the individual practices of XP are not new as such, in XP they have been collected and lined up to function with each other in a novel way thus forming a new methodology for software development. The term 'extreme' comes from taking these commonsense principles and practices to extreme levels (Beck 1999b).
In the following, these phases are introduced according to Beck's (1999b) description:
In the Exploration phase, the customers write out the story cards that they wish to be included in the first release. Each story card describes a feature to be added into the program. At the same time the project team familiarize themselves with the tools, technology and practices they will be using in the project. The technology to be used will be tested and the architecture possibilities for the system are explored by building a prototype of the system. The exploration phase takes between a few weeks to a few months, depending largely on how familiar the technology is to the programmers.
The Planning phase sets the priority order for the stories and an agreement of the contents of the first small release is made. The programmers first estimate how much effort each story requires and the schedule is then agreed upon. The time span of the schedule of the first release does not normally exceed two months. The planning phase itself takes a couple of days.
The Iterations to release phase includes several iterations of the systems before the first release. The schedule set in the planning stage is broken down to a number of iterations that will each take one to four weeks to implement. The first iteration creates a system with the architecture of the whole system. This is achieved by selecting the stories that will enforce building the structure for the whole system. The customer decides the stories to be selected for each iteration. The functional tests created by the customer are run at the end of every iteration. At the end of the last iteration the system is ready for production.
The Productionizing phase requires extra testing and checking of the performance of the system before the system can be released to the customer. At this phase, new changes may still be found and the decision has to be made if they are included in the current release. During this phase, the iterations may need to be quickened from three weeks to one week. The postponed ideas and suggestions are documented for later implementation during, e.g., the maintenance phase.
After the first release is productionized for customer use, the XP project must both keep the system in the production running while also producing new iterations.
In order to do this, the Maintenance phase requires an effort also for customer support tasks. Thus, the development velocity may decelerate after the system is in production. The maintenance phase may require incorporating new people into the team and changing the team structure.
The Death phase is near when the customer does no longer have any stories to be implemented. This requires that the system satisfies customer needs also in other respects (e.g., concerning performance and reliability). This is the time in the XP process when the necessary documentation of the system is finally written as no more changes to the architecture, design or code are made. Death may also occur if the system is not delivering the desired outcomes, or if it becomes too expensive for further development.
The first references in the literature to the term 'Scrum' point to the article of Takeuchi and Nonaka (1986) in which an adaptive, quick, self-organizing product development process originating from Japan is presented (Schwaber and Beedle 2002). The term 'scrum' originally derives from a strategy in the game of rugby where it denotes "getting an out-of play ball back into the game" with teamwork (Schwaber and Beedle 2002). The Scrum approach has been developed for managing the systems development process. It is an empirical approach applying the ideas of industrial process control theory to systems development resulting in an approach that reintroduces the ideas of flexibility, adaptability and productivity (Schwaber and Beedle 2002). It does not define any specific software development techniques for the implementation phase. Scrum concentrates on how the team members should function in order to produce the system flexibly in a constantly changing environment.
The main idea of Scrum is that systems development involves several environmental and technical variables (e.g. requirements, time frame, resources, and technology) that are likely to change during the process. This makes the development process unpredictable and complex, requiring flexibility of the systems development process for it to be able to respond to the changes. As a result of the development process, a system is produced which is useful when delivered (Schwaber 1995).
Scrum helps to improve the existing engineering practices (e.g. testing practices) in an organization, for it involves frequent management activities aiming at consistently identifying any deficiencies or impediments in the development process as well as the practices that are used.
In the following, the Scrum phases are introduced according to Schwaber (1995; Schwaber and Beedle 2002).
The pre-game phase includes two sub-phases: Planning and Architecture/High level design.
Planning includes the definition of the system being developed. A Product Backlog list (see 3.2.3) is created containing all the requirements that are currently known. The requirements can originate from the customer, sales and marketing division, customer support or software developers. The requirements are prioritized and the effort needed for their implementation is estimated. The product Backlog list is constantly updated with new and more detailed items, as well as with more accurate estimations and new priority orders. Planning also includes the definition of the project team, tools and other resources, risk assessment and controlling issues, training needs and verification management approval. At every iteration, the updated product Backlog is reviewed by the Scrum Team(s) so as to gain their commitment for the next iteration.
In the architecture phase, the high level design of the system including the architecture is planned based on the current items in the Product Backlog. In case of an enhancement to an existing system, the changes needed for implementing the Backlog items are identified along with the problems they may cause. A design review meeting is held to go over the proposals for the implementation and decisions are made on the basis of this review. In addition, preliminary plans for the contents of releases are prepared.
The development phase (also called the game phase) is the agile part of the Scrum approach. This phase is treated as a "black box" where the unpredictable is expected. The different environmental and technical variables (such as time frame, quality, requirements, resources, implementation technologies and tools, and even development methods) identified in Scrum, which may change during the process, are observed and controlled through various Scrum practices during the Sprints (see the section below) of the development phase. Rather than taking these matters into consideration only at the beginning of the software development project, Scrum aims at controlling them constantly in order to be able to flexibly adapt to the changes.
In the development phase the system is developed in Sprints. Sprints are iterative cycles where the functionality is developed or enhanced to produce new increments. Each Sprint includes the traditional phases of software development: requirements, analysis, design, evolution and delivery phases. The architecture and the design of the system evolve during the Sprint development. One Sprint is planned to last from one week to one month. There may be, for example, three to eight Sprints in one systems development process before the system is ready for distribution. Also there may be more than one team building the increment.
The post-game phase contains the closure of the release. This phase is entered when an agreement has been made that the environmental variables such as the requirements are completed. In this case, no more items and issues can be found nor can any new ones be invented. The system is now ready for the release and the preparation for this is done during the post-game phase, including the tasks such as the integration, system testing and documentation.
Dynamic Systems Development Method
Since its origin in 1994, DSDM, the Dynamic Systems Development Method, has gradually become the number one framework for rapid application development (RAD) in the UK (Stapleton 1997). DSDM is a non-profit and nonproprietary framework for RAD development, maintained by the DSDM Consortium10. The developers of the method maintain that in addition to serving as a method in the generally accepted sense DSDM also provides a framework of controls for RAD, supplemented with guidance on how to efficiently use those controls.
The fundamental idea behind DSDM is that instead of fixing the amount of functionality in a product, and then adjusting time and resources to reach that functionality, it is preferred to fix time and resources, and then adjust the amount of functionality accordingly.
DSDM consists of five phases: feasibility study, business study, functional model iteration, design and build iteration, and implementation. The first two phases are sequential, and done only once. The last three phases, during which the actual development work is done, are iterative and incremental. DSDM approaches iterations as timeboxes. A timebox lasts for a predefined period of time, and the iteration has to end within the timebox. The time allowed for each iteration to take is planned beforehand, along with the results the iteration is guaranteed to produce. In DSDM, a typical timebox duration is from a few days to a few weeks.
In the following, the phases are introduced, with their essential output documents.
The feasibility study phase is where the suitability of DSDM for the given project is assessed. Judging by the type of project and, most of all, organizational and people issues, the decision is made, whether to use DSDM or not. In addition, the feasibility study phase is concerned with the technical possibilities of going through with the project, and the risks therein. Two work products are prepared - a feasibility report, and an outline plan for development. Optionally, a fast prototype can also be made if the business or technology are not known well enough to be able to decide whether to proceed to the next phase or not. The feasibility study phase is not expected to take more than a few weeks.
The business study is a phase where the essential characteristics of the business and technology are analyzed. The recommended approach is to organize workshops, where a sufficient number of the customer's experts are gathered to be able to consider all relevant facets of the system, and to be able to agree on development priorities. The affected business processes and user classes are described in a Business Area Definition. The identification of the affected user classes helps involving the customer, as the key people in the customer's organization can be recognized and involved at an early stage. High level descriptions of the processes are presented in the Business Area Definition, in a suitable format (ER diagrams, business object models, etc.).
Another two outputs are done in the business study phase. One is a System Architecture Definition, and the other an Outline Prototyping Plan. The architecture definition is the first system architecture sketch, and it is allowed to change during the course of the DSDM project. The prototyping plan should state the prototyping strategy for the following stages, and a plan for configuration management.
The functional model iteration phase is the first iterative and incremental phase. In every iteration, the contents and approach for the iteration are planned, the iteration gone through, and the results analyzed for further iterations. Both analysis and coding are done; prototypes are built, and the experiences gained from them are used in improving the analysis models. The prototypes are not to be entirely discarded, but gradually steered towards such quality that they can be included in the final system. A Functional Model is produced as an output, containing the prototype code and the analysis models. Testing is also a continuing, essential part of this phase.
There are four further outputs in the phase (at different stages in the phase).
Prioritized functions is a prioritized list of the functions that are delivered at the end of the iteration. Functional prototyping review documents collect the users' comments about the current increment, working as input for subsequent iterations. Non-functional requirements are listed, mainly to be dealt with in the next phase. Risk analysis of further development is an important document in the functional model iteration phase, because from the next phase (design and build iteration) onwards, encountered problems will be more difficult to address.
The design and build iteration is where the system is mainly built. The output is a Tested System that fulfils at least the minimum agreed set of requirements. Design and build are iterative, and the design and functional prototypes are reviewed by the users, and further development is based on the users' comments.
The final implementation phase is where the system is transferred from the development environment into the actual production environment. Training is given to users, and the system handed over to them. If the roll-out concerns a wide number of users, and done over a period of time, the implementation phase may also be iterated. Apart from the delivered system, the output of the implementation phase also includes a User Manual, and a Project Review Report. The latter summarizes the outcome of the project, and based on the results, the course of further development is set. DSDM defines four possible courses of development. If the system fulfils all requirements, no further work is needed. On the other hand, if a substantial amount of requirements have to be left aside (for example, if they were not discovered until the system was in development), the process may be run through again, from start to finish. If some less-critical functionality has to be omitted, the process may be run again from the functional model iteration phase onwards. Lastly, if some technical issues can not be addressed due to time constraints, they may be now done by iterating again, starting from the design and build iteration phase.
Other Methodologies (which are not discussed in this paper)
- Crystal family of methodologies
- Feature driven development
Methods and Materials
Project management approaches
There are a number of approaches to managing project activities including agile, interactive, incremental, and phased approaches.
Regardless of the methodology employed, careful consideration must be given to the overall project objectives, timeline, and cost, as well as the roles and responsibilities of all participants and stakeholders.
The traditional approach
A traditional phased approach identifies a sequence of steps to be completed. In the "traditional approach", we can distinguish 5 components of a project (4 stages plus control) in the development of a project:
Typical development phases of a project
- Project initiation stage;
- Project planning or design stage;
- Project execution or production stage;
- Project monitoring and controlling systems;
- Project completion stage.
Not all the projects will visit every stage as projects can be terminated before they reach completion. Some projects do not follow a structured planning and/or monitoring stages. Some projects will go through steps 2, 3 and 4 multiple times.
Many industries use variations on these project stages. For example, when working on a brick and mortar design and construction, projects will typically progress through stages like Pre-Planning, Conceptual Design, Schematic Design, Design Development, Construction Drawings (or Contract Documents), and Construction Administration. In software development, this approach is often known as the waterfall model, i.e., one series of tasks after another in linear sequence. In software development many organizations have adapted the Rational Unified Process (RUP) to fit this methodology, although RUP does not require or explicitly recommend this practice. Waterfall development works well for small, well defined projects, but often fails in larger projects of undefined and ambiguous nature. The Cone of Uncertainty explains some of this as the planning made on the initial phase of the project suffers from a high degree of uncertainty. This becomes especially true as software development is often the realization of a new or novel product. In projects where requirements have not been finalized and can change, requirements management is used to develop an accurate and complete definition of the behavior of software that can serve as the basis for software development. While the terms may differ from industry to industry, the actual stages typically follow common steps to problem solving — "defining the problem, weighing options, choosing a path, implementation and evaluation."
Critical Chain Project Management
Critical Chain Project Management (CCPM) is a method of planning and managing projects that puts more emphasis on the resources (physical and human) needed in order to execute project tasks. It is an application of the Theory of Constraints (TOC) to projects. The goal is to increase the rate of throughput (or completion rates) of projects in an organization. Applying the first three of the five focusing steps of TOC, the system constraint for all projects is identified as are the resources. To exploit the constraint, tasks on the critical chain are given priority over all other activities. Finally, projects are planned and managed to ensure that the resources are ready when the critical chain tasks must start, subordinating all other resources to the critical chain.
Regardless of project type, the project plan should undergo Resource Leveling, and the longest sequence of resource-constrained tasks should be identified as the critical chain. In multi-project environments, resource leveling should be performed across projects. However, it is often enough to identify (or simply select) a single "drum" resource—a resource that acts as a constraint across projects—and stagger projects based on the availability of that single resource.
Planning and feedback loops in Extreme Programming (XP) with the time frames of the multiple loops.
Extreme Project Management
In critical studies of Project Management, it has been noted that several of these fundamentally PERT-based models are not well suited for the multi-project company environment of today. Most of them are aimed at very large-scale, one-time, non-routine projects, and nowadays all kinds of management are expressed in terms of projects.
Using complex models for "projects" (or rather "tasks") spanning a few weeks has been proven to cause unnecessary costs and low maneuverability in several cases. Instead, project management experts try to identify different "lightweight" models, such as Agile Project Management methods including Extreme Programming for software development and Scrum techniques.
The generalization of Extreme Programming to other kinds of projects is extreme project management, which may be used in combination with the process modeling and management principles of human interaction management.
Event chain methodology
Event chain methodology is another method that complements critical path method and critical chain project management methodologies.
Event chain methodology is an uncertainty modeling and schedule network analysis technique that is focused on identifying and managing events and event chains that affect project schedules. Event chain methodology helps to mitigate the negative impact of psychological heuristics and biases, as well as to allow for easy modeling of uncertainties in the project schedules. Event chain methodology is based on the following principles.
- Probabilistic moment of risk: An activity (task) in most real life processes is not a continuous uniform process. Tasks are affected by external events, which can occur at some point in the middle of the task.
- Event chains: Events can cause other events, which will create event chains. These event chains can significantly affect the course of the project. Quantitative analysis is used to determine a cumulative effect of these event chains on the project schedule.
- Critical events or event chains: The single events or the event chains that have the most potential to affect the projects are the "critical events" or "critical chains of events." They can be determined by the analysis.
- Project tracking with events: Even if a project is partially completed and data about the project duration, cost, and events occurred is available, it is still possible to refine information about future potential events and helps to forecast future project performance.
- Event chain visualization: Events and event chains can be visualized using event chain diagrams on a Gantt chart.
PRINCE2 is a structured approach to project management, released in 1996 as a generic project management method. It combined the original PROMPT methodology (which evolved into the PRINCE methodology) with IBM's MITP (managing the implementation of the total project) methodology. PRINCE2 provides a method for managing projects within a clearly defined framework. PRINCE2 describes procedures to coordinate people and activities in a project, how to design and supervise the project, and what to do if the project has to be adjusted if it does not develop as planned.
In the method, each process is specified with its key inputs and outputs and with specific goals and activities to be carried out. This allows for automatic control of any deviations from the plan. Divided into manageable stages, the method enables an efficient control of resources. On the basis of close monitoring, the project can be carried out in a controlled and organized way.
PRINCE2 provides a common language for all participants in the project. The various management roles and responsibilities involved in a project are fully described and are adaptable to suit the complexity of the project and skills of the organization.
Capability Maturity Model, predecessor of the CMMI Model
Also furthering the concept of project control is the incorporation of process-based management. This area has been driven by the use of Maturity models such as the CMMI (Capability Maturity Model Integration) and ISO/IEC15504 (SPICE - Software Process Improvement and Capability Estimation).
Agile Project Management approaches based on the principles of human interaction management are founded on a process view of human collaboration. This contrasts sharply with the traditional approach. In the agile software development or flexible product development approach, the project is seen as a series of relatively small tasks conceived and executed as the situation demands in an adaptive manner, rather than as a completely pre-planned process.
Looking at Agile Methodologies from Project Management Knowledge Areas view
According to the PMBOK, "The objectives of Project Risk Management are to increase the probability and impact of positive events and decrease the probability and impact of events adverse to the project." The framework of the agile software development process fosters these objectives by making risk management an intrinsic part of the project lifecycle. Continuously identifying, analyzing, monitoring, and responding to risk triggers and risk events are part of the agile team's iterative planning discussions, which is to say that risks are addressed by everybody all the time. Daily stand-ups, iteration planning meetings, release planning meetings, and retrospective and review meetings are all venues for risk management activities on an agile project.
Agile teams can perform these risk management activities either overtly or organically. Overt risk management on an agile team is clearly stated as such, and things are labeled as risks, mitigation, etc. Organic risk management is that which is intrinsic to the agile process, and risk management emerges out of iterative planning and review activities. Potential risks are instead labeled as "obstacles," "assumptions," or "concerns." In the following examples I'll cite both overt and organic techniques used by agile teams.
The goal of risk management planning in the PMBOK is to create a plan that describes how the team will do risk management during the course of the project. In an agile environment there is no need to create a formal risk management plan; the method of addressing risk is built into agile procedures. The only decision the team may need to make is whether to conduct risk management activities overtly or organically.
In traditional project environments, risk identification is conducted in meetings with only a subset of the team members, who use checklists, document reviews, assumption analysis, and various other information gathering techniques to identify project risks and record them in a risk register (what we commonly call a "spreadsheet"). In an agile environment, the whole team does this exercise on an iterative basis during planning meetings, recording results on whiteboards or flipcharts. If the agile team is managing risk overtly, then an agenda item for the team to identify and prioritize risks is included as part of the meeting, with the results influencing the work that is being planned for that iteration. If the agile team is managing risk organically, then potential risks are identified as part of the agenda items "What assumptions are we making?" and "What concerns do we have?". Risks continue to be identified daily as part of stand-up meetings, either as obstacles (organic) or new risks (overt).
Traditional projects use both quantitative analysis (assigning real numbers to the costs of safeguards and the amount of damage that can occur) and qualitative analysis (using judgment, intuition, and experience in determining risks and potential losses). Agile projects generally perform only qualitative analysis, agile's short development cycles and constant reviews making this feasible and effective. The end result in both cases is a prioritized list of risks to respond to and risks to watch. In an agile environment these emerge from the planning meetings and are posted in a highly visible fashion, as a constant reminder to the team.
Risk Response Planning
Developing options and actions to reduce threats and increase opportunities is performed in both traditional and agile environments. The key difference is that in an agile environment the entire team participates in developing options and actions to reduce threats, a task that is conducted with more frequency than is common in traditional plan-driven projects. Many agile teams doing overt risk management follow Tom DeMarco and Tim Lister's recommended category breakout as explained in their bookWaltzing with Bears": Avoid, Mitigate, Contain, or Evade.
Risk Monitoring and Controlling
Risk audits, variance/trend analysis, and technical performance measurements are conducted at the end of each agile iteration as part of the iteration review meeting. This meeting provides a forum for the team to review the burndown chart, team velocity, and any other types of metrics the team may be noting. Risk reassessment occurs during the agile iteration retrospective meeting, where previous risks or concerns are revisited as part of determining changes that need to be made going forward. And finally, risks are monitored on a daily basis by the use of highly visible information radiators, such as task boards and burndown charts, which show the current status. Daily stand-up meetings contribute to the constant monitoring process by exposing potential risk triggers and new obstacles.
The team owns risk management in agile projects. The agile project manager facilitates the process and makes the results visible--whiteboards and flipcharts for collocated teams or in an online information-sharing tool for geographically dispersed teams. Risks are identified in all planning meetings: daily stand-ups, iteration planning meetings, and release planning meetings. Risks are then analyzed and addressed in these same iteration and release planning meetings, with the focus being on qualitative analysis rather than quantitative. Risks are subsequently monitored by the use of high-visibility information radiators, daily stand-ups, and iteration reviews and retrospectives. Risk management in an agile environment is incredibly successful, due to the team's involvement and the agile framework of iterative development that lends itself to active responses to risks and the continuous identification.
PIM is focused on both effectiveness and efficiency,and the project plan document is an essential of PIM. Instead of defining project plan at the beginning of the project,the agile project manager will do this in every iteration ,which is more suitable for an iteration within the fixed timebox and also can dramatically decrease the task of documentation. Release plans , iteration plans and other plans should be worked out by the whole team and are published in the most obvious way. Those parts should belong to the envision and speculation phases.
Due to the the aim of agile is only delivered to a requirement just before they are needed,there might be some changes during the software development. Backlog is just to ensure that all the feature and changes are under control by the product manager and costumers. Backlog contains functionality requirements ,features,technique and inperfections ,which are in ordered.when one iteration finished, the items which are in highest will be programmed and completed in the next iteration.
Because of the integration management , each elements or person in the development environment can absorb and penetrate mutually. The effectiveness is far greater than the simple sum of the individual elements.
Project Scope and Time Management
In the conventional project development process, "scope creep" is really bothering the project managers. Since the plan document is done, if requirements are needed to change or added continuously, technique are needed to improved and so on, the whole process of the development are disturbed. That would be a problem for project scope management, even the project risk management. However the agile development is flexible and stable, scope change is an acceptable even encouraged. Because the agile development is value incremental works by completing several iterations. Due to the plan is done before the iteration, it should beblock off within the timebox, which will be easy for the project management to keep an eye on the features change or added to in case of over scope.
For the time management, it can be strictly controlled by the iteration timebox. The programmer must critically obey the time. When they does not complete at the end of the time, they must choose to drop some features to deliver on time. They can add the features which are throw away into the next iteration.
The project leader should add some extra time during the whole system development, because it can not be avoid encountering some problems or revisiting the program.
· Change Management
Within the Agile world, change is expected and time is considered more important than functionality. So if something has to give to allow change, then functionality/scope loses and time wins, i.e. the customer must make tradeoffs.
The Customer directs development in timebox and minor changes are just accepted. Big changes are handled different depending on whether you're in a timebox or outside it, and depending on the method you're using.
Big changes outside a timebox
- Customer can add/remove/change any requirement
- Developers re-estimate
- Customer trades off requirements to fit change in Project Plan Big change inside a timebox
- Either isn't allowed at all (Scrum)
- Or customer can trade functionality in/out of timebox.
- Handle change as a new Story and replan (XP)
- Use MoSCow priorities which are revised at the 3 objectives Setting Meetings within timebox (DSDM).
Human Resource Management
Agile Developer - Life in a different world
As an Agile developer you undertake a variety of tasks (analysis, estimating, design, coding, testing) and generally do what it takes (which is pretty much all Scrum uses for a job definition). You are Client facing, in fact client facing every day as the customer is one of the team. There is less documentation to write, unless you want it. There is stress, e.g. short time frames, but this is managed (for example, XP's 40 hour week). On the other hand there is more visibility of your own work and progress: Pair programming (XP), Peer Reviews (DSDM), User Reviews (XP, Scrum, DSDM), Daily meetings (XP, Scrum). You work faster and produced better quality. You're part of a team with a shared aim and shared responsibility.
Stakeholders (User/Customer) - An Active Role
Life is different for an Agile Customer as well. You are part of the team and must actively participate. You are responsible for the requirements. You must Talk to other users and get their agreement, Write up the requirements (if necessary), and explain them to developers. You have a planning role too as you must prioritise and trade off requirements. During development you provide input into design and prototyping sessions. As the software is produced you must review and accept delivered functionality; you must also organise, control and do user testing. Finally you must provide user documentation and handle user training.
Project Manager as Shepherd
The role of the Project Manager is subtly different when using an Agile approach. From my fairly biased view an Agile PM is more a shepherd (or sheep dog) and less a military officer. There is lots of rushing around the edges of the development team but relatively little direct control of what the team does. If you merge together a description of the Project Manager, Scrum Master, Coach, and Tracker from XP, DSDM, Crystal Orange and Scrum you get:
Gather information from all stakeholders and integrate into workable plan; log estimates; advise programmers on their estimates using historical figures; log task assignments; sustain high rate of progress; make decisions; remove impediments; build and sustain team culture; ensure team sticks to process; liaise between management and team; manage customer relationships; fend off "feature creep" and other project hazards; get agreement on the prioritisation of requirements and detailed content of timeboxes; track progress against the timebox goals/plan; collect information without disturbing the process; update plans; communicate progress; ensure that the lessons are learnt; log defects; stay calm.
Things like "remove impediments" and "build and sustain team culture" aren't normally seen in traditional PM role descriptions. Then there are the softly, softly items like "collect information without disturbing the process" and "get agreement on ..."; and notice we "log task assignments" we don't "assign tasks". However, when the crunch comes we must still "make decisions".
Quality must be addressed continuously within Agile Development. There are stated Quality Goals. In XP the Product must pass 100% of unit tests and an acceptable number of functional tests. In DSDM there are documented quality criteria for all work products. Peer Review is mandatory in XP (in the form of Pair Programming) and common in DSDM. XP uses Xunit for automatic Unit Tests and insists on tests before code. In XP and Scrum integration Testing should be continuous and automatic, with a daily build and smoke test. User Acceptance Testing is in parallel to development. The Customer guides development (XP, DSDM), there is UAT during development (XP, DSDM) and formal reviews during timebox (DSDM) and at end (XP, Scrum, DSDM).
Extra testing activities like performance testing are scheduled in the same way as requirements
Testing gives us some certainty that the software does what it is meant to. But there are different types of testing, done at different times, often done by different types of people.
Test Driven Development is a practice advocated by XPers that involves writing a failing automated test before changing any code. This is a another XP practice I don't mandate. Despite the fact I personally like this practice it is something that I find comes late in the progression to Agile maturity and for more novice teams it just adds confusion.
Night Build and Smoke Test and Continuous integration provide most value if the tests are automated. Most unit tests and many acceptance tests and system tests can be automated, and others may be automated as well. Exploratory tests are never automated.
An Agile Team does unit, acceptance, system, exploratory and regression testing within each Timebox alongside coding. Other types of testing require extra hardware, people, organisation, funds and/or specialist skills that are generally not available to the team, at least not in every Timebox. These types of tests are usually carried out outside of Timeboxes and often by specialists.
One of the key goals of this second edition of Agile Project Management is to discuss performance measurement issues and introduce the Agile Triangle as a replacement for the traditional Iron Triangle of project management. If change, adaptation, and flexibility are the trademarks of agile projects, and conforming to plan is the trademark of traditional projects, then why do we still measure success on agile projects using the traditional framework? This may be the key reason that thwarts wider agile adoption in organizations: Agile teams are striving to meet one set of goals and managers and executives are measuring against another set. Until this gap is bridged, agile project management and development will be relegated to "just another technical approach" status.
Measuring success it tricky. Motorola's ill-fated, multibillion-dollar satellite-based Iridium project was a spectacular failure in the market. Meanwhile, the movie Titanic, which was severely over budget and schedule—and viewed by early pundits as a $200 million flop awaiting release—was the first movie to generate over $1 billion in worldwide revenue. By common constraint-based project management measures of success—scope, cost, and schedule—Titanic was a failure. Within some circles, Iridium was considered a success because it fulfilled the original specifications. Using the agile triangle, the Titanic project would be considered a success—it delivered value even though it exceeded its constraints. Iridium would have been considered a failure because it failed to deliver value, even though from a technical perspective it succeeded.
So to attain significant levels of improvement in an organization, we have to address how performance is measured. The evolution to an Agile Triangle shows the transition to a new way of measuring success. The traditional iron triangle of project management, the left-most triangle, consists of scope, schedule, and cost. In many cases scope was the primary driver (because the false assumption was made that scope was known early in the project) and cost and schedule varied—although many managers attempt to lock down all three dimensions.
The second triangle represents an early view of measuring agile development where schedule was fixed (timeboxed) and scope was allowed to vary—that is, time was used as a fixed constraint. Unfortunately, this second triangle still conforms to the existing iron triangle measures. Success, in many organizations and perpetuated by research organizations like the Standish Group, is still viewed as conformance to cost, schedule, and scope. If conformance to plan defines success, then how will agile projects that adapt continuously ever be deemed successful?
Simple and direct communication is key
- Co-location in open area
- Daily Stand up meeting (XP), Daily Scrum meeting (Scrum)
- Big visible charts (XP), Backlog Graph (Scrum), Information Radiators (Crystal Orange)
- Co-location in open area
- Daily Scrum of Scrums meeting (Scrum)
- Big visible charts etc
- "Inter-team spec" (Crystal Orange)
- Frequent delivery
- Big visible charts etc
- Scrum Observers (Scrum)
- "Status Report" (Crystal Orange)
The big visible chart shown is a Sprint Backlog Chart from Scrum. The X-axis is the day of the Sprint (timebox) and the Y-axis is the remaining work in days (Scrum calls this a backlog).
Time and Cost Management
As agile methods (for both hardware and software products) become widespread in organizations, the debate over serial, waterfall life cycles versus iterative life cycles is moving from an engineering-level to an executive-level discussion.
Fundamentally, governance is about making investment decisions in an environment of uncertainty. Executives have to make investment decisions, defined in terms of return on investment (ROI), and assess the probability attaining that ROI. ROI includes three components—value produced (inflows of money), costs expended (outflows of money), and time (timing of inflows and outflows). Executives have to answer two basic questions: What is the projected value or return on investment? What is the probability that this return can be achieved? Investment decisions are linear: Spend some money, receive some results, decide on the next investment increment. Dollars and time are not iterative: When they are spent, they are gone.
Operational delivery is about defining the best methods of delivering project results. Engineering, whatever the product, is inherently iterative—think a little, try an experiment, observe the results, revise. Sometimes the iterations are long, sometimes short, but engineers have never really operated on a linear, waterfall model—unless forced to by an organization process. When development was sequential, the need to differentiate between governance and operations was masked. However, as organizations began to implement iterative methods, the disconnect between governance and operations began to cause a friction between executives and project teams.
Governance is, or should be, separate from operations—although they are definitely linked. Operational delivery is about assembling the best set of methodologies, processes, practices, and people to deliver results. Although the governance framework should be common to all projects within an organization, the operational delivery approach should be matched to project type. It is important to note that executives play a vital role in both the operational delivery of products and in governance. Separating governance from operations doesn't negate the executive role in each.
Investment and Risk
Product development balances opportunity and risk—the opportunity to generate significant ROI, balanced with the probability that something will intervene to undermine the opportunity. That a huge percentage of new product development (NPD) projects fail speaks to its difficulty. Executives use various information to make project funding decisions. Some known information can be assembled into planning documents, but the crux of product development rests on discovering the unknowns, the solutions to problems that have yet to be identified.
There are two fundamentally different types of projects (although there are many variations)—production and exploration. Production projects are characterized by a known problem and a known solution. For production projects, because the solution is known, careful planning can reduce much of the project's risk. Exploration projects are very different. They are characterized by unknowns—there is a known problem and an unknown solution, a unknown problem and a known solution, or an unknown problem and an unknown solution. Both ends (the objective or problem) and the means (solution) can have significant uncertainties associated with them. For example, just because we can specify the problem doesn't mean we can design a solution. As an extreme example, imagine that plans for a new spacecraft specifies a top speed of 195,000 miles/sec. Because this specification exceeds the speed of light, coming up with a solution may be tricky. We might be reluctant to commit $14 billion to this project, no matter how high the projected returns, until we found a way to overcome this design problem.
As another example, postulate that product management in an electronics company decided that the next version of their product needed to be 50% smaller in size. Further assume that the 50% size reduction was both a critical requirement for the new product and that the engineers were concerned about meeting the size constraint with a cost that had acceptable gross margins. Solving the design issue would be critical to the continued viability of the project. A significant amount of engineering and prototyping would be needed to answer the design question. Action, not planning, produces information when problems are known but solutions aren't. So the decision might be to conduct a proof-of-concept phase for the project in which the engineers would try to solve the design problem by building prototypes or doing simulations. Other parts of the project would be delayed until this key question had been answered.
For such a project, executives might authorize a $100,000 expenditure for a concept phase to gather enough information to reduce the risk of making a five million dollar full product development decision. Executive decision making follows this scenario through the various phases—define an information gathering strategy based on key risks areas, then decide which activities to fund based on mitigating those risks as early and with as little expenditure as possible. From an executive perspective, this model is serial—spend money and time, obtain information, decide on continuing the project.
However, from an engineering perspective, a linear model doesn't serve well. In the electronics example, a serial model would have focused initially on specifying the requirements for the entire product in detail. At the end of this requirements phase (at which point possibly 20-25% of the cost would be expended), the executives would have relatively complete (in theory) information on the product's requirements, but no information on the critical size design issue. After expending $900,000, they wouldn't have reduced the key risk. The team that actually did some design work, constructed a prototype, and tested it might find the key problem was in testing—electrical interference from a component that they hadn't suspected. In a serial project, the team might not have discovered this critical issue until 80% or 90% of the project had been completed and would probably cause significant delays. By developing an iterative prototype, the problem was uncovered 10% of the way through the project for $100,000 and did not cause a project delay.
The funding model for projects should focus on what executives need to carry out their oversight and fiduciary responsibilities. They need a systematic way to view information gathered at key intervals to make the best investment decisions based on their understanding of the risks involved. These intervals are defined by phases (the period when work occurs) and gates (when decisions are made). For each gate the key decisions, decision makers, and information required to make the decisions are defined. Gate reviews aren't about deliverable check-offs; they are about providing executives and managers with relevant information to make decisions about continued funding and acceptable risk.
Executive-Level Information Requirements
Figure 12-1 shows a desirable progression in information gathering across a project of three phases. (The first set of figures on the graph is pre-project and we won't bother to name the phases at this point because they are irrelevant.) Let's first look at the progression from an executive perspective. Prior to the project, the risk factor is high at 100 (on a relative scale), whereas the investment, architecture, and features delivered are at 0%. This makes sense because prior to the project no risks have been mitigated, no planning has been done, no costs have been incurred, nothing has been built. The customers may have some ideas about target features and desired schedule, but that is all pure speculation at this point.
Even though the progression in this example is desirable, every project has a different profile. For certain projects a significant amount of money might be spent before the risk can be significantly reduced.
So what would executives like out of a first phase? Basically, they would like to spend as little money as possible to reduce the risks (not just identify them but actually reduce them) and gain a better understanding of the project's feasibility. The results of Phase 1 in Figure 12-1 indicate that this executive goal was achieved. The project risk has been reduced from 100 to 40 (for example, three critical high-risk items have been mitigated by building engineering breadboards and actual development of the software features required for the breadboard), 20% of the cost has been expended, 25% of the architecture (a skeleton architecture) has been defined, and 10% of the actual product features have been built, tested, and reviewed with the customer. Of all these accomplishments, the fact that high-risk elements have been mitigated by building and reviewing actual product features is a critically important component of the information gathered.
During Phase 2, even more progress was made. For expending 40% of the cost, the risk factor has been reduced to 20 (some of the lower impact risks are still left), 75% of the architecture is in place, and 30% of the product has actually been built. Although some executives might be concerned that for 40% of the cost only 30% of the product has actually been built, the key is that the risk and uncertainty have been driven down to a point that the remaining expenditure has an extremely high probability of delivering a releasable product on time, plus much of the architecture work was completed.
During Phase 3, the product was completed, but some small amount of risk, uncertainty, and cost remain for final product deployment. During Phase 3 the bulk of the work was reasonably well understood—no critical design problems were deferred to this phase. The risk of Phase 3 failing was very small, and yet 60% of the cost was incurred during this phase—a very appealing prospect to executives.
This three-phase example could be construed as an incremental funding model, but in reality, every project has incremental funding. Even in a waterfall project, executives retain the ability to cancel the project at the end of any phase. The difference in the agile approach is that it produces incremental product, not just documents from activities (planning, architectural definition, requirements definition, etc.). The focus of agile development is on systematically delivering high-value product features and reducing risks—exactly what executives are looking for.
As a final example, a client was building an application to support a well understood accounting process. However, the entire architecture of the application was changing from a Cobol, DB2, mainframe-based system to a multi-tiered web architecture, C#, SQL Server environment. This project had a relatively stable and knowable set of business requirements, but a risky technology implementation because it was new to the staff. In a waterfall approach, the team might spend 20% of the cost to gather and document the requirements. However, this would have little impact on reducing the risk and uncertainty of the project. This client's approach was to take a couple of small business features and go through all the activities to implement them across the entire architecture. The team was able to do this for 5% of the cost, learn the ins and outs of the technology, and thereby significantly reduce the risk factors.
Engineering-Level Information Generation
As you saw in Figure 12-1, managing projects might be simply defined as buying information. The critical success factor is buying the right information for the right price. As you have also seen, spending lots of money for planning and requirements documents, in contrast to spending money for working products or software, is rarely investment/risk-efficient.
The software development community uses the term "loose coupling" in design to indicate that two modules operate together, but that they are independent and therefore can be easily modified independently—linked, but not integrated. Project governance and operational delivery should be viewed in the same fashion—linked, but independent.
Phase-gate funding models can promote the idea that investment and risk can be managed best by a linear model, while at the same time promoting an iterative model for operational delivery. Agile projects, which consist of iterative Envision and Explore cycles, produce exactly what executives are looking for to make funding decisions. By separating, but linking, a linear funding model with an agile development model, the needs of engineers for an innovative development process and executives for critical information to make investment and risk decisions are both fulfilled.
An Enterprise-Level Governance Model
In the last section a generic phase-gate life cycle was shown, but for most companies a little more definition than Phase 1-2-3 is required. If we are going to decouple a linear funding model from an iterative development model, then we should use phase names that do not reflect activities (requirements, design, construction), but investment and risk mitigation phases. Four-phase life cycle can serve as a high-level governance model that will work with multiple operational-level delivery methods. The four phases are Concept, Expansion, Extension, and Deployment (which is somewhat activity based). The objectives of each of these phases should follow the information progression outlined in Figure 12-1.
Note1: The Rational Unified Process uses a similar phase progression—Inception, Elaboration, Construction, and Transition. However, the phases are not defined in the same investment/risk language, the "Construction" phase title detracts from the underlying iterative nature of RUP, and the Inception phase doesn't emphasize risk mitigation but is more focused on risk identification.
Note2: For long projects, the Expansion and Extension phases could be further broken down into 3-6 month sub-phases.
No projects, even agile ones, should start instantly, but they should start quickly. The Concept phase has two overriding objectives—to create and confirm the vision for the product and to identify and mitigate risk. The vision may contain product, marketing, financial, and team components. Risk areas include marketing (can we sell it?), technical (can we build it?), and financial (can we make money with it?). The Concept phase can be considered a proof-of-concept endeavor, not just a feasibility study or risk identification activity.
The Concept phase includes work identified earlier in iteration 0 such as capability-level requirements, development environment preparation, and architectural sketching. The Concept phase consists of an iteration 0 (no actual stories delivered), and a few short development iterations (stories delivered). Helping a client launch several complex medical instrument projects (hardware and software) in which high-risk research ideas were crossing from the lab into engineering convinced me of the need for a Concept phase.
The goal of the Concept phase is to identify—AND mitigate—enough high-risk and uncertain items to make it possible to realistically plan the rest of the project. Often a full release plan can't be done until the Concept Phase is completed—there are just too many unknowns.
The Concept phase is far more than a traditional feasibility phase—it should demonstrate proof of the concept, not merely analyze finances as in most feasibility documents. Many methodologies use a concept-like phase to identify risks, but what I learned from the hardware developers is to use this phase for actual risk reduction, not just identification. On these projects there were so many unknowns and uncertainties, many of which were design issues, there was no way to do an overall project plan in the beginning (for some of these hardware projects the Concept phase was six months or more because they were bringing ideas out of the research lab and trying to determine whether they really could productize them). The Concept phase is a mini-project—a planning cycle of visioning-scoping-release planning (for a few iterations) and a delivery cycle of iteration planning-developing features-adapting. It is a mini-project to gather enough information about the full project to determine whether or not to proceed.
Expansion means just that—expanding on the work done in the Concept phase, specifically expanding to areas of lesser risk than those covered in the first phase. For example, if the highest risk item was proving that all the technology worked together and that it was adequately addressed in the Concept phase, then the Expansion phase might focus on broad capability implementation to confirm that the project's scope was well understood. The objectives of the Expansion phase should be to drive out remaining risk items from the project and build high-value features.
By the end of the Expansion phase executive management should expect the completion of the project with few surprises. There will always be projects in which there remains final integration and use risks, but teams should make every effort to drive out absolutely as much risk as possible before transitioning to the Extension phase.
The objective of the Extension phase should be to do more of what we already know how to do. There should not be many, if any, surprises in this phase, even though up to a half of the total budget might be expended in this phase.
As with nearly every life cycle process model, the Deployment phase is where the product is deployed into actual use. The Deployment phase could be the "final" deployment as many agile/iterative projects deploy incremental pieces of the product.
Although most development life cycle approaches concentrate on phases, focusing on gates is more effective. Gates are critical decision points in a project when the team reports information and the executive review team considers whether or not to invest in the next phase. At any phase the decision may be to pass the project into the next phase, revisit the prior phase to gather more information or further reduce risk, or to terminate the project. For example, at the Extension gate (transitioning from the Expansion to Extension phases) the types of questions that the executives should be asking are
- Have all the significant risk items (marketing, technology, financial) been mitigated through the delivery of working features and other investigations?
- Has the product architecture stabilized?
- To what extent have the highest risk items (marketing, technology, financial) been mitigated through the development of architectural spikes, delivery of working features, and other investigations?
- Has the work on a skeleton architecture and any development spikes convinced us that the project is technically feasible and will operate within required performance specifications?
- Can the Expansion phase be estimated with a reasonable degree of confidence in having a releasable, quality product?
- Are the estimates for the entire project still within its constraints?
At the Expansion gate the questions might include
Using the Agile Governance Model
Organizations have three options in using the Agile Governance Model: 1) Don't use it, 2) Use the Concept phase, 3) Use the entire model. For many organizations, especially those with smaller projects or limited product lines, not using any of the phase-gate model may be the appropriate course of action. In these organizations, probably those on the smaller end of the size scale, in which all teams are using the same agile methods, using an iteration 0 and the standard project control reports may be completely adequate.
However, even if your organization doesn't need the entire model, using a concept phase, even for small projects, can be very beneficial. There are cases, where the technical or requirements uncertainty (or both) is high enough that developing a release plan is just not feasible. In many of these cases, a one- or two-month concept phase can significantly reduce uncertainty and provide greater confidence in the release plan when it is prepared.
Finally, in large organizations, particularly those with large projects, projects being developed with a variety of methodologies, and those that use stricter financial controls on project portfolios, using the full agile phase-gate model can be beneficial. However, as mentioned earlier, implementing such a model should be done with the agile principle of simplicity in mind—don't introduce unnecessary bureaucracy into the organization.
Advantages and disadvantages of Agile Methods
Scrum is light process framework that can be complemented with practices and processes from other more detailed defined methods like XP. Self organized teams are the core of the method and the team should take much responsibility in achieving the targets. Scrum itself does not define any programming practices for the implementation phase but it defines what is expected from the team, and some basic practices that the team should follow in order to achieve its goals (for example 15 minute daily meetings with accurately defined questions). This promoting of the team work and self organization can be seen as major advantage since it should raise the motivation level.
In Scrum any changes are denied after the sprint requirements have been defined in the beginning of each sprint. For the team this is good since it provides some peaceful time to perform all the tasks needed to complete the sprint requirement list. On the other hand, for example sprint of one month may cause unacceptable delay, if some major customer is asking a quick solution for some small specific problem. Actual worst case delay may be then nearly two months before the solution is seen in release. 40
Scrum breaks the problem in hand to small parts that are manageable by the team. The daily builds, constant integration, and testing will guarantee that the bugs are found.
Major advantage of XP is that it is widely used and there exists lot of different information sources. XP defines process with frequent builds and iterations and goes into more details in actual management and programming practices than Scrum. Disadvantages are that the method is applicable for small teams (5-10 programmers only) and that the on-site customer requirement cannot be fulfilled in our case. The organization of the team has already spread over several locations and therefore the one-site requirement presented in XP is not applicable either.
XP is somewhat free-formed allowing the developers to address new issues or requirements on the fly, which could be useful in some situations requiring rapid reactions to customer needs. The amount of formal meetings is minimized, the daily stand up meetings are seen as effective way of sharing the information.
The code quality is under constant peer review because of the pair programming. On the other hand some individuals may resist pair programming at least during the start-up period.
XP enhances the production efficiency by reducing the amount of documentation, which may be seen either good or bad thing. The product may be ready sooner, but in worst case without updated documentation to be provided for the customer.
XP has some useful practices which could be taken into use in parts of the team but it does not provide a complete solution for our team.
DSDM is the most formal of all the agile methods and requires more documentation as well. The process contains much architectural design in the beginning of the projects. Testing is addressed heavily and each project team is required to have at least one test engineer, which is good.
Business value is expected to have highest priority and a specific approach is presented to define how important a certain requirement is for on-going iteration.
Major disadvantages are that the process seems to be very heavy and that the access to material describing the practices in detail is charged and controlled by Consortium.
- License to hack (Highly ordered environments),
- Can not use in outsourcing and off shoring,
- Can not use in Safety-critical environments,
- Can not use in project that Experts are not available)
- Can not use in Cultures that there is no Right personalities.
- Can not use in Distributed environments
- Can not use in Stable environments (rework could be too expensive)
One big problem when transferring to the agile was developers' resistance of the changes. Resistance existed no matter which way to transfer to the agile; in one go or gradually with a few practice at the time. Resistance of the changes fell naturally off in process of time when new method was learned and when it was noticed, that there were also good aspects.
Every interviewee found good aspects from agile, nobody thought, that it is better to return back to the old way of work. Pair programming was found to be a good practice, though it was not used officially in any of the teams. Especially continuous code review of the pair programming felt as a positive thing, which brings down the number of the faults. One team used pair programming mostly when doing difficult tasks, for example when developing new functionality to the product.
Also common code ownership was brought up as a thing which improves the competence transfer to all team members. This assures that for example in case of acute illness, developing of the program can continue, because basically anyone can carry on the work of other team member.
One positive thing what agile brought along was that every member of the project knows in more detailed level what is the situation of the developing process and project. Daily meetings versus normal weekly meetings helps everybody to be more familiar with meaning of the own work and with the general view of the project.
Agile project demands that a customer commits to participate to the project for the whole project life cycle. Customer must be a one member of the team. Otherwise it might happen, that program is developed differently from the desires of the customer. Hence the kinds of changes are accepted to the program, which customer does not prioritize and important features might be left off against customer's will. It is important that the customer get the product they need, and the customer should also know what kind of product they are going to get.
Combining Scrum and XP
Scrum would be the most natural selection for an agile method to take into use in our team, since some competence to take it into use already exists in our team at the moment and it has been used in several other teams in our company. We could find a "Scrum mentor" from our company to guide our team into use of Scrum and give us help in troubled situations. This sort of a person is needed in the first projects, when the process is developed and tuned to meet our needs, and we do not ourselves have the know how to do the analysis and tune the process into the right direction. Mentor has to be someone outside of the team to be able to give fair recommendations how the team practices should be developed.
However Scrum is a framework for project management and it misses detailed practices for everyday work. For instance XP could provide some more detailed practices that could be useful and taken into use in development of our product. It is possible to complete Scrum with XP practices, and there exists several studies and user experience on the subject, which support the opinion that this might be good solution for our team also (Sliwa 2002). Testing and agile expert Elisabeth Hendrickson gave a lecture on 20.6.2007 and recommended combination of Scrum and XP for getting the best possible outcome from agile methods. For example ADM (Advanced Development Methods Inc.), the enterprise behind the Control Chaos web-pages, has developed XP@Scrum method.
How to make Agile work in large or complex organizations or environments?
As agile methods have spread to larger projects and larger organizations over the last five years, a more encompassing enterprise framework has become necessary. In an organization that spans multiple countries, for example, not all projects will be agile projects and even if they were, some locations may use different agile methods than others. It isn't rare for one organization location to use Scrum, another Extreme Programming (XP), and yet another Feature Driven Development (FDD). Furthermore, this multiplicity of approaches should not be discouraged! It may well be that a location in China can get good support (training, coaching, etc.) for Scrum, whereas another in Australia can get good support for FDD.
One of the tenets of agile development is adapting to different situations. One of the six principles of the Declaration of Interdependence is, "We improve effectiveness and reliability through situational specific strategies, processes, and practices." It is therefore difficult to make a compelling argument for standardizing on a single agile method across a multinational organization. However, utilizing a common framework within which individual choices can be made does appeal to larger organizations.
An Agile Enterprise Framework
The overall Agile Enterprise Framework is shown Figure 5-1. The Portfolio Governance layer can offer some common checkpoints, and the Project Management layer can offer guidance to managing a variety of projects. The differentiation between the Project Management and Iteration Management layers can offer insight into differences between running a project and creating a release plan versus the day-to-day management of a short iteration. Finally, separating the Iteration Management layer from Technical Practices can assist in melding a core of technical practices into several project or iteration management approaches.
This structure facilitates organizations in building hybrid agile methods to suit their specific needs—positioning different agile methods for each layer. The framework encourages flexibility at the lower tier (technical practices) and less flexibility at the upper tier (project governance). This structure recognizes that no one agile method covers all the tiers—that in fact, all agile methods used in organizations are hybrids. For example, one organization might use APM for the Project Management layer (plus parts of the PMBOK), Scrum for Iteration Management, and selected XP practices in the Technical Layer. By drawing on the strengths of several agile methods, companies can build highly effective hybrid methodologies, or build several hybrids for different parts of the organization.
Portfolio Governance Layer
Large companies have hundreds, if not thousands, of projects—some agile, some traditional; some using one agile method, some another agile method, some a hybrid of agile and traditional methods. Even when an organization has committed to a major agile transformation there will be a several-year transition period when there will be a mix of methods used. What executives need is a common framework for evaluating all these projects; a framework that addresses the major executive concerns—investment and risk. Executives want to know the value of the project (in terms of ROI), and the certainty or uncertainty of obtaining that ROI. Executives don't really care whether a requirements document has been finished; they want to know about project progress, investment, and risk. No matter what the project type—agile or otherwise—a governance mechanism can be created that addresses these two key project status attributes.
Project Management Layer
Many people think about project versus iteration management as dealing with stakeholders external to the core team versus dealing with internal core team members. Although that is part of the differentiation, it is only half. The other critical differentiation between these two is managing the release versus managing the iteration.
Project management also includes working with stakeholders and suppliers outside the core team. Therefore, the Project Management layer focuses on overall project/release activities, assisting coordination among multiple feature teams, and managing the project externals. In addition, project management practices such as risk analysis or contract management practices, that are useful in a variety of projects, agile or not, can be brought into this layer (they may come from sources such at the Project Management Institute's Project Management Body of Knowledge).
A quick note here is that the Project Management and Iteration Management layers can have the same leader or different leaders, depending on the project's size. For example, a large project with four teams might have an iteration manager for each team and one overall project leader.
Iteration Management Layer
Iteration management focuses on planning, execution, and team leadership during short individual iterations. The reasons for separating the Iteration and Project Management layers, basically separating release and iteration work and internal and external management activities.
Technical Practices Layer
Technical practices, in software projects, run the gamut from continuous integration to pair-programming, from test-driven development to refactoring. Hardware projects might employ a range of engineering practices, from electrical to mechanical. The other three layers, the foundation of effective project delivery lies in the technical arena. In implementing agile methods in a wide variety of organizations, transforming technical practices are critical. For example, continuous integration and ruthless automated testing are core agile software practices that can't be left out.
Another reason for separating out a Technical Practices layer is to make APM more amenable to a variety of project and product types. Although I've had difficulty getting electrical or mechanical engineers to warm up to pair programming, equivalents of agile software practices have proved useful in a wide variety of product development areas. Furthermore, except for potentially longer iteration lengths in hardware projects, the Governance, Project, and Iteration layers work for companies wanting to apply agile methods to non-software projects.
Agile Methodology "Fit"
In a large organization, not every project will be agile. Determining what projects do fit an agile profile, and how agile practices should be adapted for different types of projects, is an important portfolio management topic. There are a number of models for selecting a methodology or set of practices to match a specific project. In his Crystal series of methodologies, Alistair Cockburn (2006) identifies a range of methodologies—clear, yellow, orange—that are determined by size, criticality, and other factors. In Effective Project Management, Bob Wysocki (2003) identifies uncertainty as the key factor in determining whether a traditional, adaptive, or extreme project management model fits best. Todd Little (Pixton, Little, Nickolaisen and McDonald 2009) from Landmark Graphics uses a quadrant analysis (in which the axes are uncertainty and complexity) to match project type with approach. Frameworks that included organizational culture were presented in Highsmith (2002) and Boehm and Turner (2004).
In examining all these approaches, there are three critical elements in determining methodology fit:
- Project factors (complexity and uncertainty)
- Cultural factors
- Governance and compliance factors.
For project factors, which should dominate methodology selection, I like Todd Little's model. In this model, complexity factors include team size, team distribution, mission criticality, and domain knowledge gaps, whereas uncertainty factors include market uncertainty, technical uncertainty, and project duration. Each factor has a 1-10 value, and then all factors are combined and used to plot projects to a quadrant.
Managing increasing uncertainty is best accomplished by agile, flexible practices, whereas managing complexity requires additional structure. The most difficult projects are those that have high uncertainty, requiring greater agility, and high complexity that requires additional structure. Finding project leaders who can handle both agility and structure is a challenge.
The second aspect of methodology fit has to do with culture. Highly uncertain projects are best handled by organizations with agile, flexible, collaborative cultures. Organizations with very structured, conformance-to-plan cultures will have difficulties with highly uncertain projects. Similarly, a very low formality, flexible, ad hoc culture may have difficulty with highly complex projects.
As organizations look at their project portfolios, they first need to look at project type and determine the best approach based on the requirements of the project. They then need to assess their culture to see whether it fits. Unfortunately, many companies are unwilling to face this second aspect of fit: They try to impose a single methodology on a multitude of project types and cultures. Splitting project governance from operational delivery will help organizations address this issue by enabling them to create a single governance model fed by multiple operational models.
The third aspect of methodology selection is governance and compliance factors. The biggest mistake organizations make is letting governance and compliance factors drive the development process rather than figuring out the best development process and then adding practices for compliance. For example, several years ago a company that produced medical software felt that FDA compliance dictated a waterfall lifecycle. After some discussion, they figured out how an iterative lifecycle was both best for their development and could satisfy the compliance requirements.
Governance and compliance factors are part of the cost of doing business today. They are important and necessary (at least some of them are), but they should not drive the engineering processes. Engineering practices are the ones that deliver customer value. Compliance and governance processes add cost—a necessary cost of doing business, but a cost nonetheless. Focusing on engineering practices, streamlining them as much as possible, and freeing them from compliance work should be the winning strategy. Offloading as much compliance and governance work from the team as possible helps deliver value and meet compliance demands in the most efficient and effective way.
These "methodology" fit factors should be used as a guide in determining what type of method to use on a project, but they may also have a greater use in developing hybrid methodologies to use on multiple types of projects. If we look at all the levels—project governance, project management, iteration management, and technical practices—and different sizes and distribution of projects, then the range of useful practices—both agile and traditional—is very wide. The factors discussed in this section can help managers and teams adapt practices to their project at hand.
Disappearing the Manager Role
By development of Agile methods and their utilization in almost all types of projects (Software development and other types of projects), the need for expert software which can handle all the responsibilities of project manager in project management field such as control, monitoring and navigating is essential.
In our opinion, in near future projects and especially agile projects which can managed by software will use expert software to handle managing tasks, because the nature of agile projects let us to do this.
By developing such software, the position of project manager is going to disappear and their responsibilities will delegated to expert software which is more reliable and fast.
Using Agile Methods in Atypical Areas
There are new types of projects that have the potential of using Agile methods for their development, for example developing Service-Oriented applications can use agile methods.