by Karunesh Johri
The major activities required in a software development life cycle (SDLC) were identified in the waterfall model. The agile concurrent software process model proposes that the activities identified in the waterfall model are not done sequentially; these activities progress concurrently with varying intensities during the entire software life cycle. Also, a focus on a working software subset as opposed to documents, giving high importance to customer satisfaction and sensitivity to human factors go a long way in contributing towards a software project's overall success.
Software Process Models tell us about the major steps that need to be followed in a software development life cycle (SDLC). Software development life cycle, or the software development process is basically a breakdown of total work that needs to be done into several well defined activities for creating a software product. The concept of software process model is thus a central idea in Software Engineering, which is a discipline involved in study and dissemination of sound software development and management practices.
Software Process Models have evolved during the last forty years.
In the late 1960s,
software crisis was identified as the lack of ability
to develop correct and reliable software for large applications that had become
possible due to advances in hardware technology. Software projects were typically
plagued by cost and time overruns and the performance of software developed was often not reliable.
Software Engineering emerged as an important discipline, similar to production or
operations management in industry. People started putting their experiences and
intuition together to discover the basic principles which, if followed, would make a significant contribution
towards achieving success in creation of software products.
It was understood that software had a
life-cycle starting from the decision to
make that software to the time the software became operational. The term software development
life-cycle was based on a basic software process model, indicating the stages software
would go through in its life-cycle. The first response to the software crisis was the
discovery of the waterfall software process model. It was thought that the root cause of
the problem was that the requirements were not understood well enough before the software design
and proper design was not done prior to coding. Applying the engineering paradigm of freezing
requirements before design and freezing design before manufacture, the waterfall model
stipulated that software should be made in a sequential series of stages, like requirements'
analysis, design, detailed design, coding, integration, installation, operations and
maintenance. Slowly, it became apparent that it was not possible to have water-tight
compartments and it was desirable to have feedback between successive stages. Thus a design
error found during coding would lead to a correction of design and then the coding would
continue. This was in line with the engineering practice for a design error found during
manufacture is first corrected in design and than manufacturing is done as per the modified
design. Also, taking the clue from engineering again, it was felt that making a prototype
of the system in early stages would help in understanding the requirements better and would
eventually help in making a better system.
Figure 1: The Waterfall Software Process model
The problems encountered with the waterfall model are listed in the classic paper from Parnas and Clements . It was felt that software development was too intricate a process to be captured in a sequential waterfall model. Normally requirements are not understood at the beginning (the first stage) and when a project goes on with erroneous requirements, it is doomed to be a failure. Similarly design errors found late in the testing or integration stages would put the project in jeopardy as some requirements would not be achieved. One had to live with these errors for the rest of the life cycle. To overcome these problems, the evolutionary model, was formulated. In this model, software would be developed incrementally, Initially, a subset of requirements would be taken up. These were designed, coded and tested. With time, more and more requirements would be added. This helped in better control as the size of software under development, at any time, was smaller. However, the problem was that if the subsets were not well chosen, the simpler ones got developed earlier and would not work well with the more complex developed later. Or the design for the earlier developed parts was not suitable for the parts developed later.
The spiral model  was a further improvement on the evolutionary model. It
integrates the concepts of project objectives, risk management, prototyping and
project cost to the evolutionary model. In the spiral model the development is
cycles of a spiral. Each cycle starts with determination of objectives,
alternatives and constraints for the cycle. With a good project overview, the
objectives of a cycle can be decided. Having the objectives in place, the next
step in a cycle is to evaluate alternatives and identify and resolve risks. In
order to manage the risks involved in a cycle, prototyping, simulation,
benchmarking, etc. are done. Thus the project team is able to zero-in on the
best methods and tools for development. The next step in the cycle is to develop
as per the objectives and methods decided for the cycle. In an early cycle, the
development might involve development of concept of operation or software
requirements, whereas in a later cycle it might mean development of software
design, detailed design, or code. Development also involves validation of the
developed output. After development, the last part of the cycle is to reflect on
the work done in the cycle and plan for the next cycle. In an early cycle, it
might be requirements or life cycle plan, whereas later on, it might be an
update of earlier plans (based on the experiences of the cycle), development
plan or integration and test plan. Each cycle has an associated cost. The
radial dimension of the spiral is the project cost and the angular dimension is
the project progress for a cycle. The spiral model was a big improvement as it
stressed on evolutionary incremental development, risk management, prototyping,
and project overview and planning.
Figure 2: The Spiral Software Process model
THE EVOLUTION OF SOFTWARE PROCESS MODELS
If we look at the evolution of software process models, certain trends are evident. First, software can not be developed in an ad-hoc code and fix way, we have to be clear about software process model principles so that we take definite steps forward in the right direction. There are clear-cut activities like understanding and statement of requirements, software architecture and design, coding, testing, integration, system testing and installation. However since software is intangible and software products affect the environment in a many ways, timely feedback is necessary for making a correct usable product. Also, it helps the developers to make a scaled-down prototype, see how it works and then make a bigger prototype or the final product. Also, we need to keep in mind the big picture all the time, the project overview, so that we are aware about the relationship of the parts with the whole and make sure that the parts add up to the whole.
The second point is regarding the overhead of following the software process
model. With all the knowledge-base of standards, templates, reports and stages,
this can become a lot of work and might even overshadow the actual software
design and development work. This is similar to making such elaborate arrangements
for a child's birthday celebrations that the child (and his or her happiness) is forgotten in the process.
Another analogy is to a situation where management processes become too complicated
and the production suffers. One of the objectives of engineering disciplines like
industrial engineering has been to simplify processes so that the efficiency is
improved. Closer home, simplicity and efficiency have always been key virtues in
software development. The key question to be asked is:
Is the software process
model being followed too heavy such that it is a hindrance
instead of being a facilitator for the software project?
THE CONCURRENT SOFTWARE PROCESS MODEL
One problem with the conventional software process models like the waterfall model
was that the initial stages like requirements' analysis would usually take a long time.
Since the project time span would have been fixed in advance (with a possible penalty for
delay in completion), there would be resistance to increasing overall project time span.
An attempt would be made to
speed up the coding work. The developers would have anticipated this
well in advance and would have started development and testing of key components while
requirements' study was being done. In a real life project following
the waterfall or evolutionary model, multiple activities would be going on concurrently
at any time. There would be a main activity like requirements' analysis or design or
coding. Then there would be other activities with lesser intensity. There might be a
spurt in a particular activity at a time. Like, if the hardware vendor delivers the new
computer, the developers might spend some focused time on developing a prototype on
the new machine. But it is an inescapable fact that in the waterfall model with a bigger
span or spiral model with a smaller span
of activities per cycle, it is not that a single activity being done at a time; there are
multiple activities with different intensities that are going on at any given time. So,
for example, for a typical software project, the time spent during a month during
requirements' study might be as shown in Figure 3. Although, primarily, the study of
requirements is being done, some design work, coding, testing and integration are also
Figure 3: An example of time spent on various activities during Requirements' study
In a real project, all the major activities progress at different rates all the time. For example, formulation of requirements might be being done at a particular time. But the study of requirements might trigger some design, coding or even integration activity. We might want to see whether a certain requirement is feasible or not. Or whether a desired performance level can be achieved for a particular requirement. Or simply, activities like design, coding, testing take a lot of time. Why wait till the end of requirements phase to start design when it can be started in parallel and we can get a head-start. Thus for example assuming we were in the requirements' phase, when a particular sub-activity of coding done, work done for coding accumulates. In this case, some design, coding, testing and integration work was also getting accomplished although we were in the requirements' phase.
THE AGILE SOFTWARE PROCESS
The basic requirements of the software process as well as the need to develop high quality software in the shortest span of time has made us more pragmatic and open to new ideas. The waterfall model helped to identify the major software process activities. The evolutionary model clarified that complex software systems need to be developed in an incremental way. The spiral model emphasized on the need to practice risk management, develop prototypes and proceed in an incremental way. The concurrent model tells us that all activities are under progress at all times, albeit with different intensities. The agile software process model takes this forward and says,
- at all times during the life cycle, keep in mind the final product which does the job, the working programs,
- as it is humanly not possible to define requirements accurately, the process model should be efficient at handling changes in requirements during the entire life cycle,
- software is developed for customer's requirements; customer satisfaction is most important,
- for the success, the level of motivation of software developers is very important, and
- communication, cooperation, teamwork, and not competition, should be the keywords.
THE IMPORTANCE OF PROGRAMMING
Talk is cheap. Show me the code.
The importance of programming cannot be over-emphasised. Stroustrup has observed,
... the details
of the problem and the concepts of the solution often become clearly understood only through the
effort to express them in a program and trying to get it run acceptably . Programming is the
way to ensure that definition of requirements, design, hardware and system software chosen are all
tuned to the software to be developed. Programming brings out whether there are some requirements
which are obscure and would require special algorithms and tools. If programming is not done, these
difficult and unpleasant problems would be kept aside and these would eventually become a bottleneck
for the project. Programming, likewise, checks the efficacy of design; whether the design is easy
to implement or is it too cumbersome and whether the initial results point to the desired level of
performance. The same thinking has led to development of prototypes at different project stages in
all models, waterfall, evolutionary, spiral and, of course, concurrent.
Does the emphasis on programming mean return to the pre-software engineering days of 1960s, when code and fix was the norm? Definitely not. We, now, understand the importance of requirements, design, programming, testing, integration, system testing very well and there are clear methodologies for these activities. What we are saying is that these activities need to be done in parallel and that coding and testing need to be done almost right through the life cycle, first to clarify requirements, then to achieve the right design and later on keep the system in sync with new requirements.
HANDLING CHANGES IN REQUIREMENTS
Since requirements are not known precisely at the beginning and become clear only as we proceed
on the project, we need to be geared up to accommodate changes in requirements. Each change in
requirements, in fact, takes us closer to the
real requirements. One of the reasons for the
success of Unix has been the usage of the system in parallel with the development efforts and
any error or feedback was promptly corrected or incorporated and system was quickly tried with
modification. Thompson and Ritchie have observed,
... nearly from the start, the system was able
to, and did maintain itself. ... If the designers of a system are forced to use that system, they
quickly become aware of its functional and superficial deficiencies and are strongly
motivated to correct them before it is too late. Because all our source programs were
always available on-line, we were willing to revise and re-write the system and its
software when new ideas were invented, discovered, or suggested by others .
Early correction of errors is very important. It can orient the work in the right
direction and save a lot of effort which would otherwise be required in carrying on
with the errors. The most serious errors are the requirement errors. It is important to
be positive about changes in requirements. For, if we are not positive, then people will
hesitate to point errors in requirements. Consequently, the system development would
proceed with erroneous requirements in the wrong direction.
Here customer means the end-user who would be actually using the system on a
day-to-day basis. The customer has the best knowledge of the system requirements although
they might not be able to tell it to software developers in the beginning. But as the system
development proceeds, their feedback is of paramount importance. Because of their
the end-use environment, they are able to give
key feedback which can make the development
effort successful. Early incorporation of customer feedback leads to further thinking on the
part of customers and then more feedback.
But all this feedback is worth its weight in gold. The agile software process stresses on customer
collaboration and a quick incorporation of feedback leads to a system that is close to user needs.
The feedback's follow a bell shaped curve. Early in the life cycle there is less feedback.
It suddenly picks up when the customer is able to see a tangible system. It reaches a maxima and
then the rate of feedback starts falling down sharply. It is at this point that the system has
come close to
real requirements. By early coding and customer involvement, the agile process
attempts to shift this curve towards west.
Figure 4: Change requests increase, reach a maximum and then decrease
The agile software process stresses on the human factors. Capable and committed people, with
a high level of morale, make invaluable human resources. We need to value quality over quantity.
For, a good quality 100 line code might do the same work as a poor-quality 1000 line code.
A good design could drastically reduce the amount of total work required. For all this to
happen, the motivation of developers has to be high. It is generally felt that people are not
motivated because of high financial compensation but because of good working conditions. This
might mean reasonable hours of work, empowerment, right training, hardware and other resources
and a good organizational setup. Communication between team members, between the team and the
customers and between the management and the team is all very important. An environment needs
to be created where communication is informal and relaxed. It is a myth that
leads to great effort and good results. Rather than thinking of work as a 100 meters athletics
competition, we need to think of it as a mountaineering expedition, where teamwork is most
important, and where people collaborate to do the work together. Ritchie has attributed the
success of Unix to the
calm, communicative and noncompetitive environment at Bell Labs .
The guiding principle of agile software process is that the development teams are technically
strong, cohesive, self-organizing, able set their own goals and meet them. An agile team
periodically sits down and reflects on the work done and thinks about the ways of becoming
more effective. It then goes about fine-tuning its methods so as to improve. A self organizing
team is able to make good software architectures, efficient programs and tune these programs
real user requirements quickly and effectively.
THE EFFECTIVENESS OF AGILE PROCESS
Is the agile software process too idealistic to be successful in real life? To answer this
question, we need to look at the various doubts that have been expressed of the agile process.
The first skepticism was that if the requirements were continually changing, the software life
cycle would go on for ever and would never finish. This had the financial implication that the
project cost and time frame were a part of software contract and the software development
organizations would lose money if the project was not
managed well. The second skepticism was
whether self-organizing software teams would deliver the results.
To answer these questions we need to get back to basics. The extent of success of any economic activity
depends on the quality of the management. The best possible form of management is that of creative
leadership, a sort of statesmanship in the economic world. The key to success is the teamwork
between between the customer organization and the software development organization. Rather than
thinking in terms of
us, it is best to think as
us only. If the project is
successful, both the customer and the developer benefit; if it is not, both lose. A lot of projects
are not a success and are close to a failure; these projects are a problem because of the drain they
cause to the resources of both customer and developer. Some sort of management statesmanship and
initiative at the working level are required to create this teamwork. The
ideal teamwork might
not happen, but we need to understand the ideal and the closer we are to the ideal, the more successful
we are. Once there is teamwork and mutual trust, the requirements would
not keep changing but
real requirements. The bell shaped curve of customer feedback would get past the
maxima and come downwards. The project might take a little longer, but would be better than a work
away from user requirements or a project abandoned because of cost and time overruns. Ultimately,
both the customer and developer organizations would have to rise above small financial gains and
losses and look at the bigger gains of project success. For the customer, a successful project
might mean more operational efficiency and the increasing returns from better efficiency would
compensate for a slight increase in project cost by many times. For the developer organization,
an agile process gives the best results and the project would in all probability an overall success,
even if the costs increase a little. In an environment of mutual trust both the customer and developer
organizations might discuss the increase in cost and come to a mutually beneficial solution.
That brings us to the second question. Do self-organizing teams deliver good products in the given time-frame so that the project is economically successful? There can be only one answer to this, the affirmative. The management challenge lies in creating and nurturing such teams. Of course, any commercial organization would have adequate checks and balances that these teams do not do something wrong. But it is necessary (for success) to create and empower such teams. The teams should have the power to set their goals. This is better than thrusting some goals on the team. If the goals are set by the team themselves, the chances of their meeting the goals is better. Once a team is able to set their goals, they should be empowered to get the resources needed for meeting the goals. When the goals and resources make economic sense, that is, the revenue earned by the team is more than the costs, we need not have any doubts about the economic viability of self-organizing teams.
We tried to trace the development of software process models. From the waterfall model to agile model, the principles of software process models have become clear for practice. If we follow the agile software process practices and the concurrent process model and create self-organizing strong development teams, we would be on the right track for software project success.
1. David Lorge Parnas and Paul C. Clements,
A Rational Design Process: How and Why to fake It, IEEE
Transactions on Software Engineering, Vol. SE-12, No. 2, February 1986.
2. Barry W. Boehm,
A Spiral Model of Software Development and Enhancement, IEEE Computer, May 1988.
3. Roger S. Pressman,
4. Bjarne Stroustrup,
5. D. M. Ritchie and K. Thompson,
The UNIX Time-Sharing System, AT&T Bell System Technical Journal,
Vol 57, No. 6, July-August 1978.
6. Dennis M. Ritchie,
Reflections on Software Research, Turing Award Lecture, Communications of the
ACM, Volume 27, Number 8, August 1984.