What is 'configuration management'?
Configuration management covers the processes used to
control, coordinate, and track: code, requirements,
documentation, problems, change requests, designs,
tools/compilers/libraries/patches, changes made to them,
and who makes the changes. (See the 'Tools' section for
web resources with listings of configuration management
tools. Also see the Bookstore section's 'Configuration
Management' category for useful books with more
What if the software is so buggy it can't really be
tested at all?
The best bet in this situation is for the testers to go
through the process of reporting whatever bugs or
blocking-type problems initially show up, with the focus
being on critical bugs. Since this type of problem can
severely affect schedules, and indicates deeper problems
in the software development process (such as
insufficient unit testing or insufficient integration
testing, poor design, improper build or release
procedures, etc.) managers should be notified, and
provided with some documentation as evidence of the
How can it be known when to stop testing?
This can be difficult to determine. Many modern software
applications are so complex, and run in such an
interdependent environment, that complete testing can
never be done. Common factors in deciding when to stop
• Deadlines (release deadlines, testing deadlines, etc.)
• Test cases completed with certain percentage passed
• Test budget depleted
• Coverage of code/functionality/requirements reaches a
• Bug rate falls below a certain level
• Beta or alpha testing period ends
What if there isn't enough time for thorough testing?
Use risk analysis to determine where testing should be
Since it's rarely possible to test every possible aspect
of an application, every possible combination of events,
every dependency, or everything that could go wrong,
risk analysis is appropriate to most software
development projects. This requires judgement skills,
common sense, and experience. (If warranted, formal
methods are also available.) Considerations can include:
• Which functionality is most important to the project's
• Which functionality is most visible to the user?
• Which functionality has the largest safety impact?
• Which functionality has the largest financial impact
• Which aspects of the application are most important to
• Which aspects of the application can be tested early
in the development cycle?
• Which parts of the code are most complex, and thus
most subject to errors?
• Which parts of the application were developed in rush
or panic mode?
• Which aspects of similar/related previous projects
• Which aspects of similar/related previous projects had
large maintenance expenses?
• Which parts of the requirements and design are unclear
or poorly thought out?
• What do the developers think are the highest-risk
aspects of the application?
• What kinds of problems would cause the worst
• What kinds of problems would cause the most customer
• What kinds of tests could easily cover multiple
• Which tests will have the best high-risk-coverage to
What if the project isn't big enough to justify
Consider the impact of project errors, not the size of
the project. However, if extensive testing is still not
justified, risk analysis is again needed and the same
considerations as described previously in 'What if there
isn't enough time for thorough testing?' apply. The
tester might then do ad hoc testing, or write up a
limited test plan based on the risk analysis.
What can be done if requirements are changing
A common problem and a major headache.
• Work with the project's stakeholders early on to
understand how requirements might change so that
alternate test plans and strategies can be worked out in
advance, if possible.
• It's helpful if the application's initial design
allows for some adaptability so that later changes do
not require redoing the application from scratch.
• If the code is well-commented and well-documented this
makes changes easier for the developers.
• Use rapid prototyping whenever possible to help
customers feel sure of their requirements and minimize
• The project's initial schedule should allow for some
extra time commensurate with the possibility of changes.
• Try to move new requirements to a 'Phase 2' version of
an application, while using the original requirements
for the 'Phase 1' version.
• Negotiate to allow only easily-implemented new
requirements into the project, while moving more
difficult new requirements into future versions of the
• Be sure that customers and management understand the
scheduling impacts, inherent risks, and costs of
significant requirements changes. Then let management or
the customers (not the developers or testers) decide if
the changes are warranted - after all, that's their job.
• Balance the effort put into setting up automated
testing with the expected effort required to re-do them
to deal with changes.
• Try to design some flexibility into automated test
• Focus initial automated testing on application aspects
that are most likely to remain unchanged.
• Devote appropriate effort to risk analysis of changes
to minimize regression testing needs.
• Design some flexibility into test cases (this is not
easily done; the best bet might be to minimize the
detail in the test cases, or set up only higher-level
generic-type test plans)
• Focus less on detailed test plans and test cases and
more on ad hoc testing (with an understanding of the
added risk that this entails).
Page Numbers : 1