What if the application has functionality that wasn't
in the requirements?
It may take serious effort to determine if an
application has significant unexpected or hidden
functionality, and it would indicate deeper problems in
the software development process. If the functionality
isn't necessary to the purpose of the application, it
should be removed, as it may have unknown impacts or
dependencies that were not taken into account by the
designer or the customer. If not removed, design
information will be needed to determine added testing
needs or regression testing needs. Management should be
made aware of any significant added risks as a result of
the unexpected functionality. If the functionality only
effects areas such as minor improvements in the user
interface, for example, it may not be a significant
How can Software QA processes be implemented without
By implementing QA processes slowly over time, using
consensus to reach agreement on processes, and adjusting
and experimenting as an organization grows and matures,
productivity will be improved instead of stifled.
Problem prevention will lessen the need for problem
detection, panics and burn-out will decrease, and there
will be improved focus and less wasted effort. At the
same time, attempts should be made to keep processes
simple and efficient, minimize paperwork, promote
computer-based processes and automated tracking and
reporting, minimize time required in meetings, and
promote training as part of the QA process. However, no
one - especially talented technical types - likes rules
or bureacracy, and in the short run things may slow down
a bit. A typical scenario would be that more days of
planning and development will be needed, but less time
will be required for late-night bug-fixing and calming
of irate customers.
What if an organization is growing so fast that fixed QA
processes are impossible?
This is a common problem in the software industry,
especially in new technology areas. There is no easy
solution in this situation, other than:
• Hire good people
• Management should 'ruthlessly prioritize' quality
issues and maintain focus on the customer
• Everyone in the organization should be clear on what
'quality' means to the customer
How does a client/server environment affect testing?
Client/server applications can be quite complex due to
the multiple dependencies among clients, data
communications, hardware, and servers. Thus testing
requirements can be extensive. When time is limited (as
it usually is) the focus should be on integration and
system testing. Additionally, load/stress/performance
testing may be useful in determining client/server
application limitations and capabilities. There are
commercial tools to assist with such testing. (See the
'Tools' section for web resources with listings that
include these kinds of test tools.)
How can World Wide Web sites be tested?
Web sites are essentially client/server applications -
with web servers and 'browser' clients. Consideration
should be given to the interactions between html pages,
TCP/IP communications, Internet connections, firewalls,
applications that run in web pages (such as applets,
run on the server side (such as cgi scripts, database
interfaces, logging applications, dynamic page
generators, asp, etc.). Additionally, there are a wide
variety of servers and browsers, various versions of
each, small but sometimes significant differences
between them, variations in connection speeds, rapidly
changing technologies, and multiple standards and
protocols. The end result is that testing for web sites
can become a major ongoing effort. Other considerations
• What are the expected loads on the server (e.g.,
number of hits per unit time?), and what kind of
performance is required under such loads (such as web
server response time, database query response times).
What kinds of tools will be needed for performance
testing (such as web load testing tools, other tools
already in house that can be adapted, web robot
downloading tools, etc.)?
• Who is the target audience? What kind of browsers will
they be using? What kind of connection speeds will they
by using? Are they intra- organization (thus with likely
high connection speeds and similar browsers) or
Internet-wide (thus with a wide variety of connection
speeds and browser types)?
• What kind of performance is expected on the client
side (e.g., how fast should pages appear, how fast
should animations, applets, etc. load and run)?
• Will down time for server and content
maintenance/upgrades be allowed? how much?
• What kinds of security (firewalls, encryptions,
passwords, etc.) will be required and what is it
expected to do? How can it be tested?
• How reliable are the site's Internet connections
required to be? And how does that affect backup system
or redundant connection requirements and testing?
• What processes will be required to manage updates to
the web site's content, and what are the requirements
for maintaining, tracking, and controlling page content,
graphics, links, etc.?
• Which HTML specification will be adhered to? How
strictly? What variations will be allowed for targeted
• Will there be any standards or requirements for page
appearance and/or graphics throughout a site or parts of
• How will internal and external links be validated and
updated? how often?
• Can testing be done on the production system, or will
a separate test system be required? How are browser
caching, variations in browser option settings, dial-up
connection variabilities, and real-world internet
'traffic congestion' problems to be accounted for in
• How extensive or customized are the server logging and
reporting requirements; are they considered an integral
part of the system and do they require testing?
components, etc. to be maintained, tracked, controlled,
Some sources of site security information include the
Usenet newsgroup 'comp.security.announce' and links
concerning web site security in the 'Other Resources'
Some usability guidelines to consider - these are
subjective and may or may not apply to a given situation
(Note: more information on usability testing issues can
be found in articles about web site usability in the
'Other Resources' section):
• Pages should be 3-5 screens max unless content is
tightly focused on a single topic. If larger, provide
internal links within the page.
• The page layouts and design elements should be
consistent throughout a site, so that it's clear to the
user that they're still within a site.
• Pages should be as browser-independent as possible, or
pages should be provided or generated based on the
• All pages should have links external to the page;
there should be no dead-end pages.
• The page owner, revision date, and a link to a contact
person or organization should be included on each page.
Many new web site test tools have appeared in the recent
years and more than 280 of them are listed in the 'Web
Test Tools' section.
Page Numbers : 1