0115 966 7955 Today's Opening Times 10:00 - 20:00 (BST)

Factors Affecting Web Applications Maintenance

Disclaimer: This dissertation has been submitted by a student. This is not an example of the work written by our professional dissertation writers. You can view samples of our professional work here.

Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of UK Essays.

Chapter 1

1.1 Introduction

Software engineering [PRE01] is the process associated with industrial quality software development, the methods used to analyze, design & test computer Software, the management techniques associated with the control & monitoring of Software projects & the tools used to support process, methods, & techniques. In Software Development Life Cycle, the focus is on the activities like feasibility study, requirement analysis, design, coding, testing, & maintenance.

Feasibility study involves the issues like technical/economical/ behavioral feasibility of project. Requirement analysis [DAV93] emphasizes on identifying the needs of the system & producing the Software Requirements Specification document (SRS), [JAL04] that describes all data, functional & behavioral requirements, constraints, & validation requirements for Software.

Software Design is to plan a solution of the problem specified by the SRS document, a step in moving from the problem domain to the solution domain. The output of this phase is the design document. Coding is to translate the design of the system into code in a programming language. Testing is the process to detect defects & minimize the risk associated with the residual defects. The activities carried out after the delivery of the software comprises the maintenance phase.

1.2 Evolution of Software Testing Discipline

The effective functioning of modern systems depends on our ability to produce software in a cost-effective way. The term software engineering was first used at a 1968 NATO workshop in West Germany. It focused on the growing software crisis. Thus we see that the software crisis on quality, reliability, high costs etc. started way back when most of today's software testers were not even born.

The attitude towards Software Testing [BEI90] underwent a major positive change in the recent years. In the 1950's when Machine languages were used, testing was nothing but debugging. When in the 1960's, compilers were developed, testing started to be considered a separate activity from debugging.

In the 1970's when the software engineering concepts were introduced, software testing began to evolve as a technical discipline. Over the last two decades there has been an increased focus on better, faster and cost-effective software. Also there has been a growing interest in software safety, protection and security and hence an increased acceptance of testing as a technical discipline and also a career choice.

Now to answer, What is Testing? we can go by the famous definition of Myers [MYE79], which says, Testing is the process of executing a program with the intent of finding errors. According to Humphrey, software testing is defined as, the execution of a program to find its faults. Testing is the process to prove that the software works correctly [PRA06].

Software testing is a crucial aspect of the software life cycle. In some form or the other it is present at each phase of (any) software development or maintenance model. The importance of software testing and its impact on software cannot be underestimated. Software testing is a fundamental component of software quality assurance and represents a review of specification, design and coding. The greater visibility of software systems and the cost associated with software failure are motivating factors for planning, through testing. It is not uncommon for a software organization to spend 40-50% of its effort on testing.

During testing, the software engineering produces a series of test cases that are used to rip apart the software they have produced. Testing is the one step in the software process that can be seen by the developer as destructive instead of constructive. Software engineers are typically constructive people and testing requires them to overcome preconceived concepts of correctness and deal with conflicts when errors are identified.

A successful test is one that finds a defect. This sounds simple enough, but there is much to consider when we want to do software testing. Besides finding faults, we may also be interested in testing performance, safety, fault-tolerance or security. Testing often becomes a question of economics. For projects of a large size, more testing will usually reveal more bugs. The question then becomes when to stop testing, and what is an acceptable level of bugs. This is the question of good enough software.

Testing is the process of verifying that a product meets all requirements. A test is never complete. When testing software the goal should never be a product completely free from defects, because it's impossible. According to Peter Nielsen, The average is 16 faults per 1000 lines of code when the programmer has tested his code and it is believed to be correct. When looking at a larger project, there are millions of lines of code, which makes it impossible to find all present faults. Far too often products are released on the market with poor quality. Errors are often uncovered by users, and in that stage the cost of removing errors is large in amount.

1.3 Objectives of Testing

Glen Myers [MYE79] states a number of rules that can serve well as testing objectives:

  • Testing is a process of executing a program with the intent of finding an error.
  • A good test is one that has a high probability of finding an as yet undiscovered error.
  • A successful test is one that uncovers an as yet undiscovered error.
  • The objective is to design tests that systematically uncover different classes of errors & do so with a minimum amount of time & effort.

Secondary benefits include

  • Demonstrate that Software functions appear to be working according to specification.
  • That performance requirements appear to have been met.
  • Data collected during testing provides a good indication of Software reliability & some indication of Software quality.
  • Testing cannot show the absence of defects, it can only show that Software defects are present.

1.4 Software Testing & Its Relation with Software Life Cycle

Software testing should be thought of as an integral part of the Software process & an activity that must be carried out throughout the life cycle.

Each phase in the Software lifecycle has a clearly different end product such as the Software requirements specification (SRS) documentation, program unit design & program unit code. Each end product can be checked for conformance with a previous phase & against the original requirements. Thus, errors can be detected at each phase of development.

  • Validation & Verification should occur throughout the Software lifecycle.
  • Verification is the process of evaluating each phase end product to ensure consistency with the end product of the previous phase.
  • Validation is the process of testing Software, or a specification, to ensure that it matches user requirements.

Software testing is that part of validation & verification associated with evaluating & analysing program code. It is one of the two most expensive stages within the Software lifecycle, the other being maintenance. Software testing of a product begins after the development of the program units & continues until the product is obsolete.

Testing & fixing can be done at any stage in the life cycle. However, the cost of finding & fixing errors increases dramatically as development progresses.

Changing a Requirements document during the first review is inexpensive. It costs more when requirements change after the code has been written: the code must be rewritten. Bug fixes are much cheaper when programmers find their own errors. Fixing an error before releasing a program is much cheaper than sending new disks, or even a technician to each customer's site to fix it later. It is illustrated in Figure 1.1.

The types of testing required during several phases of Software lifecycle are described below:


Requirements must be reviewed with the client; rapid prototyping can refine requirements & accommodate changing requirements.


The specifications document must be checked for feasibility, traceability, completeness, & absence of contradictions & ambiguities.

Specification reviews (walkthroughs or inspections) are especially effective.


Design reviews are similar to specification reviews, but more technical.

The design must be checked for logic faults, interface faults, lack of exception handling, & non-conformance to specifications.


Code modules are informally tested by the programmer while they are being implemented (desk checking).

Thereafter, formal testing of modules is done methodically by a testing team. This formal testing can include non-execution-based methods (code inspections & walkthroughs) & execution-based methods (black-box testing, white-box testing).


Integration testing is performed to ensure that the modules combine together correctly to achieve a product that meets its specifications. Particular care must be given to the interfaces between modules.

The appropriate order of combination must be determined as top-down, bottom-up, or a combination thereof.

Product Testing

The functionality of the product as a whole is checked against its specifications. Test cases are derived directly from the specifications document. The product is also tested for robustness (error-handling capabilities & stress tests).

All source code & documentation are checked for completeness & consistency.

Acceptance Testing

The Software is delivered to the client, who tests the Software on the actual h/w, using actual data instead of test data. A product cannot be considered to satisfy its specifications until it has passed an acceptance test.

Commercial off-the-shelf (or shrink-wrapped) Software usually undergoes alpha & beta testing as a form of acceptance test.


Modified versions of the original product must be tested to ensure that changes have been correctly implemented.

Also, the product must be tested against previous test cases to ensure that no inadvertent changes have been introduced. This latter consideration is termed regression testing.

Software Process Management

The Software process management plan must undergo scrutiny. It is especially important that cost & duration estimates be checked thoroughly.

If left unchecked, errors can propagate through the development lifecycle & amplify in number & cost. The cost of detecting & fixing an error is well documented & is known to be more costly as the system develops. An error found during the operation phase is the most costly to fix.

1.5 Principles of Software Testing

Software testing is an extremely creative & intellectually challenging task. The following are some important principles [DAV95] that should be kept in mind while carrying Software testing [PRE01] [SUM02]:

Testing should be based on user requirements: This is in order to uncover any defects that might cause the program or system to fail to meet the client's requirements.

Testing time & resources are limited: Avoid redundant tests.

It is impossible to test everything: Exhaustive tests of all possible scenarios are impossible, because of the many different variables affecting the system & the number of paths a program flow might take.

Use effective resources to test: This represents use of the most suitable tools, procedures & individuals to conduct the tests. Only those tools should be used by the test team that they are confident & familiar with. Testing procedures should be clearly defined. Testing personnel may be a technical group of people independent of the developers.

Test planning should be done early: This is because test planning can begin independently of coding & as soon as the client requirements are set.

Test for invalid & unexpected input conditions as well as valid conditions: The program should generate correct messages when an invalid test is encountered & should generate correct results when the test is valid.

The probability of the existence of more errors in a module or group of modules is directly proportional to the number of errors already found.

Testing should begin at the module: The focus of testing should be concentrated on the smallest programming units first & then expand to other parts of the system.

Testing must be done by an independent party: Testing should not be performed by the person or team that developed the Software since they tend to defend the correctness of the program.

Assign best personnel to the task: Because testing requires high creativity & responsibility only the best personnel must be assigned to design, implement, & analyze test cases, test data & test results.

Testing should not be planned under the implicit assumption that no errors will be found.

Testing is the process of executing Software with the intention of finding errors.

Keep Software static during test: The program must not be modified during the implementation of the set of designed test cases.

Document test cases & test results.

Provide expected test results if possible: A necessary part of test documentation is the specification of expected results, even though it is impractical.

1.6 Software Testability & Its Characteristics

Testability is the ability of Software (or program) with which it can easily be tested [PRE01] [SUM02]. The following are some key characteristics of testability:

  • The better it works, the more efficient is testing process.
  • What you see is what you test (WYSIWYT).
  • The better it is controlled, the more we can automate or optimize the testing process.
  • By controlling the scope of testing we can isolate problems & perform smarter retesting.
  • The less there is to test, the more quickly we can test it.
  • The fewer the changes, the fewer the disruptions to testing.
  • The more information we have, the smarter we will test.

1.7 Stages in Software Testing Process

Except for small programs, systems should not be tested as a single unit. Large systems are built out of sub-systems, which are built out of modules that are composed of procedures & functions. The testing process should therefore proceed in stages where testing is carried out incrementally in conjunction with system implementation.

The most widely used testing process consists of five stages that are illustrated in Table 1.1.

Errors in program components, say may come to light at a later stage of the testing process. The process is therefore an iterative one with information being fed back from later stages to earlier parts of the process. The iterative testing process is illustrated in Figure 1.2 and described below:

Unit Testing: Unit testing is code-oriented testing. Individual components are tested to ensure that they operate correctly. Each component is tested independently, without other system components.

Module Testing: A module is a collection of dependent components such as an object class, an abstract data type or some looser collection of procedures & functions. A module encapsulates related components so it can be tested without other system modules.

Sub-system (Integration) Testing: This phase involves testing collections of modules, which have been integrated into sub-systems. It is a design-oriented testing & is also known as integration testing.

Sub-systems may be independently designed & implemented. The most common problems, which arise in large Software systems, are sub-systems interface mismatches. The sub-system test process should therefore concentrate on the detection of interface errors by rigorously exercising these interfaces.

System Testing: The sub-systems are integrated to make up the entire system. The testing process is concerned with finding errors that result from unanticipated interactions between sub-systems & system components. It is also concerned with validating that the system meets its functional & non-functional requirements.

Acceptance Testing: This is the final stage in the testing process before the system is accepted for operational use. The system is tested with data supplied by the system client rather than simulated test data. Acceptance testing may reveal errors & omissions in the systems requirements definition (user-oriented) because real data exercises the system in different ways from the test data.

Acceptance testing may also reveal requirement problems where the system facilities do not really meet the user's needs (functional) or the system performance (non-functional) is unacceptable.

1.8 The V-model of Testing

To test an entire software system, tests on different levels are performed. The V model [FEW99], shown in figure 1.3, illustrates the hierarchy of tests usually performed in software development projects. The left part of the V represents the documentation of an application, which are the Requirement specification, the Functional specification, System design, the Unit design.

Code is written to fulfill the requirements in these specifications, as illustrated in the bottom of the V. The right part of the V represents the test activities that are performed during development to ensure that an application corresponding to its requirements.

Unit tests are used to test that all functions and methods in a module are working as intended. When the modules have been tested, they are combined and integration tests are used to test that they work together as a group. The unit- and integration test complement the system test. System testing is done on a complete system to validate that it corresponds to the system specification. A system test includes checking if all functional and all non-functional requirements have been met.

Unit, integration and system tests are developer focused, while acceptance tests are customer focused. Acceptance testing checks that the system contains the functionality requested by the customer, in the Requirement specification. Customers are usually responsible for the acceptance tests since they are the only persons qualified to make the judgment of approval. The purpose of the acceptance tests is that after they are preformed, the customer knows which parts of the Requirement specification the system satisfies.

1.9 The Testing Techniques

To perform these types of testing, there are three widely used testing techniques. The above said testing types are performed based on the following testing techniques:

Black-Box testing technique

Black box testing (Figure 1.4) is concerned only with testing the specification. It cannot guarantee that the complete specification has been implemented. Thus black box testing is testing against the specification and will discover faultsofomission, indicating that part of the specification has not been fulfilled. It is used for testing based solely on analysis of requirements (specification, user documentation).

In Black box testing, test cases are designed using only the functional specification of the software i.e without any knowledge of the internal structure of the software. For this reason, black-box testing is also known as functional testing. Black box tests are performed to assess how well a program meets its requirements, looking for missing or incorrect functionality. Functional testing typically exercise code with valid or nearly valid input for which the expected output is known. This includes concepts such as 'boundary values'.

Performance tests evaluate response time, memory usage, throughput, device utilization, and execution time. Stress tests push the system to or beyond its specified limits to evaluate its robustness and error handling capabilities. Reliability tests monitor system response to represent user input, counting failures over time to measure or certify reliability.

Black box Testing refers to analyzing a running program by probing it with various inputs. This kind of testing requires only a running program and does not make use of source code testing of any kind. In the security paradigm, malicious input can be supplied to the program in an effort to cause it to break. If the program breaks during a particular test, then a security problem may have been discovered.

Black box testing is possible even without access to binary code. That is, a program can be tested remotely over a network. All that is required is a program running somewhere that is accepting input. If the tester can supply input that the program consumes (and can observe the effect of the test), then black box testing is possible. This is one reason that real attackers often resort to black box techniques. Black box testing is not an alternative to white box techniques. It is a complementary approach that is likely to uncover a different type of errors that the white box approaches.

Black box testing tries to find errors in the following categories:

  • Incorrect or missing functions
  • Interface errors
  • Errors in data structures or external database access
  • Performance errors, and
  • Initialization and termination errors.

By applying black box approaches we produce a set of test cases that fulfill requirements:

  • Test cases that reduce the number of test cases to achieve reasonable testing
  • Test cases that tell us something about the presence or absence of classes of errors.

The methodologies used for black box testing have been discussed below: Equivalent Partitioning

Equivalence partitioning is a black box testing approach that splits the input domain of a program into classes of data from which test cases can be produced. An ideal test case uncovers a class of errors that may otherwise before the error is detected. Equivalence partitioning tries to outline a test case that identifies classes of errors.

Test case design for equivalent partitioning is founded on an evaluation of equivalence classes for an input condition [BEI95]. An equivalence class depicts a set of valid or invalid states for the input condition. Equivalence classes can be defined based on the following [PRE01]:

  • If an input condition specifies a range, one valid and two invalid equivalence classes are defined.
  • If an input condition needs a specific value, one valid and two invalid equivalence classes are defined.
  • If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined.
  • If an input condition is Boolean, one valid and invalid class is outlined. Boundary Value Analysis

A great many errors happen at the boundaries of the input domain and for this reason boundary value analysis was developed. Boundary value analysis is test case design approach that complements equivalence partitioning. BVA produces test cases from the output domain also [MYE79].

Guidelines for BVA are close to those for equivalence partitioning [PRE01]:

  • If an input condition specifies a range bounded by values a and b, test cases should be produced with values a and b, just above and just below a and b, respectively.
  • If an input condition specifies various values, test cases should be produced to exercise the minimum and maximum numbers.
  • Apply guidelines above to output conditions.
  • If internal program data structures have prescribed boundaries, produce test cases to exercise that data structure at its boundary.

White-Box testing technique

White box testing (Figure 1.5) is testing against the implementation as it is based on analysis of internal logic (design, code etc.) and will discover faultsofcommission, indicating that part of the implementation is faulty. Designing white-box test cases requires thorough knowledge of the internal structure of software, and therefore the white-box testing is also called the structural testing. White box testing is performed to reveal problems with the internal structure of a program.

A common goal of white-box testing is to ensure a test case exercises every path through a program. A fundamental strength that all white box testing strategies share is that the entire software implementation is taken into account during testing, which facilitates error detection even when the software specification is vague or incomplete. The effectiveness or thoroughness of white-box testing is commonly expressed in terms of test or code coverage metrics, which measure the fraction of code exercised by test cases.

White box Testing involves analyzing and understanding source code. Sometimes only binary code is available, but if you decompile a binary to get source code and then study the code, this can be considered a kind of white box testing as well. White box testing is typically very effective in finding programming errors and implementation errors in software. In some cases this activity amounts to pattern matching and can even be automated with a static analyzer.

White box testing is a test case design approach that employs the control architecture of the procedural design to produce test cases. Using white box testing approaches, the software engineering can produce test cases that:

  • Guarantee that all independent paths in a module have been exercised at least once
  • Exercise all logical decisions
  • Execute all loops at their boundaries and in their operational bounds
  • Exercise internal data structures to maintain their validity.

There are several methodologies used for white box testing. We discuss some important ones below. Statement Coverage

The statement coverage methodology aims to design test cases so as to force the executions of every statement in a program at least once. The principal idea governing the statement coverage methodology is that unless a statement is executed, we have way of determining if an error existed in that statement. In other words, the statement coverage criterion [RAP85] is based on the observation that an error existing in one part of a program cannot be discovered if the part of the program containing the error and generating the failure is not executed. However, executed a statement once and that too for just one input value and observing that it behaves properly for that input value is no guarantee that it will behave correctly for all inputs. Branch Coverage

In branch coverage testing, test cases are designed such that the different branch conditions are given true and false values in turn. It is obvious that branch testing guarantees statement coverage and thus is a stronger testing criterion than the statement coverage testing [RAP85]. Path Coverage

The path coverage based testing strategy requires designing test cases such that all linearly independents paths in the program are executed at least once. A linearly independent path is defined in terms of the control flow graph (CFG) of the program. Loop testing

Loops are very important constructs for generally all the algorithms. Loop testing is a white box testing technique. It focuses exclusively on the validity of loop constructs. Simple loop, concatenated loop, nested loop, and unstructured loop are four different types of loops [BEI90] as shown in figure 1.6.

Simple Loop: The following set of tests should be applied to simple loop where n is the maximum number of allowable passes thru the loop:

  • Skip the loop entirely.
  • Only one pass thru the loop.
  • Two passes thru the loop.
  • M passes thru the loop where m < n.
  • N-1, n, n+1 passes thru the loop.
  • Nested Loop: Beizer [BEI90] approach to the nested loop
  • Start at the innermost loop. Set all other loops to minimum value.
  • Conduct the simple loop test for the innermost loop while holding the outer loops at their minimum iteration parameter value.
  • Work outward, conducting tests for next loop, but keeping all other outer loops at minimum values and other nested loops to typical values.
  • Continue until all loops have been tested.

Concatenated loops: These can be tested using the approach of simple loops if each loop is independent of other. However, if the loop counter of loop 1 is used as the initial value for loop 2 then approach of nested loop is to be used.

Unstructured loop: This class of loops should be redesigned to reflect the use of the structured programming constructs. McCabe's Cyclomatic Complexity

The McCabe's Cyclomatic Complexity [MCC76] of a program defines the number of independent paths in a program. Given a control flow Graph G of a program, the McCabe's Cyclomatic Complexity V(G) can be computed as:


Where E is the number of edges in the control flow graph and N is the number of nodes of the control flow graph.

The cyclomatic complexity value of a program defines the number of independent paths in the basis set of the program and provides a lower bound for the number of test cases that must be conducted to ensure that all statements have been executed at least once. Knowing the number of test cases required does not make it easy to derive the test cases, it only gives an indication of the minimum number of test cases required.

The following is the sequences of steps that need to be undertaken for deriving the path coverage based test case of a program.

  • Draw the CFG.
  • Calculate Cyclomatic Complexity V(G).
  • Calculate the basis set of linearly independent paths.
  • Prepare a test case that will force execution of each path in the basis set. Data Flow based Testing

The data flow testing method chooses test paths of a program based on the locations of definitions and uses of variables in the program. Various data flow testing approaches have been examined [FRA88] [NTA88] [FRA93]. For data flow testing each statement in program is allocated a unique statement number and that each function does not alter its parameters or global variables. For a statement with S as its statement number,

DEF(S) = {X| statement S contains a definition of X}

USE(S) = {X| statement S contains a use of X}

If statement S is if or loop statement, its DEF set is left empty and its USE set is founded on the condition of statement S. The definition of a variable X at statement S is live at statement S, if there exists a path from statement S to S' which does not contain any condition of X.

A definition-use chain (or DU chain) of variable X is of the type [X,S,S'] where S and S' are statement numbers, X is in DEF(S), USE(S'), and the definition of X in statement S is live at statement S'.

One basic data flow testing strategy is that each DU chain be covered at least once. Data flow testing strategies are helpful for choosing test paths of a program including nested if and loop statements

1.9.3 Grey-Box testing technique

Grey box testing [BIN99] designs test cases using both responsibility-based (black box) and implementation-based (white box) approaches. To completely test a web application one needs to combine the two approaches, White-box and Black-box testing. It is used for testing of Web based applications. The Gray-box testing approach takes into account all components making up a complete web application, including the environment on witch it resides.

Nguyen Hung Q [NGU01] states that Gray-box testing is integral to web application testing because of the numerous components, both software and hardware, that make up the complete application. The factors considered in Gray-box testing are high-level design, environment and interoperability conditions.

1.10 Some more testing types

From the V-model (Figure 1.3) [FEW99], we see that there are various levels or phases of testing, namely, Unit testing, Integration testing, System testing, User Acceptance testing etc. A brief definition on some more widely employed types of testing is given below:

Regression Testing: Selective re-testing of a system to verify the modification (bug fixes) have not caused unintended effects and that system still complies with its specified requirements

Performance Testing: To evaluate the time taken or response time of the system to perform it's required functions in comparison.

Stress Testing: To evaluate a system beyond the limits of the specified requirements or system resources (such as disk space, memory, processor utilization) to ensure the system do not break unexpectedly.

Load Testing: Load Testing, a subset of stress testing, verifies that a web site can handle a particular number of concurrent users while maintaining acceptable response times.

Alpha Testing: Testing of a software product or system conducted at the developer's site by the customer.

Beta Testing: Testing conducted at one or more customer sites by the end user of a delivered software product system.

Chapter 2 Literature Survey & Objectives of Research

2.1 Literature Survey

In order to carry out the research in the domain of software testing, an exhaustive survey of the literature was carried out resulting in the elicitation and acquisition of the knowledge. The crux of various research papers related to the field of software testing and web testing in chronological order is discussed in the following section:

Kung David C. et al [KUN00] in the paper An Object-Oriented Web Test Model for Testing Web Applications presented a methodology that used an Object-Oriented Web Test Model (WTM) to support Web application testing. The test model captured both structural and behavioral test artifacts of Web applications and represented the artifacts form the object, behavior, and structure perspectives. Based on the test model, both structural and behavioral test cases were derived automatically to ensure the quality of Web applications.

From the object perspective, entities of a Web application were represented using Object Relation Diagram (ORD) in terms of objects and inter-dependent relationships. From the behaviour perspective, a Page Navigation Diagram (PND) was used to depict the navigation behaviour of a Web application. At the same time, a set of Object State Diagrams (OSDs) were used to describe the state behaviour of interacting objects.

From the structure perspective, the Block Branch Diagram (BBD) and Function Cluster Diagram (FCD) were employed to capture the control flow and data flow information of scripts and functions in a Web application. Moreover, special testing concerns were expressed in terms of textural test constraints that could be imposed on the corresponding objects. The model could also be used as a road map to identify change ripple effects and to find cost-effective testing strategies for reducing test efforts required in the regression testing.

In the study Thinking Aloud as a Method for Testing the Usability of Websites: The Influence of Task Variation on the Evaluation of Hypertext, Waes Luuk Van [WAE00] focused on three variants of task types for the thinking-aloud method. In the study, the participants were asked to think aloud while looking for information on a website from different perspectives.

The author analyzed the results of three usability studies in which different thinking-aloud tasks were used: a simple searching task, an application task, and a prediction task. In the task setting the profile of the Web-surfer, the communication goal of the website and other quality aspects were taken into account. The qualitative analysis of these studies showed that the task variation had some influence on the results of usability testing and that, consequently, tasks should be matched with the evaluation goals put forward.

The problem of analyzing frequently traversed spots in a Web site has been explored by Lin Yuh-Chi and Paul Hadingham [LIN00] in the paper Tracking Frequent Traversal Areas in a Web Site via Log Analysis. Firstly, a support knowledge base from Web logs was constructed and the problem of tracking frequent traversal areas as a problem of finding strongly connected components in a digraph was formulated. The tracking features into their test Web site were successfully incorporated. This approach could be useful in the innovative applications such as Web promotion, banner ads, and Web content design in the electronic commerce.

Data flow testing techniques to Web applications have been extended by Liu Chien-Hung et al [LIU00a] in the paper Structural Testing of Web Applications. Several dataflow issues for analyzing HTML and extensible Markup Language (XML) documents in Web applications were discussed. An approach to supporting dataflow testing for Web applications was presented by the authors. In this approach, the structural test artifacts of a Web application were captured in a Web Application Test Model (WATM) where each component of the Web application was treated as an object.

An Object Relation Diagram was employed to depict objects and their relationships. The elements of an HTML or XML document were considered as structured variables within an object. For the variables of interest, data flow test cases were derived in five levels from corresponding flow graphs based on the scope of data flow interactions within an object, across objects, or across clients. An example was used to demonstrate each level of the testing approach based on the types of def-use chain for the variables of interest.

Statistical testing and reliability analysis can be used effectively to assure quality for Web applications. This was stated by Kallepalli Chaitanya and Tian Jeff [KAL01] in their paper Measuring and Modeling Usage and Reliability for Statistical Web Testing. To support this strategy, Web usage and failure information from existing Web logs was extracted. The usage information was used to build models for statistical Web testing.

The related failure information was used to measure the reliability of Web applications and the potential effectiveness of statistical Web testing. This approach was applied to analyze some actual Web logs. The results demonstrated the viability and effectiveness of this approach. Various tools and utility programs previously developed by the authors for general test management and reliability analysis were also used to support the reliability analysis for Web applications. This combination provided a comprehensive yet inexpensive alternative to the automated support for this approach.

A proposal in the field of structural testing of Web applications was suggested by Ricca and Tonella [RIC01], who proposed a first approach for white box testing of primarily static Web applications. This approach was based on a test model named the navigational model that focuses on HTML pages and navigational links of the application. Later, the same authors presented an additional lower layer model, the control flow model, representing the internal structure of Web pages in terms of the execution flow followed [RIC04]. This latter model has also been used to carry out structural testing.

In the navigational model two types of HTML pages are represented: static pages, whose content is immutable, and dynamic pages, whose content is established at run time by server computation, on the basis of user input and server status. Server programs (such as scripts or other executable objects) running on the server side of the application, and other page components that are relevant for navigational purposes, such as forms and frames, are also part of the model. Hyperlinks between HTML pages and various types of link between pages and other model components are included in this code representation.

As for the control flow model, it takes into account the heterogeneous nature of statements written in different coding languages, and the different mechanisms used to transfer control between statements in a Web application. It is represented by a directed graph whose nodes correspond to statements that are executed either by the Web server or by the Internet browser on the client side, and whose edges represent control transfer.

Different types of nodes are shown in this model, according to the programming language of the respective statements. A test case for a Web application is defined as a sequence of pages to be visited, plus the input values to be provided to pages containing forms. Various coverage criteria applicable to both models have been proposed to design test cases: they include path coverage (requiring that all paths in the Web application model are traversed in some test case), branch coverage (requiring that all branches in the model are traversed in some test case), and node coverage (requiring that all nodes in the model are traversed in some test case).

McGough Jeff et al [MCG01] described one solution to the problem of presenting students with dynamically generated browser based exams with significant engineering mathematics content in the paper A Web-Based Testing System with Dynamic Question Generation. This proof of concept was called Web Testing Markup Language (WTML), an extension of HTML.

The web server was extended to process the WTML and support features necessary for mathematics exams. The result was then supplied to the client (student's browser). The student completed an exam and returned it for grading and database recording. The system was used in a calculus class and then assessed. This was a complete success in terms of proof of concept. A web based testing system was constructed, which supported the needs of mathematics intensive courses.

Scholtz Jean [SCH01] in his paper Adaptation of Traditional Usability Testing Methods for Remote Testing discussed the approach to design the usability testing tools. Several tools available for public use such as WebCAT, WebVIP and VISVIP were also discussed in the paper. Remote testing could be used to assess a current site or it could be used when a new site or a new version of a site was being developed.

A link from the current site to a remote evaluation test might be setup to solicit input from current users. This was valuable in two ways. First, it provided information about the usability of the new site. Second, it helped prepare users for a change. It was difficult to drastically change a heavily used web site as users were often frustrated when navigation paths and shortcuts they finally figured out were changed. For web sites that had a global user population, remote assessment and analysis methods were necessary to continually evaluate global usability.

Some open issues in the field of web application testing were discussed by Lucca Giuseppe Antonio Di et al [LUC02] in the paper Testing Web Applications. These issues included the problem of defining a test model, for representing the components of the application to be tested and their inter-connections; a test strategy to create test cases from the test model; various testing levels, specifying the different scopes of the tests to be run; and a testing tool supporting the test engineer during test case design and execution.

An object-oriented model of a web application was exploited as a test model, and a definition of the unit level for testing the web application was proposed. Based on this model, a method to test the single units of a web application and for the integration testing was proposed. In order to experiment the proposed technique and strategy, an integrated platform of tools comprising a web application analyzer, a repository, a test case generator and a test case executor, had been developed. The proposed testing approach and the supporting tool had been submitted to a preliminary experimentation, carried out for validating them. The obtained results showed the feasibility of the testing approach, and the effectiveness of the tool in automating many activities required by the testing process.

Kirchner Michele [KIR02] dealt with the concept of accessibility of Web sites to people with disabilities. Major problems arose when there was the need of special browsers, which could understand the standard HTML, but not the extensions introduced by the most used browsers. The WAI (Web Accessibility Initiative) had pursued some guidelines in order to build accessible sites and the new W3C standards were converging towards those guidelines. It was shown that several tools existed that were able to check the Web pages for the accessibility for impaired people, and some tools were able to repair non accessible pages.

A strategy was proposed by integrating existing testing techniques and reliability analyses in a hierarchical framework by Tian Jeff et al [TIA03]. This strategy combined various usage models for statistical testing to perform high level testing and to guide selective testing of critical and frequently used subparts or components using traditional coverage-based structural testing. Reliability analysis and risk identification were an integral part of this strategy to help assure and improve the overall reliability for web-based applications. Some preliminary results were included to demonstrate the general viability and effectiveness of the approach.

This three-tiered hierarchical approach offered an integrated framework that provided both a potentially effective testing strategy and a vehicle for reliability assessment and improvement. The user focus of web-based applications were supported in this strategy by testing functions, usage scenarios, and navigation patterns important to and frequently used by end users under our top-tier usage model based on the list-like operational profiles as well as our middle-tier usage models based on Markov chains. Internal components and structures for web based applications could also be thoroughly exercised by using these bottom-tier models based on traditional coverage based testing, under the guidance of the upper-level usage models.

An application of agent technology in the testing of web-based applications was presented by Huo Qingning et al [HUO03] in A Multi-Agent Software Environment for Testing Web-based Applications. A prototype was described and discussed whose multi-agent architecture and the infrastructure of the system satisfied the requirements of testing web-based applications. The dynamic nature of web information systems demanded constant monitoring the changes of the system and its environment. Sometimes, the change in the system and its environment might require changes in testing strategy and method accordingly.

Agents were adaptive, and they could adjust their behaviours based on environment changes. These changes could be integrated to the system lively. The diversity of platforms and the formats of media and data of web information systems demand using a wide variety of test methods and tools. Multi-agent systems could provide a promising solution to this problem. Different agents were built to handle different types of information, to implement different testing methods and to integrate different tools. Thus, each individual agent could be relatively simple while the whole system was powerful and extendible.

The distribution of large volume of information and functionality over a large geographic area required testing tasks carried out at different geographic locations and to transfer information between computer systems. The agents could be distributed among many computers to reduce the communication traffic. Although a single agent could only perform as a normal program, the strength of agents came from the intelligent dynamic task assignment, the dynamic control of agent pool, the dynamic interactions between agents, the live update of agent systems.

Xu Lei et al [XU03] put the slicing method into practice to simplify the problems in regression testing of web applications, i.e., only analysed the related contents and did not consider the irrelated ones, in order to carry through the regression testing quickly and effectively. So focus was on the simplified contents and to improve the working efficiency. In order to make sure of what to test, the possible changes and their influences in Web applications were analysed, and the dependent relationships were divided into direct and indirect.

On the base of the dependent relationships, it was emphasized on how to test the indirect-dependent among data, and the regression testing method for Web applications based on slicing was introduced, which obtained the dependant sets of Web changes based on the Forward and Backward Search Method and generated the testing suits. The quality and efficiency of the regression testing can be greatly improved since the testing suits were tightly related to the changes and there was no lack or redundancies in the testing suits.

Some useful statistic data in the log file on server side depending on the users' responses and attitudes was used by Lei Xu et al [XU03a] in Website Evolution Based on Statistic Data. Then, based on the user-visiting information, maintenance and evolution was carried out more efficiently. The authors firstly proposed a simplified Website model, which was a weighted structure model; then gained the critical pages, predominant pages and user-visiting patterns based on this model.

These concepts were then discussed for using them in the evolution for Websites in detail. A usability improving method which aims at critical pages was proposed, then interaction function testing method which focused on predominant pages was introduced; and finally the group testing method which takes place after dividing the module based on user-visiting pattern was produced. The authors focused on how to obtain a valid and evolutive Website in the maintenance phase, and gave some practicable testing methods. Since this work was performed in the maintenance phase, it considered recursive testing also, which was different from the testing occurred in before phases.

Arnold Tom [ARN03] in the paper Testing Web Services (.NET and Otherwise) presented an overview about a web service, gave an example of how web services were being put to use, and determined how to approach testing a web service manually through a web page and with automation using different programming languages.

In cases where the source code wasn't available to the test engineer, working with a web service was very much a black box approach to testing, even though using a programming language to test a web service's exposed interfaces into its functionality seems to indicate otherwise. It all boiled down to sending and receiving HTTP traffic, and the data that was sent depended on what format the web service expected. The required format could be as simple as a query string sent from a browser's Address line, or, an XML document wrapped in a SOAP envelope, all embedded within an HTTP header. The return values also depended on how the web service was written, anything could be returned: XML, HTML, or a single byte of information indicating the response.

Building a web page to provide a user interface for testing a web service was a good way to verify the service is behaving as expected. However, getting into the guts of testing a web service through the use of a programming language like Perl, or an automation tool like Application Center Test, that provided specific support for sending and receiving HTTP data, these programming languages allowed a test engineer to take more of a data-driven approach to testing. Perl, Java, Python, C, and even VB Script can all be used for testing. What the test engineer needed to focus on was what the web service expected as input and what it defined as its output.

The quality issue of the web systems was investigated and a systematic framework for web testing was proposed by Chang Wen-Kui et al [CHA03], in which a web application model was established to denote all components of a website by the Markov usage chain. To demonstrate its usability, a practical example of link validity testing was presented. The suggested validation mechanism not only generated automatically a complete set of the test scripts for link validation, but also provided quantitative analysis of the invalid components in a web application.

In comparison with other existing tools, the proposed systematic framework was shown to be effective for quickly certifying link validity in websites applications. It had many benefits such as: helping in forming a testing plan and allocating testing resources, generating test scripts systematically, and reaching maximal testing coverage. Hence, the proposed testing mechanism was more practical and efficient than those web-testing technologies.

An approach to complement Web Application (WA) testing strategies, such as black box, white box or even strategies exploiting session data, with a state-based testing to verify the correct interaction of the WA with a web browser was proposed by Lucca Giuseppe A. Di et al [LUC03] in the paper Considering Browser Interaction in Web Application Testing. The aim was to discover failures due to particular sequences of navigations performed using the web browser buttons.

A testing model, considering the role of the browser while navigating a web application, and some coverage criteria, were presented in the paper. The test case generation could be automated once baseline test cases, produced with other traditional approaches, were available. The test execution could also be automatically performed developing a tool that simulated the actions of a web browser, with the ability of going forward, backward and reloading a page. Finally, heuristics could be adopted to automatically reveal some inconsistencies caused by following particular sequences of actions to navigate among pages.

Elbaum Sebastian et al [ELB03] in the paper Improving Web Application Testing with User Session Data explored the notion that user session data gathered as users operate web applications can be successfully employed in the testing of those applications, particularly as those applications evolved and experienced different usage profiles.

Results of an experiment comparing new and existing test generation techniques for web applications were reported, assessing both the adequacy of the generated tests and their ability to detect faults on a point-of-sale web application. Results showed that user session data could produce test suites as effective overall as those produced by existing white-box techniques, but at less expense. Moreover, the classes of faults detected differ somewhat across approaches, suggesting that the techniques might be complimentary.

Sampath et al [SAM04] explored the possibility of using concept analysis to achieve scalability in user-session based testing of Web applications. Concept analysis was a technique for clustering objects that have common discrete attributes. It was used in to reduce a set of user sessions to a minimal test suite, which still represented actual executed user behaviour. In particular, a user session was considered as a sequence of URLs requested by the user, and represented a separate use case offered by the application. Starting from an original test suite including a number of user sessions, this test suite was reduced by finding the smallest set of user sessions that covers all the URLs of the original test suite.

At the same time, it represented the common URL of the different use cases represented by the original test suite. This technique enabled an incremental approach that updating the test suite on-the-fly, by incrementally analysing additional user sessions. The experiments carried out showed the actual test suite reduction is achievable by the approach, while preserving the coverage obtained by the original user sessions' suite, and with a minimal loss of fault detection. A framework was thus developed that automated the entire testing process, from gathering user sessions through the identification of a reduced test suite to the reuse of that test suite for coverage analysis and fault detection.

Zhu Hong [ZHU04] discussed why testing web applications and quality assurance was difficult and proposed a cooperative agent approach to overcome the problems. The application of Lehman's theory of software evolution to web-based applications showed that they are by nature evolutionary and, hence, satisfied Lehman's laws of evolution. The essence of web applications implied that supporting their sustainable long term evolution should play the central role in developing quality assurance and testing techniques and tools. Therefore, two basic requirements of such a software environment were identified.

First, the environment should facilitate flexible integrations of tools for developing, maintaining and testing various kinds of software arte-facts in a variety of formats over a long period of evolution. Second, it should enable effective communications between human beings and the environment so that the knowledge about the system and its evolution process could be recorded, retrieved and effectively used for future modification of the system.

The solution to meet these requirements was a cooperative multi-agent software growth environment. In the environment, various tools were implemented as cooperative agents interacting with each other and with human users at a high level of abstraction using ontology. A prototype system for testing web applications designed and implemented whose preliminary experiments with the prototype showed some promising results.

A framework for Web application testing was established by Xu Lei et al [XU04]. Based on the analysis for Web application characters and traditional software testing process, the process for Web application testing was modeled, which described a series of testing flows such as the testing requirement analysis, test cases generation and selection, testing execution, and testing results analysis and measurement. The realization techniques were also investigated so as to integrate each testing step and implemented the whole testing process harmoniously and effectively. Thus the framework considered the specialties of the Web testing fully and it guided each task during the testing process, which was focused on the unique characters of Web applications.

An agent-based framework based on the Belief-Desire-Intention (BDI) model of rational agents and the Unified Modeling Language (UML) for testing of web applications was proposed by David Kung [KUN04]. It was described how Web applications testing could be modeled and reasoned using the framework. An agent-oriented framework for testing Web applications was sketched. The framework was an application of a general agent-oriented software engineering methodology.

The general agent-oriented software engineering framework had been applied to modeling and design of an intelligent elevator system and a garden care system that consisted of several garden care agents and the easiest action plan using the classical AI approach (theorem-proving approach) was successfully generated.

An experimental framework for testing a huge variety of web architecture related aspects in order to provide flexibility and accuracy was proposed by Domenech Josep et al [DOM04]. This framework was a hybrid implementation that combined both real and simulated parts in order to provide flexibility and accuracy. It reproduced in detail the behavior of web users, proxy servers and original servers. The simulator also included a module to provide performance results such as precision (prefetching accuracy), recall & response time and byte transference. It was quantified how the prefetch requests could interfere in the performance of the user requests.

Self-interference and the cross-interference could be quantified since the environment included the entire system. The evaluation of prefetching techniques for comparison has been standardized. The authors also created novel prefetching architectures that interrelate the three parts of the global system. Therefore, prefetching algorithms in any point of the system as well as to check their impact on the overall system performance can be implemented.

The paper Testing a Web Application by Sneed Harry M. [SNE04] described the requirements and constraints of testing a web application system for the Austrian Chamber of Commerce. It separated the testing task into two parts - one for testing the web architecture and one for testing the internet application itself. The high costs of testing an internet application were driven not only by the comlexity of the architecture, but also by the complex usage profiles of the application.

Non-functional quality characteristics such as usability, performance, security and interoperability took on particular importance in web-based systems. The evolution of such systems required that the evolved application be tested against the old one. A comparison tool was developed to validate both the backend data base contents and the frontend web page contents.

Sneed Harry M. [SNE05] in the paper Testing an E-Government Web Site dealt with the problem of testing web sites where the code was not available. The website in question was an E-Government website of the state of Saxony in eastern Germany. The experience of the author in extracting test cases from natural language specifications, generating test data for relational databases from the database schemas and generating XML/WSDL requests from the interface descriptions was described.

The test included testing the web pages in dialog mode as well as testing the web services in batch mode. In the end more than 450 errors were reported of which some 52 were critical. Most of the errors were related to misinterpretations of the specification. There were also some errors in the accessing of the database and many in the processing of the web service requests. The test was ended in May 2005 and the system turned over.

Xu Wuzhi et al [XU05] in their paper Testing Web Services by XML Perturbation had introduced a novel test method for XML based communication. A formal model for XML schema was defined and a method to create virtual schema when XML schemas did not exist was presented. Based on this formal model, perturbation operators were designed. The operators were used to modify virtual or real schema to satisfy defined test coverage criteria. Test cases, as XML messages, were generated to satisfy the perturbed XML schema. The method was applied as a case study to two web services. As case studies, there was no expectation of external validity of the results.

Mei Hong et al [MEI05] focused on facilitating the testing of Web services and proposed a framework for testing Web services, which can help a tester of Web services in two ways: Firstly, it could help the tester to acquire effective test data. Secondly, it could help the tester to execute the test data for the testing. In the framework, Hong mainly focused on test data acquisition, test data selection, and test data execution. For test acquisition, approaches suggested were:

1) generating test data using contract information of the service, and 2) obtaining test data via recording runtime information in real execution. For test data selection, the use of contract-based mutation was exploited that was effective in killing mutants. For test data execution, a general-purpose test driver to execute test data was used. Experimental study to evaluate the test data generation and selection strategy, showed the effectiveness of the strategy.

The black box a

To export a reference to this article please select a referencing stye below:

Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.

Request Removal

If you are the original writer of this dissertation and no longer wish to have the dissertation published on the UK Essays website then please click on the link below to request removal:

More from UK Essays

Get help with your dissertation
Find out more