Software Testing In Cloud Computing Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Software testing is a technique used to evaluate the different attributes of applications such as performance; reliability etc. There are many open source software testing tools available but there is no common platform from where user can access multiple open source software testing tools on demand for performing different types of testing, with capability of scaling up and down the resources according to the need. Now-a- days the Cloud computing has changed the way how IT services are managed and delivered. In this paper we propose open source Software Testing Framework on cloud (STFC), it will provide multiple open source software testing tools on one platform, with ease of access and range of resources available on demand.

Agility, shorter time to market, rising complexity, market competition and many other factors compel the organizations to ensure the quality, performance and the reliability of software. Modern software testing includes complete software analysis including review of the source code, design and the architecture. There are several open source software's testing tools available in the market that are used for performing different types of software testing like JMeter is used for load testing and measuring performance , Juint for unit testing of the code etc. but the matter is that organizations have to manage and maintain these multiple software for testing tenacity. And even when they succeed in implementing the tools and instilling the best practices, these things takes Time, Money & Resources.

Cloud Computing allows vendors to deliver computing resources on a large scale to the developer and the end users. It actually provides the alternative way of delivering and managing different IT services. The prosperity of provisioning the resources, storage and computing capabilities on demand with pay as you go method is actually the driving feature of the cloud computing that is fetching the attraction of the market place.

To get rid of all these issues of managing and maintaining the tools and resources and for tackling the issues of storage and computation powers, the industry is locating for a common platform for multiple software testing services so we proposed STFC. It lays at the intersection of these key areas i.e. software testing and cloud computing for providing multiple open source software testing tools on one platform, with a range of maintainable resources available on demand.

Fig. 1 shows the main idea behind the screen; it depicts that there are multiple open source software testing tools available for the respective types of testing these tools but the need is to put them over one platform not only for ease of access and usability.

Type of Testing Software









Types of testing and their respective software

This paper shows that how the power of cloud computing can be more beneficial in the world of software testing with the development of STFC. Section II gives background knowledge about the software testing trends and a small comparison between different service providers providing software testing in the cloud. Section III describes the main design of the STFC and detail of steps performed to create a common software testing framework after selecting a suitable cloud vendor and getting assured about the data security. Currently we have implemented Jmeter in the cloud; the ultimate goal is to implement multiple open source software testing tools in the cloud.


Future of Software Testing

The testing of an application becomes more crucial and time consuming with increase in complexity. Trends in the industry suggest that software testing will become more difficult in future. In order to drive the changes in effective and efficient way, more emphasis must be placed on analysis and design phases. Emphasis should be towards the 4Rs, repeatability, reliability, re-use and robustness.

Testing and Quality Assurance will become more important and add more value as we move into architectures and technologies that support the business in their goals of bringing products and services to the market as rapidly as possible, with minimal risk [1].

Recent studies show that the software testing is now moving into cloud as more and more companies understand the importance and value of cloud [2] and want to get of the problem of physical storage and computation power's limitation.

Traditional Vs. Cloud based Testing

In order to reduce the expenses and to achieve the better control on the resources, many organizations are starting using the cloud computing delivery model. As by using cloud computing resources can be accessed on demand, from anywhere, for as long as they are needed without worrying about the maintenance and the management.

Traditionally, when the organizations have to perform large scale testing they have to purchase additional servers and require extra staff which ultimately end in the increase of expenditures but by using cloud computing, test environments can be now easily and quickly replicated to the cloud and the resources can be used on demand and for as long as they are needed. This is a relatively more cost effective way [3].

There are several service providers that are providing different services of software testing on cloud; a comparison on the base of services provided is shown in the Table I.

comparision betwEen providers of sodtware testing in cloud


Performance Testing

Load Testing

Unit Testing

Functional Testing

Cloud- Intelligence[4]















HP Load Runner [7]





It is concluded from the Table I that although the power of cloud computing is being utilized in the software testing but there is still lack of some common platform for accessibility of multiple software testing services on demand.

Benefits of Testing in the Cloud

In term of resources and cost, testing on cloud have many benefits, some are as followed

To perform the load and stress testing when numerous resources are needed, cloud computing helps to reduce the infrastructure cost.

It provides the facility of scaling up and down the resources according to the need; means we have flexibility of using the resources when they are needed while paying as we go.

We can skip the time consuming process of setup and procurement by simply using the preconfigured cloud resources.

The service provider is supposed to ensure that latest version of the tool is provided. So instead of paying a heavy amount for buying a tool, keeping track of latest version, and updating the software accordingly; we just need to pay-as-we-use.

It somehow reduces the need of hiring the individual tool experts [8].

Because of such benefits, the cloud computing is becoming the best suited environment for the software testing and many organizations have started using it to save their Time, Money and resources.

software testing framework on cloud(STFC)

We present a STFC that will provide multiple open source testing tools over one platform. The STFC provides several facilities as increases the processing capabilities, saves time and generates the accurate and credible results. Traditionally while testing an application in the local environment, results are not credible and accurate all the time because of the involvement of some hardware and software factors such as processing, computing and transmission time or storage differences etc. also on the other hand if an application needs to be tested under certain load for rationale of performance testing, in some cases not only the personal computers but even the private clusters become unsuitable. They usually prohibit deficiency of lack of resources for generating the large scale load so these issues can be tackled easily by deploying testing tools in the cloud environment and utilizing the range of resources available on demand.

Selecting Cloud Vendor

Before going into the development phase the first step was to select a well suited cloud vendor. Selecting a well suited cloud vendor is elementary building block of starting using cloud [9]. The selection of well suited cloud vendor is based on multiple factors [10]. As there are multiple cloud vendors available in the market for example MS Azure, Google App engine, sales force etc. that are providing different facilities according to their architecture so making a right decision in choosing the well suited vender is a quite difficult task. There are several questions that arise in one's mind while selecting the cloud vendor like how much trust worthy the cloud vendor is? What about the security of data? Etc.

One may face several problems because of wrong selection of the cloud vendor and after getting into work it might be very difficult to switch between the cloud vendors. There are several factors that must be kept in mind while selecting the vendor for example the language support pricing and offerings etc. A comparison on the base of different factor is given in the Table II, which we have done for selecting a well suited cloud vendor for the STFC.

Comparison of cloud vendors






Windows Azure , SQL Azure, AppFabric

Google App Engine

S3, Ec2, SQS, SDB, FWS.

Load Balancing









pay-per-use pricing

compute + storage + transaction + bandwidth

Free resources up to a level After that enable billing

Pay as you go + EC2 spot Instances

Language support

Java (Not Completely)

Python, Ruby, PHP,

.Net, C#, C++, VB



-Java 6 runtime (white listed JRE classes)

Python 2.5


- Developer libraries: C++, Ruby, Python, Java, PHP

Command line utilities for EC2





On the base of such factors like elasticity, language support and services provided, we selected services of Amazon Elastic Compute Cloud (EC2) for STFC.

Data Security Confirmation on EC2

Amazon provides a number of ways for authorization and secure access to the account such as private and public key, X.509 certificates etc. Along with all these, it also provides two additional security options that enable customers to control the access to the account and to protect the account. These are Multi-Factor Authentication and Key Rotation; these methods are used as an additional security credentials along with the traditional one. AWS also provides significant security over the network layer by tackling the security attacks such as Distributed Denial of Service (DDoS) Attack, Man in the Middle (MITM) Attack, IP Spoofing, and Port Scanning etc. Fig.2 shows the security mechanism of the Amazon EC2.




Virtual Interfaces

Customer n

Security Group

Customer 2 Security Group

Customer 1 Security Group



Physical Interfaces

Amazon EC2 Security mechanism [11]

Multi-level security is provided on Amazon EC2 i.e. on the host operating system, instance operating system, on firewall, and signed API calls [11]. The basic purpose is to get confirmation that instances are secure and unauthorized users can't interrupt with the data contained in EC2 instances. Instances on ec2 are decoupled from each other through Xen Hypervision which effectively isolates operating system and its applications from the underlying physical server. AWS firewall resides within the hypervisor layer, between the physical interface and the instance's virtual interface [11]. All packets must pass through this layer, thus instances are treated as if they are on isolated physical host. Using the similar mechanism the physical RAM is also divided.

Detailed Design

After selecting a suitable vendor the next step was to create an AMI (Amazon machine Image), an AMI contains all the necessary information to launch the instances of such a system that have desirable environment and software. We selected JMeter as a starting point, and created such an AMI in which we installed and configured the JMeter so that for the next time we can simply launch the instance of this AMI and can start our work without going into the installation and configuration phases.



AMI (having multiple open source testing tools)

Load Test

Unit Test








Fig. 3 shows the architecture of the STFC; it depicts that user can launch multiple instances of an AMI (having multiple open source software testing tools) for performing different types of testing. For example he may launch an instance to load test his application using JMeter and on the same time he can also launch another instance for code review i.e. unit testing of his application, without going into the installation and management phases.


STFC Archetechture

Amazon has provided Amazon web services (AWS) console to interact with EC2, through which instances can be launched and monitored effectively. We created JMeter test plan on the local machine and our ultimate goal was to run the test plan on (EC2). We launched an Ubuntu instance and after going through installation and configuration phases of the JMeter we created an AMI of it. Fig. 4 shows the process of creating an AMI.

Launch an Instance

Install Java

Install JMeter

Environmental variables

Bundle the Image

Upload the bundle

Register the AMI

Launch instance of new AMI

Execute JMeter Test plan

Install EC2 tools

AMI Creation Steps

Once all the software are installed and configured the image of the running instance is bundled and uploaded [12] to the S3 (Simple storage service) bucket by following commands.

ec2-bundle-vol -k private key -u user-id -s size [-d destination-dir] [-e exlcude-dir-1,exclude-dir-2...] [-p ami-prefix] [-v volume]

ec2-upload-bundle -b S3-bucket -m manifest-path -a AWS-access-key-id -s AWS-secret-key [--acl acl] [--ec2certificate path] [-d directory] [--part part] [--url url] [--retry] [--skipmanifest]


Response Time (ms)

After successfully uploading the image the image is registered to get a unique AMI-ID so that we can launch the instance of the newly created AMI on the base of this unique ID. We launched the instance of our new AMI and run the JMeter test plan on it. Fig. 5 shows the graph of response time when multiple Google pages were tested under the same load in local environment and on EC2.

Response Time comparison on EC2 and Local envirnoment

Load and performance testing from the Cloud is the best way to impersonate real world traffic coming from outside the firewall, around the world as we tested the multiple Google pages over the EC2 and found that the results had the remarkable difference. The main focus in this paper is on the differences between the results, which occur because of involvement of the factors like transmission time, bandwidth, distance etc. The results show that because of involvement of such factors there is remarkable difference between the results.


Software testing in cloud helps the organizations in reducing the expenditures and saving time by bowing down the curve of maintenance cost and management time etc. it also provides more credible and accurate results as compared to the local environment.

In this paper we proposed the STFC for providing different software testing tools over one platform that not only provides ease of use and maintainability as well as provides more accurate and credible results. We performed tests in the cloud using Jmeter and the results had a remarkable difference.