Cloud testing


Cloud testing

Cloud testing is a form of software testing in which web applications use cloud computing environments (a "cloud") to simulate real-world user traffic.

Contents

Overview

Cloud Testing uses cloud infrastructure for software testing.[1] Organizations pursuing testing in general and load, performance testing and production service monitoring in particular are challenged by several problems like limited test budget, meeting deadlines. High costs per test, large number of test cases, and little or no reuse of tests and geographical distribution of users add to the challenges. Moreover ensuring high quality service delivery and avoiding outages requires testing in one's datacenter, outside the data-center, or both. Cloud Testing is the solution to all these problems. Effective unlimited storage, quick availability of the infrastructure with scalability, flexibility and availability of distributed testing environment reduce the execution time of testing of large applications and lead to cost-effective solutions.

Need for Cloud testing

Today majority of Applications run on internet and depend on good performance of the network.[2]

Causes of performance degradation

The following factors cause degradation in the performance.[3]

  • Unpredictable rate of increase in number of users
  • Fast paced addition of new features to existing applications
  • Changes in the third party allied software
  • Variation in deployment environment depending on client requirements

Limitations of traditional testing techniques

Traditional approach to test these applications use internal testing techniques that have the following limitations.[4]

  • Testing at web scale
  • Testing firewalls and load balancers[5]
  • Simulation of user activity from different geographic locations
  • Testing external components
  • High Cost of hardware, software and its maintenance[6]

Cloud testing is a new approach which helps us to measure accurate performance by externally load testing these applications and overcome the limitations of traditional testing.

Types of testings supported

Types of testing done in cloud

Stress test

Stress Test is used to determine ability of application to maintain a certain level of effectiveness beyond breaking point. It is essential for any application to work even under excessive stress and maintain stability.[7] Stress testing assures this by creating peak loads using simulators.But the cost of creating such scenarios is enormous. Instead of investing capital in building on-premise testing environments, cloud testing offers affordable and scalable alternative.

Load Test

Load testing of an application involves creation of heavy user traffic, and measuring its response. There is also a need to tune the performance of any application to meet certain standards.

Performance Test

Finding out thresholds, bottlenecks & limitations is a part of performance testing.[8] For this, testing performance under a particular workload is necessary.[9] By using cloud testing, it is easy to create such environment and vary the nature of traffic on-demand. This effectively reduces cost and time by simulating thousands of geographically targeted users.

Functional testing, latency testing, browser performance, compatibility testing are some other types of testing that can be performed in the cloud.

Steps

Companies simulate real world Web users by using cloud testing services that are provided by cloud service vendors such as SOASTA, HP, Load Impact, Compuware and Keynote systems. Once user scenarios are developed and the test is designed, these service providers leverage cloud servers (provided by cloud platform vendors such as Amazon.com, Google, Rackspace, etc.) to generate Web traffic that originates from around the world. Once the test is complete, the cloud service providers deliver results and analytics back to corporate IT professionals through real-time dashboards for a complete analysis of how their applications and the internet will perform during peak volumes.

Steps for Cloud Testing

Keys to successful testing

  1. Understanding a platform provider's elasticity model/dynamic configuration method
  2. Staying abreast of the provider's evolving monitoring services and Service Level Agreements (SLAs)
  3. Potentially engaging the service provider as an on-going operations partner if producing commercial off-the-shelf (COTS) software
  4. Being willing to be used as a case study by the cloud service provider. The latter may lead to cost reductions.

Applications

Cloud testing is often seen as only performance or load tests, however, as discussed ealier it covers many other types of testing. Cloud computing itself is often referred to as the marriage of Software as a Service (SaaS) and Utility Computing. In regard to test execution, the software offered as a service may be a transaction generator and the cloud provider's infrastructure software, or may just be the latter. Distributed Systems and Parallel Systems mainly use this approach for testing, because of their inherent complex nature. D-Cloud[10] is a software testing environment which uses Cloud Testing for designing Fault-tolerant systems.

Tools

Leading cloud computing service providers include, among others, Amazon, 3-terra, Skytap, HP and SOASTA.[11]
Some of the tools for cloud testing include :

  1. SOASTA
  2. Cloud Assault
  3. Parasoft® SOAtest™
  4. HP LoadRunner
  5. Keynote
  6. LoadStorm™
  7. Cloud Testing

Benefits

The ability and cost to simulate web traffic for software testing purposes has been an inhibitor to overall web reliability. The low cost and accessibility of the cloud's extremely large computing resources provides the ability to replicate real world usage of these systems by geographically distributed users, executing wide varieties of user scenarios, at scales previously unattainable in traditional testing environments. Minimal start-up time along with quality assurance can be achieved by cloud testing.

Following are some of the key benefits:

Issues

The initial setup cost for migrating testing to cloud is very high as it involves modifying some of the test cases to suit cloud environment. Thus, making the decision of migration crucial.[14] Cloud testing is not necessarily the best solution to all testing problems.[15]
Legacy systems & services need to be modified in order to be tested on cloud.Usage of robust interfaces with these legacy systems may solve this problem.[16] Also like any other cloud services, cloud testing is vulnerable to security issues.
The test results may not be accurate due to varying performance of service providers’ network and internet.[17]

Current scenario

An evolving cloud testing community is forming under the auspices of the Software Testing & Quality Assurance group hosted by LinkedIn. Testing professionals openly share their experiences and exchange ideas related to cloud testing in order to enhance each-others proficiencies.

References

  1. ^ Tilley, S.; Parveen, T. (12). "Migrating software testing to the cloud". Software Maintenance (ICSM), 2010 IEEE International Conference. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5610422. 
  2. ^ "Business Needs vs. Network Performance: Critical Challenges Facing Network Managers". http://www.shunra.com/resources/articles/business-needs-vs-network-performance-critical-challenges-facing-network-managers. Retrieved 12 October 2011. 
  3. ^ Dubie, Denise. "Poor application performance translates to lost revenue, research shows". http://www.networkworld.com/newsletters/2008/080408nsm2.html?page=1. Retrieved 12 October 2011. 
  4. ^ Girmonsky, Alon. "Cloud-testing compared to traditional testing with-in the corporate LAN". http://stress-testing.org/content/cloud-testing-compared-traditional-testing-corporate-lan. Retrieved 12 October 2011. 
  5. ^ "Challenges in Testing Web Based Applications". Exforsys Inc.. http://www.exforsys.com/tutorials/testing/challenges-in-testing-web-based-applications.html. Retrieved 12 October 2011. 
  6. ^ Ariola, Wayne. "The Next Generation of Test Environment Management". http://virtualization.sys-con.com/node/1905782. Retrieved 12 October 2011. 
  7. ^ "http://www.soasta.com/2009/01/26/using-the-cloud-to-stress-test-your-web-applications/". http://www.soasta.com/2009/01/26/using-the-cloud-to-stress-test-your-web-applications/. Retrieved 12 October 2011. 
  8. ^ "Performance and Load Testing Services using Cloud Computing". http://www.trigent.com/solutions/isvs/cloud-testing-services.htm. Retrieved 12 October 2011. 
  9. ^ Ganon, Z.; Zilbershtein, I.E. (12). "Cloud-based Performance Testing of Network Management Systems". Computer Aided Modeling and Design of Communication Links and Networks: 1–6. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5161466. Retrieved 12 October 2011. 
  10. ^ Banzai, Takayuki; Koizumi, Hitoshi; Kanbayashi, Ryo; Imada, Takayuki; Hanawa, Toshihiro; Sato, Mitsuhisa (17). "D-Cloud: Design of a Software Testing Environment for Reliable Distributed Systems Using Cloud Computing Technology". Cluster, Cloud and Grid Computing (CCGrid), 2010 10th IEEE/ACM International Conference: 631–636. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5493414. 
  11. ^ "Top 10 Cloud Computing Load Test and Performance Monitoring Companies". http://www.cloudtweaks.com/2010/08/top-10-cloud-computing-load-test-and-performance-monitoring-companies/. 
  12. ^ "Test infrastructure in the Cloud business case; a cost reduction, or not?". http://www.testingthefuture.net/2010/11/test-infrastructure-in-the-cloud-business-case-a-cost-reduction-or-not/. Retrieved 12 October 2011. 
  13. ^ "Five Benefits of Software Testing On Cloud". http://itknowledgeexchange.techtarget.com/quality-assurance/five-benefits-of-software-testing-on-cloud/. Retrieved 12 October 2011. 
  14. ^ T., Parveen; Tilley, S. (6-10). "When to Migrate Software Testing to the Cloud?". Software Testing, Verification, and Validation Workshops (ICSTW): 424–427. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5463680. Retrieved 12 October 2011. 
  15. ^ S., Tilley; Parveen, T. (12-18). "Migrating software testing to the cloud". Software Maintenance (ICSM): 1. http://ieeexplore.ieee.org/search/srchabstract.jsp?tp=&arnumber=5610422&queryText%3DMigrating+Software+Testing+to+the+Cloud%26openedRefinements%3D*%26filter%3DAND%28NOT%284283010803%29%29%26searchField%3DSearch+All&tag=1. Retrieved 19 October 2011. 
  16. ^ Roodenrijs, Ewald (1). Testing on the Cloud. http://www.sogeti.com/upload/Curious%20about%20us/Documents/PoV%20-%20A%20Sogeti%20Test%20Cloud_v1%200.pdf. 
  17. ^ MacVittie, Lori. "Cloud testing: The next generation". http://www.networkworld.com/news/tech/2011/020911-cloud-testing.html?page=1. Retrieved 12 October 2011. 

External links


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Cloud seeding — can be done by ground generators, plane, or rocket (not shown). Cloud seeding, a form of intentional weather modification, is the attempt to change the amount or type of precipitation that falls from clouds, by dispersing substances into the air… …   Wikipedia

  • Cloud computing security — (sometimes referred to simply as cloud security ) is an evolving sub domain of computer security, network security, and, more broadly, information security. It refers to a broad set of policies, technologies, and controls deployed to protect data …   Wikipedia

  • Cloud reflectivity modification — Cloud reflectivity enhancement is also known as marine cloud brightening or cloud whitening on low cloud. An opposite scheme exists to reduce the reflectivity of higher, colder cirrus clouds.[1] It is a geoengineering technique that works by… …   Wikipedia

  • St. Cloud Hospital — Infobox Hospital Name = St. Cloud Hospital Org/Group = CentraCare Health System Caption = map type = latitude = longitude = Location = Region = St. Cloud State = Minnesota Country = United States Coordinates = coord|45|34|31|N|94|10|9|W|region:US …   Wikipedia

  • materials testing — Introduction       measurement of the characteristics and behaviour of such substances as metals, ceramics, or plastics under various conditions. The data thus obtained can be used in specifying the suitability of materials for various… …   Universalium

  • Underground nuclear testing — Preparation for an underground nuclear test at the Nevada Test Site in the 1990s. Nuclear weapons …   Wikipedia

  • Rackspace Cloud — Type Division Industry Cloud computing Founded March 4, 2006 Parent …   Wikipedia

  • Mushroom cloud — created by smoke billows Ascending cloud from Redoubt Vol …   Wikipedia

  • Nuclear testing — Nuclear weapons tests are experiments carried out to determine the effectiveness, yield and explosive capability of nuclear weapons. Throughout the twentieth century, most nations that have developed nuclear weapons have tested them. Testing… …   Wikipedia

  • Nuclear weapons testing — Nuclear weapons History Warfare Arms race Design Testing Effects Delivery Espionage …   Wikipedia


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.