Stress testing (software)

Stress testing (software)

In software testing, stress testing refers to tests that determine the robustness of software by testing beyond the limits of normal operation. Stress testing is particularly important for "mission critical" software, but is used for all types of software. Stress tests commonly put a greater emphasis on robustness, availability, and error handling under a heavy load, than on what would be considered correct behavior under normal circumstances.

Field experience

A major software company has identified that failure to execute load and stress tests occurred in about 1 of 10 engagements. Another failure that frequently occurs in the field is to use non production like environments. In some cases testing is done against smaller, hence faster responding databases than the ones that are used in production.

Rationale

Reasons for stress testing include:
* The software being tested is "mission critical", that is, failure of the software (such as a crash) would have disastrous consequences.
* The amount of time and resources dedicated to testing is usually not sufficient, with traditional testing methods, to test all of the situations in which the software will be used when it is released.
* Even with sufficient time and resources for writing tests, it may not be possible to determine beforehand all of the different ways in which the software will be used. This is particularly true for operating systems and middleware, which will eventually be used by software that doesn't even exist at the time of the testing.
* Customers may use the software on computers that have significantly fewer computational resources (such as memory or disk space) than the computers used for testing.
* Concurrency is particularly difficult to test with traditional testing methods. Stress testing may be necessary to find race conditions and deadlocks.
* Software such as web servers that will be accessible over the Internet may be subject to denial of service attacks.
* Under normal conditions, certain types of bugs, such as memory leaks, can be fairly benign and difficult to detect over the short periods of time in which testing is performed. However, these bugs can still be potentially serious. In a sense, stress testing for a relatively short period of time can be seen as simulating normal operation for a longer period of time.

Relationship to branch coverage

"Branch coverage" (a specific type of code coverage) is a metric of the number of branches executed under test, where "100% branch coverage" means that every branch in a program has been executed at least once under some test. Branch coverage is one of the most important metrics for software testing; software for which the branch coverage is low is not generally considered to be thoroughly tested. Note that code coverage metrics are a property of the tests for a piece of software, not of the software being tested.

Achieving high branch coverage often involves writing "negative" test variations, that is, variations where the software is supposed to fail in some way, in addition to the usual "positive" test variations, which test intended usage. An example of a negative variation would be calling a function with illegal parameters. There is a limit to the branch coverage that can be achieved even with negative variations, however, as some branches may only be used for handling of errors that are beyond the control of the test. For example, a test would normally have no control over memory allocation, so branches that handle an "out of memory" error are difficult to test.

Stress testing can achieve higher branch coverage by producing the conditions under which certain error handling branches are followed. The coverage can be further improved by using fault injection.

Examples

* A web server may be stress tested using scripts, bots, and various denial of service tools to observe the performance of a web site during peak loads.


Wikimedia Foundation. 2010.

Игры ⚽ Нужна курсовая?

Look at other dictionaries:

  • Stress testing — is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. Stress testing may have a more specific… …   Wikipedia

  • Inquisitor (hardware testing software) — Infobox OS name = Inquisitor caption = Inquisitor 3.0 developer = Mikhail Yakshin, et al family = Linux source model = Open source working state = Current latest release version = 3.0 latest release date = release date|2008|07|20 kernel type =… …   Wikipedia

  • Software performance testing — In software engineering, performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such… …   Wikipedia

  • Software quality assurance — (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. It does this by means of audits of the quality management system under which the software system is created. These audits are backed by …   Wikipedia

  • Stress analysis — is an engineering discipline that determines the stress in materials and structures subjected to static or dynamic forces or loads (see statics and dynamics) (alternately, in linear elastic systems, strain can be used in place of stress). The aim …   Wikipedia

  • Software testing outsourcing — provides for software testing carried out by the forces of an additionally engaged company or a group of people not directly involved in the process of software development. Contemporary testing outsourcing is an independent IT field , the so… …   Wikipedia

  • Software house — A software house is a company whose primary products are composed of software, i.e., computer programs. Types of software houses There are a number of different types of software houses: *Large and well known companies such as Microsoft, Oracle… …   Wikipedia

  • Cloud testing — is a form of software testing in which web applications use cloud computing environments (a cloud ) to simulate real world user traffic. Contents 1 Overview 2 Need for Cloud testing 2.1 Causes of performance degradation …   Wikipedia

  • System testing — of software or hardware is testing conducted on a complete, integrated system to evaluate the system s compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no… …   Wikipedia

  • Lightweight software test automation — is the process of creating and using relatively short and simple computer programs, called lightweight test harnesses, designed to test a software system. Lightweight test automation harnesses are not tied to a particular programming language but …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”