Posts tagged Testing
Automated Testing – Manual Vs Automation
Feb 19th
Automated Testing – Manual Vs Automation
If you‘re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying ?use common sense? when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate.
Writing the test cases and writing or configuring the automate framework you‘re using costs more initially than running the test manually.
Can‘t automate visual references, for example, if you can‘t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). More bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
1. Running tests manually can be very time consuming
2. Each time there is a new build, the tester must rerun all required tests – which after a while would become very mundane and tiresome.
Other deciding factors
1. What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
2. Is the return on investment worth automating?
3. Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case: Is this test scenario automatable?
1. Yes, and it will cost a little
2. Yes, but it will cost a lot
3. No, it is no possible to automate
How important is this test scenario?
1. I must absolutely test this scenario whenever possible
2. I need to test this scenario regularly
3. I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can‘t automate?
Let‘s say that you have a test that you absolutely need to run whenever possible, but it isn‘t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What‘s the cost of doing this test manually?
Look for new testing tools
Consider test hooks
Our Software Testing Partner
Software testing institute
Outsource Testing Services in Australiya
CresTech provides software testing
training,software testing, corporate trainings and
IT certifications & workshops on HP LoadRunner,
WinRunner, ISTQB, CSTE, QA, QC, IBM Rational, QTP,
RFT and many more. CresTech has operations in all
major cities in India including Delhi, NCR, Noida,
Gurgaon, Jaipur.
Testing of Windows Drivers
Feb 19th
1. Introduction
As we know, Windows drivers can run and work in either user mode or kernel mode.
Kernel-mode drivers run as part of the operating system and possess the privileged abilities. The drivers reviewed below are the drivers of kernel level. As any software testing procedure, driver testing includes automatic and manual testing, but in this article we will focus mainly on the last one.
Later we will review testing process for the followings types of drivers:
File System Filter Drivers Network Redirection Drivers Virtual Storage Drivers USB Client Device Drivers
2. General aspects of driver testing
First of all, I would like to say “Don’t install the driver to be tested on your working system!” However, there are some exceptions on this occasion, for example:
You are looking for a reason to reinstall your system… You don’t want to waste time on searching bugs on the clean test systems… You are sure that the driver is stable… You have to test the work of the connected devices…
Secondly, driver tester must have a virtual machine installed on the working computer and at least one real test computer.
Thirdly, before starting the testing procedure, it is necessary:
1. To define the number of operating systems for which it is necessary to check the driver functionality. Usually, first of all you use the OS confirmed by the specification. Let’s suppose that a driver must successfully work under Windows 2000/XP/Server 2003/Vista accordingly to the requirements. Applying our experience in driver testing, we extend this list to the following:
Windows 2000 Windows 2000 sp4 Windows XP_sp0 Windows XP_sp1 Windows XP_sp2 Windows XP_(Windows is installed on FAT32) (obligatory system) Windows Server 2003 Standard Edition Windows Server 2003 Enterprise Edition Windows Vista
Additionally:
Windows Server 2003 Small Business Windows Server 2003 Enterprise EditionSp1
At that, it’s desirable for all of these systems to be Checked Build versions. Using the Checked version slows down Windows functioning, but multiplies probability of finding bug in the driver functioning. If there is no a single Checked Windows, it is necessary to purchase the Checked version of at least one system, for example from the Windows XP series.
Thus Windows XP_Chk is added to the OS list. But you shouldn’t limit you work only to this list: the more the number of OS, under which driver passed the complete testing successfully, the better quality of product guaranteed.
2. If possible, select test computers with as different hardware and configurations as possible. I recommend obligatory test on machines with single processor, single processor with Hyper-Threading, two-core processor and dual processor machines.
3. To make the following settings for each OS where driver testing will be performed: in System Properties\Advanced\Startup and Recovery Settings\Write debugging information it’s necessary to select Kernel memory dump or Complete memory dump, and uncheck Automatically restart option for the case of you won’t be nearby in the moment of Blue Screen appearance.
4. To set verification by standard Driver Verifier Manager of the system with all options, except for Low resource simulation. This is determined by the fact that at first you must obtain stable functioning of driver without the option of Low resource simulation, and then test the functioning with this option on smoke tests.
5. To select antivirus software with which it is desirable to test the driver performance. Usually it’s recommended to take the most commonly used antivirus programs, such as DrWeb, Kaspersky Antivirus, V3, Symantec Norton Antivirus, NOD32. The list is not limited to these names, of course. The more programs you will test, the better driver performance will be; this will only increase the quality of product.
6. To verify compatibility of your driver with other drivers, the system ones and the drivers by other producers. It is desirable for the last ones to be stable enough – in this case it will be easier for you to discover reasons for improper work of the system or Blue Screen appearance, and to expose bugs in driver performance.
The next general aspect of driver testing is verification of driver loading and unloading, i.e. you set the group of tests that are relevant to each driver:
Driver installation. The script for driver installation can be written by programmer, but the tester verifies whether it’s done correctly. Here we check the following: Has the system crashed after driver installation or not? Has *.sys file been written in the C:\WINDOWS\system32\drivers directory? Have necessary keys been registered in Registry, if they are presented in our case?
The driver installation not always supposes its start – it depends on implementation of driver. Some of them run after system restart and cannot be stopped during the process of work (usually these are File System Filter Drivers), therefore the following questions arise: Does system start correctly with installed driver? Does system crash when the driver is started manually? System Restart while the driver is working. System must be normally rebooted without Blue Screen; after system start, the driver and its settings must comply with the statuses described in the specification. System Reset while the driver is working. System must perform the Reset operation and start without Blue Screen; after system start the driver and its settings must comply with the statuses described in the specification. System shutdown while the driver is functioning. System must perform the Shut down operation successfully without Blue Screen. System Start with the pre-installed driver. System must be rebooted without Blue Screen; after the system starts, the driver and its settings must comply with the statuses described in the specification.
Further work of driver tester includes functional testing of the testing object and search of BSOD, which features depend on the type of testing driver.
3. File System Filter Drivers Testing
The drivers of this type are designed for enhancement of the file system functionality, for example for catching operations of OS in order to watch, forbid or permit certain processes, operations, files in certain disk space. The set of tests to verify the driver functionality is formed in accordance with the type of driver and here you have to pay special attention to the software products workings with network places.
Let’s consider an example when we use a database with the multi-users connection status. Let’s suppose that the real multi-users DB1 is on the computer N1 and we can work with DB1 on the computer N2 without problems. Let’s install some File System Filter Driver on the computer N1 and computer N2. When trying to get access to DB1 from the computer N2 we can get such results:
DB1 on the computer N1 is not accessible for the computer N2; N1 computer has crashed with BSOD; N2 computer has crashed with BSOD; DB1 remained accessible for using on the computer N2.
It is necessary to test an access to Shared Folders from one machine to another. Thus you have to test standard operations of creation, deleting, copying, renaming, moving of small and large folders and files in Shared Folders.
An insidious test is the starting of some *.exe file both from network space and locally.
On the test system, it’s useful to run software products which perform file system indexing (for example, Google Desktop). BSOD or Dead Lock of system with installed File System Filter Driver can occur when such programs are started or functioning.
System can work inadequately after installation of new Virtual Storage Driver (these drivers are developed for creation of virtual encrypted disks) or while it’s functioning. BSOD or noticeable slow down in system performance can happen as a result of File System Filter Driver and Virtual Storage Driver interaction – naturally, it’s not good.
In my opinion, this type of drivers is the hardest from the view of covering full set of tests, and the most unforeseeable in system crashes while driver is functioning.
4. Network Redirection Drivers Testing
Network Redirection Driver is a network file system which gives access to the resources of remote computer. In practice, such drivers are used for creation of network disks, and we will examine such drivers in this paragraph.
First of all, it’s necessary to select test data, so we will take:
Files and folders of small size (to 10Mb); Files and folders of large size(more than 600Mb); Folders containing some nested files and folders (one folder for one file); Folders containing a great number of nested files and folders (nesting is equal to 5; the number of files (size to 1K) is more than 100 in each folder).
In this case main tests are divided into the followings groups:
Group #1 is “Verification of network disk displaying”. After connecting network disk that is implemented on the base of Network Redirection Driver, this disk must be displayed in all of the file managers.
Group #2 is “Verification of functionality”
Testing the standard operations (opening, change, hiding, creation, deleting and other) with files and folders on a network disk. Testing the operations of copying, saving files and folders to/from a network disk from/to a local disk. It’s necessary to pay special attention to all of applications included in MSOffice package, MindManager and also to verify saving project of MS Visial Studio from local space to a network disk and compiling this project. Testing the operations of copying when size of copied files is changed and also when an operation of copying/saving files is interrupted incorrectly. Testing software product on the different language versions of OS, for example under English and Korean OS, and performing tests from the first three points of Group #2. The driver must automatically pick up the language of the system and successfully perform all available operations with files and folders. Testing search function on a network disk. Checking the data integrity when moving them from a local disk to a network one and vice versa.
Group #3 is “Verification of data exchange speed”
Testing speed on every set from the group of test data when downloading to Network Disk from Local Disk. Testing speed on every set from the group of test data when unloading from Network Disk to Local Disk.
At that, it is necessary to take into account the capabilities of data transfer over the local network and capabilities of network protocols both with data cashing and without it. If you can get an analogue product, then compare its speed with the speed of your product.
Group #4 is “Disconnect”
Checking the reaction of the system to disconnecting the network cable with the network disk connected: during uploading of each set from the group of test data; during downloading of each set from the group of test data. Testing the reconnect of network disk after plugging connecting network cable to the computer.
Group #5 is “Users sessions”
Testing network disk performance for different users in the terminal session of Windows Server2003. Testing network disk performance for different users with Switch option on Windows XP. Testing network disk performance for different users with Login switching on Windows Server2003.
5. Virtual Storage Drivers Testing
Virtual Storage Driver is virtual data storage (for example, it can use random-access memory or file). Usually it is used for implementation of virtual encrypted disks.
Main points when testing this type of drivers are:
Standard operations (opening, change, hiding, creation, deleting and other) over files and folders on a virtual disk. Creation of image of maximum possible size; its formatting. Data safety when working with different encrypting algorithms. Driver speed (reading / writing to/from a virtual disk). Place of images localization (USB memory, Network Disk, Local Disk) and ability to access it. Data integrity on repeated connection of image after incorrectly disabled access to image had happened.
6. USB Client Device Drivers Testing
USB Client Device Driver manages USB device, interacts with it by means of USB packages (URB), for example, for redirecting requests to another computer. The real USB devices are always involved in testing of such drivers; therefore the critical errors occur mostly during interaction between the driver that is tested and devices’ drivers. For example, BSOD can happen on a computer with USB Client Device Driver installed when the following actions are performed:
connecting new USB device; correct removing of USB device; incorrect removing of USB device; installing drivers for USB devices; working with USB device.
7. Conclusion
Remember that correctly chosen set of tests for the certain type of driver helps your to improve the quality of your product.
You can learn more details about how the software driver testing is performed in Apriorit here.
Testing Your Way to Website Profits
Feb 18th
Testing has always an important part in marketing and advertising. Companies that took the time to test were the ones that were able to zero in on offers yielding high conversion rates.
The problem is that, for the small business owner, large scale direct mail testing is too expensive and not practical. The solution? Use your website as a means to quickly and easily test page elements, finding the combination that maximizes conversion rate.
We are going to look at two basic types of web testing; A/B split testing and multivariate testing.
A/B Split Testing
This is the most basic type of test you can perform. The way it works is by splitting your web traffic evenly across two different versions of a page. Let’s say you have a landing page for the sale of your ebook. You get decent sales from the page but you want to find a way to improve those conversions.
With an A/B split test, you would test an alternate version of your sales page against the current one. A program works in the background to deliver the current version of the page to 50% of visitors and the new test page to the other 50%. After a set period of time, you see which page is converting better and use that as your new landing page (as well as the page to beat in future A/B split tests.)
One disadvantage of A/B split testing is that you can really only test one item at a time. For example, suppose you test a page by changing both the heading and the image. After the test, you find the page has a 25% higher conversion rate than the original. What you don’t know is whether it was the new heading, the new image, or the combination of the two that made the difference.
Because of this, A/B split testing is typically done by changing only one variable at a time. This means that you cannot test variable interactions and combinations. So, A/B split testing is best for when you only want to test a single page element. But what if you need to test multiple factors? That’s a job for multivariate testing.
Multivariate Testing
Multivariate testing allows you to simultaneously test several variables on a page. This allows for variable interactive testing, something that just can’t be done with A/B Split testing.
For example, with multivariate testing, you could test a page’s headline, images, pricing and guarantees all at the same time. Each of your visitors would see a different combination of elements and, given enough time, you would see the combination that produced the highest conversion rate.
You would then change out your existing sales page with the one the produced the highest conversion rate from your multivariate test. Because you are testing multiple elements, you need to have a good amount of traffic coming to your site to complete the test in a reasonable time. If you don’t have at least 200 unique visitors per day, then stick with A/B split testing until your traffic increases.
The Google Website Optimizer
The Google Website Optimizer is a free program allows you to run both A/B split and multivariate tests on your website. You’ve just have to give credit to the folks at Google. They have made it possible for the small business owner to use sophisticated conversion rate tools without the need to purchase expensive software or on-site consultants.
Check out the Google Website Optimizer homepage today for instructions on setting up your tests. Make no mistake, testing your landing pages is the number one way to increase your conversion rates. Start testing your way to higher profits today.
Corte Swearingen has been a marketing professional for 20 years and is the creator of the Integral Marketing System, CEO of SmallBiz Marketing Tips and the The SmallBiz Marketing Blog.
Medical Protective Clothing Testing
Feb 18th
Labthink, the excellent provider of testing instruments and testing services, is devoted to provide most excellent and complete quality control solutions for pharmaceutical, foodstuff, cosmetics, packaging, printing, adhesive, automotive, petrochemistry, environment, biology, new energy, construction, aviation and electronic industries worldwide.
Medical Protective clothing testing mainly includes the following items: barrier property testing (gas transmission quantity testing and moisture transmission testing), coefficient of friction(smoothness of material surfaces),tensile and elongation, peeling strength, heat seal strength, leak & seal, tearing and printing quality testing. Labthink Instruments Co. Ltd., meeting the demands for medical protective clothing testing, integrates medical protective clothing instruments as follows, so as to contribute to the quality testing of medical protective clothing enterprises.
1. Medical Protective Clothing Barrier Property Testing
Barrier property is defined as the barrier effects of packaging materials on gas (e.g., oxygen), liquid (e.g., water vapor) and other infiltrations. Barrier Property is the important factor affecting the product shelf-life. Relevant instruments are listed below:
A. PERME-DM2/330 Differential and Equal Pressure Methods Gas Permeability Tester: this instrument combines differential pressure method and equal pressure method in one unit. It is applicable for the gas transmission rate testing of plastic films, laminated films and sheets as well as finished packages. This instrument is fully automatic with three independent chambers and random temperature data curve fitting. It’s with temperature control; LAN data management.
B. PERME OX2/230 Oxygen Transmission Rate Testing System: is of coulometric principle and equal pressure method. This instrument is applicable for the oxygen transmission property testing of films and sheets, as well as bottles, cans and boxes. This instrument is of master base and satellite module mode: one master base can support 9 satellite modules with the realization of 30 samples tested simultaneously.
C. VAC-V Series Differential Pressure Gas Permeability Tester: is applicable for the oxygen, nitrogen and carbon dioxide permeability testing of plastic films and sheets as well as laminated films. This series of instruments, including VAC-V1, VAC-V2 and VAC-V3, etc., complies with differential pressure method.
D. PERME?W3/330 Water Vapor Transmission Rate Testing System: is used for the water vapor transmission rate testing of films and sheets as well as bottles, bags, cans and boxes. This instrument is of Master–satellite mode, one master base can connect 9 satellite modules with the realization of 30 samples tested simultaneously.
E. TSY-T Series Water Vapor Permeability Tester: is applicable in the water vapor transmission testing of packaging materials such as films and laminated films as well as high polymer products. This series of products are fully automatic, computer control and of weighing method. The series includes TSY-T1H, TSY-T1L and TSY-T3, etc.
2. Medical Protective Clothing Strength Testing
Medical protective clothing strength testing includes index testing like material tensile strength testing, peeling strength testing of laminated films, hot seal strength testing, tearing strength and pouch resistance testing. Tensile strength is the maximum strength value just before the material is snapped. Through testing, the package damage and rapture caused by external forces and insufficient packaging material mechanical strength can be resolved effectively. Peeling strength testing, also called complex strength testing, tests the bond strength between the layers of laminated films. If the bond strength is too low, leakage is easily caused by separation among the layers. Hot seal strength testing is to test the strength of the seals. During storage and transportation of products, low hot seal strength will cause problems such as rending and leakage. Pouch resistance is the index evaluating the anti-puncture strength of the package against rigid material.
Relevant testing instruments are as follows:
A. XLW Series Auto Tensile Testers: are for the peeling, tensile, heal seal and tearing tests of films, laminated films, adhesives and tapes. This series of products includes XLW?XLW?B??XLW?M??XLW?PC?and XLW?G?.
B. BLD-200S Auto Stripping Tester: is for the peeling, tensile and heat seal testing for films, laminated films, adhesive and tapes. The standard specifications are 200N (100N, 50N and 30N); mini-printer and computer connective.
C. HST-H3 Heat Seal Tester: is for the specimen preparation for the heat seal strength testing of films. Testing pressure, temperature and time are adjustable. This instrument adopts microcontroller for intellectual operation and PID high precision components for temperature so as to ascertain accurate temperature.
D. RTD-R2 Gradation Heat-Sealing Instrument: can prepare five groups of plastic films or flexible packaging laminated films simultaneously under five independent temperatures; and seal five groups of specimen simultaneously and efficiently.
3.Medical Protective Clothing Sealing Property Testing
The medical protective clothing sealing property testing can guarantee the completeness of the whole package and avoid leakage and deterioration caused by the inferior sealing property.
A.MFY-01 Leak Tester: is applicable for the package seal reliability testing by vacuumizing test chamber. Micro-computer control, panel operation, digital parameter pre-set and full-automatic testing.
B. LSSD-01 Leak and Seal Strength Detector: is for the heat seal and sealing completeness testing of bags, plastic bags and packages. The instrument is with intellectual testing process, LCD display, statistics communication and positive principle.
4.Medical protective clothing impact resistance testing can guarantee the effective protection of user. Impact resistance property testing can be classified as dart impact and pendulum impact.
A. Labthink BMC-B1 Falling Dart Impact Tester: is applicable in quality and energy testing of plastic films and sheets under the attack of free-falling dart at given heights, according to free-falling dart theory.
B. Labthink FIT-01 Film Impact Tester: is applicable to evaluate the force damaging the plastic film, laminated film, sheet and foil sample with its hemisphere head falling at certain speeds, so as to bring forward shock resistance evaluation of the material by measuring the consumed energy of the head.
5.Medical Protective Clothing Tearing Property testing
Tearing of medical protective clothing may occur because of external forces in storage, transportation and usage. Sufficient tearing resistant forces can reduce tearing and avoid dangers.
SLY-S1 Tearing Tester complies with Elmendorf method, and is applicable in tearing resistance test of films, laminated films, foils, paper and paper boards.
6.Medical Protective Clothing Coefficient of Friction Testing (Surface Smoothness)
Proper smoothness of the two sides of the clothing material can guarantee the good performances as well as smooth transportation and packaging in the assembly line.
A. MXD Series of Coefficient of Friction Tester is applicable in static and kinetic coefficient of friction test to plastic film, foils, sheets and paper. Through testing the smoothness of the materials, comfortableness and safety of medical protective clothing can be guaranteed to meet the demands. This series of products includes MXD-01, MXD-01A and MXD-02.
B. FTP-F1 Friction/Peel Tester: is computer control, and can test the material coefficient of friction between room temperature and 99.9 Centigrade as well as peeling strength of laminated films and adhesive products.
7.Medical Protective Clothing Testing for Thickness
Thickness is the basic index to test films. Uneven thickness would influence the tensile strength and barrier property as well as the following processing.
A. CHY-C2 Thickness Tester: is for the measurement of films, sheets and paper with mechanical contacting method, with no limitation for measured materials. Import high quality sensor with the resolution up to 0.1 ?m. The contact area and pressure between measure head and film/paper complies with relevant standards strictly, including multi-standards such as ISO and ASTM.
B. CHY-CA Thickness Tester: is of high precision up to 0.1?m, multi-point automatic continuous testing, LCD display, micro-printer and other distinguished functions of statistics and communication.
8.Medical Protective Cloth Printing Quality Testers
Ink-layer Bond Strength: YGJ-02 Adhesive Tape Roller and BLJ-01 Disc Peeling Tester work together to test the ink-layer bond strength of plastic films and cellophane paper (including laminated film printing products). Those testers can also be applied to test the adhesive state between the layers of vacuuming coating, exterior coating and compound.
MCJ-01A Rub Tester: is applicable for the abrasion resistance testing of ink layers and the photosensitive coating of PS board, so as to effectively analyze the issues of low abrasion resistance, ink layer falling-off, low press resistance and inferior coating rigidness of other products.
RT-01 Rub Tester: is for ink layer abrasion resistance testing of printing products. Arc moving mode; and double station design. This tester is highly effective and complies with ASTM, TAPPI and other international standards.
Labthink Instruments Co., Ltd.
No. 144 Wuyingshan Road, Jinan 250031, China
Tracy Bao info@labthink.cn
Tel: 0086 531 85061153 fax:0086 531 85812140
Labthink, the excellent provider of testing instruments and testing services, is devoted to provide most excellent and complete quality control solutions for pharmaceutical, foodstuff, cosmetics, packaging, printing, adhesive, automotive, petrochemistry, environment, biology, new energy, construction, aviation and electronic industries worldwide.
Web Testing: a Critical Step in Web Application Development
Feb 17th
Web testing: a critical step in web application development
We all want our web sites to work well and to make a good impression upon visitors. What can we do to be sure that we provide an excellent web site performance without any unpleasant surprises? The only solution here is to perform web tests where test volume and characteristics of expected traffic are simulated as realistically as possible.
In this article we will explain the importance of web testing and the advantages you get using it. Also we’ll look more carefully on three main types of web tests according to the tasks they perform.
Business view on web testing
Web site response time should be less than 10 seconds. This is shown by the following graph, taken from the “Load time – the three important limits” article by Jakob Nielsen:
Poor performance on the Internet leads to dissatisfied users, and dissatisfied users may leave a web site and never return again. Thus, load testing implemented beforehand and corresponding measures taken to make you response time less than 10 seconds can prevent these losses.
Company expenditures on a Web site’s support should be adequate to site’s expected traffic. Ideally, you want to spend enough, and no more, allocating resources where they’ll generate the most benefit. For example, you don’t need to upgrade your Web servers if customers experience most delays in the database server or in some scripts. Thus, to maximize your ROI, you should determine when and how to upgrade IT infrastructure. Web testing can measure performance of your web system and its components allowing you to invest with maximum effectiveness.
Technical view on web testing
Complete web testing of a system before going live is the primary step to get assured of an entire web application’s ability to work properly. It can help address such issues like readiness of your web server for the traffic you are expecting and for the increasing number of users, the ability to survive a massive spike in user traffic, your server hardware sufficiency and so on. After performing web tests you’ll be able to find bottlenecks in your systems before they happen in a production environment.
Neglecting performance problems can lead not only to poor end-user experience, but even application crashes. Many bugs can only be found when the system is placed under load. You won’t be able to discover these bugs by testing the system as a single user during the development process. That is why testing of web systems under load is really important.
Performance, load and stress testing – what’s the difference?
Performance testing is the process of identifying how an application responds to a specified set of conditions.
Load testing is a type of testing where realistic (or hyper-realistic) workloads are specified and simulated. The load testing goal is to determine the maximum time (from a usability and customer experience standpoint, not a technical one) it should take for all page elements to load.
The goal of stress testing is to estimate the maximum load that your web server can support. The point is that most performance issues arise only when the server is stressed with a high user load, because many important bugs simply do not surface unless you deal with very large entities such as thousands of users, thousands of mail server mailboxes, multi-gigabyte tables in databases, deep file/directory hierarchies on file systems, etc.
Performance Testing Load Testing Stress Testing
Purpose
You do performance testing in order to find any bottlenecks in your web-application code and eliminate them.
You do load testing in order to verify that your application can meet your desired performance objectives with a comfort for the users.
You do stress testing in order to observe how the system reacts to failure and how to improve it.
How it works?
Performance tests verify each part of the web server or the web application to discover how to optimize them. Most often this is done by testing various implementations of single web pages/scripts to check what version of the code is the fastest. All the activities take a white-box approach – the system is inspected and monitored “from the inside out” and from a variety of angles.
Load testing uses the best estimate of the traffic your website needs to support to guarantee that the maximum number of users will see your web pages and feel comfortable with your site. This is done by running regression tests against the system at a maximum load.
Stress tests simulate “brute force” attacks that apply excessive load to your Web server. “Real world” situations like this can be created by a massive spike of users – caused by a large referrer (imagine your website being mentioned on national TV…).
When to perform?
The best time to execute performance testing is at the earliest opportunity. Performance test at an early stage provides opportunity to prevent serious performance problems and to find bottlenecks before load testing.
Run load testing just after your eliminated the bottlenecks in your code (after performance testing).
It is recommended that a stress test is performed after a load test phase.
Web testing software – how does it work?
There is a wide range of web testing software which have different features and, of course, different price. Choosing a tool most suitable for one’s business often becomes rather difficult. You should consider what you need exactly to spend your money most effectively. When choosing a web testing tool you’d better try to find one that simulates real load conditions more effective than others do. This approach is much more efficient than producing stress test by sending many identical requests to the server. Usually it’s performed by simulating many different users coming from different IP addresses, each with their own parameters: cookies, input data for various page forms, name and password, connection speed and their own specific path through the site. Let’s take a closer look at some features of web testing tools.
Virtual users and profiles
Web testing software emulates the activity of human users with the help of virtual users. Each virtual user emulates the actions of a single real user. A test run can contain hundreds or even thousands of virtual users to let you emulate the load experienced by a web site when the corresponding number of real users browse it. User profiles describe the behavior of virtual users. This allows to emulate a real workload on a tested server. A workload (or test volume) is the total amount of activity placed on the tested web application. It consists of a certain number of virtual users who process a defined set of transactions in a specified time period. For each user profile, you should specify web pages that will be requested (visited). It can be easily done with the help of recording tool.
Recording tool
Usually web testing software lets you create profiles for virtual users by recording your own actions when you browse the web site. You should perform all the steps of the virtual user for which you create the profile. The system then reproduces these steps during test run. All you have to do is to make some requests by clicking on necessary links like you do when surfing web pages in your browser. You can use this scenario later as many times as you want.
Flexible load level definition
You can either choose a constant load level for the whole test time, or increase the load after the specified time intervals during the test, or periodically increase/decrease the number of users by introducing 2 phases changing one another: one with a low load level and another with a high load level. You also specify the duration of test run and the number of virtual users for each profile. Overall load significantly depends on the type of users, so you can change it by introducing new types of users making them “slow” or “fast”.
Test reports and graphs
Summary reports and graphs are useful to determine the presence of any problems in the overall performance. It’s better to choose a web testing tool that provides you with the highest level of detail in report. When using a good web testing application you can check through more detailed reports down to the level of separate user profiles and pages to find specific bottlenecks. All reports are usually created in HTML format, so you can easily share them with your colleagues. Some tools also support saving test results in MS Excel compatible format.
Conclusion
Web testing software gives you an assurance of reliability and stable work of your web sites or applications. Of course, you always can neglect purchasing testing software and hope that everything would be ok. But in that case nobody guarantees that one day your web site won’t crash. So, web testing tool is a worthy purchase, isn’t it?