By Ravinder Singroha

In this series of blog, we will understand what we mean by performance testing of a web service, and with each upcoming series, we will take the various commercial and open source tools available to assist us in this venture. Let us first begin with a very basic question:

What is Web Service?

Web services are an XML based technology that allow applications to communicate with each other, regardless of the environment, by exchanging messages in a standardized format ( XML ) via web interfaces ( SOAP and WSDL APIs ). In simpler terms a Web services is a web enabled API. To read and understand further about web services you can visit, Pallavi’s blog.
Lets us now understand what we mean by the Performance Testing

What is Performance Testing?

Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such as reliability, scalability and interoperability may also be evaluated. Performance testing can verify whether a system or software meets the specifications claimed by its manufacturer or vendor. The process can compare two or more devices or programs in terms of parameters such as speed, data transfer rate, bandwidth, throughput, efficiency or reliability. For further reading, visit Himanshu’s blog.

Why Web Service Needs Performance Testing?

The next question that comes to our mind is why our web services need performance testing. Here I am going to assume that you are either an experienced reader or you had taken my recommendations seriously and visited the blogs mentioned above.
As you can see in the diagram below that, there is common service provider, which is used by all service requestors. These requestors would want assurance that the web service meets acceptable performance criteria when under stress/load. The goal of performance or load testing of a Web service application is to find how the web service scales as the number of clients accessing it increases. Also, to gather the performance and stability information of the server such as throughput, CPU usage, the time taken to get a response from a web method, etc. when there are, say, 5, 50, 200, 500 concurrent users or more.
The service requestors require these statistics to ensure that the provider is meeting the SLA’s set. Whether they can expect a reliable user experience when a web service is used under load, which may be caused due to, increase in the number of users accessing it and under volume stress, which may be caused when large requests and responses are generated.
In addition, the service providers need to ensure that they have enough web server resources where the service is hosted to cater to the expected load and the web service is scalable enough to function rightly under stress.

Web Service Performance Criteria:

There are various performance criteria, which one should consider when testing a web service for performance.

Server Side:

Throughput: It is measured as number of request sent per second.

Latency (Transaction Time): It is the time taken between service request arriving and being serviced per second. We test the response time as we,
a. Increase the size of web service request, b. Increase the number of virtual users

Resource Utilization: We should be able to figure out the resource demand of the web service under various virtual user workloads/requests volume, so that an optimum resource can be provided for the expected load.

Client Side:

Latency: Time taken for service call to return the earliest response bytes, includes network latency.

Throughput: Average bytes flow per unit time including latency.

Conclusion:
Loosely coupled service environments provide unique benefit for the organizations that utilize them but they introduce new challenges. Both the service providers and consumers should be aware of the performance of the service so that they can ensure it meets the SLA’s specified. Many tools and profilers are available to detect and optimize the performance of a web service. In the next series of blogs, I would be taking a few tools and understand how to do performance testing of a web service using the various tools.

(to be continued…)

By Upendra Patel

Selenium and Watir are the best front end based web application testing frameworks (Open Source) at present. Both selenium and watir supports test suites in Java or Ruby and you can replay them subsequently.

Selenium has a firefox based recorder tool (Selenium IDE), that can capture browser actions and create test cases in multiple languages like Java and Ruby.

The Selenium IDE can show you Java/HTML/Perl/C#//PHP/Python/Ruby code for the recorded test case. Copy that and run as java test case with selenium apis in classpath.

In Selenium, after requests submit is done by the script, it doesn’t know how to detect the arrival of response. So, technically it waits for a specified amount of time and then assumes the page has been loaded with response content. In test suites, it comes like this - selenium.waitForPageToLoad(”30000″); defining this wait period becomes the script writer’s (tester) responsibility.

If you have a process to run test cases on multiple environments, it can worsen the scenario quickly.
Watir, on the other hand works embedded within the browser. This means, it can detect when the server response comes back. Hence the tester can focus on what the response value should be rather figuring out what the approximate wait period on each page should be. Watir is also considered to be better for Ajax based testing.

On the other side, Watir doesn’t have a mature recorder tool. Watir is a Ruby framework therefore knowledge of Ruby is mandatory for using Watir.

Selenium uses a unique mechanism for driving browsers. Selenium’s automation engine is written in JavaScript and runs inside a browser. The engine, called a browser bot, is embedded in a page that accesses the application under test in a separate frame.

Watir is a Ruby library that wraps the COM interface to Internet Explorer. COM is a long-standing. This allows access to the Document Object Model (DOM) so it doesn’t matter where on the page an object exists, what matters is how you identify it (id, name, title, etc).

A brief Comparison between Watir and Selenium

By Pankaj Goel

Visual Studio Team System testing tools provides several test types that can be used for specific software testing purposes. Section below describes those test types.

Unit Test
Deals with generating and authoring unit tests, including ASP.NET unit tests and data-driven unit tests.

Web Tests
Works with Integrated Web Test Recorder to record the actions you perform while you browse a Web site.

Load Tests
Refers to Simulating load of multiple users accessing the web application simultaneously

Manual Tests
Manual test steps can be written in text format and can be referred by functional Testing Team

Generic Tests
Wrap an existing test, program, or third-party tool as a generic test.

Ordered Tests
Refers to Ordered test that would execute in a specific sequence.

About Web Test

Web Test is a tool with VSTS Test edition that is used for automating functional tests. The scripts for Web test can be used for performance testing as well.
A Web test simulates how an end user might interact with a Web application. Web test is created by recording HTTP requests using the Web Test Recorder in a browser session. However Web tests can also be building manually using the Web Test Editor.

In contrast to most of the test automation tools on the market, Web Test records HTTP transactions, not Windows API calls. For instance when we click on a [Submit] button, what’s recorded is not the click but the transaction going to the server as a result.

The language used with this tool is C# or VB dot net

We can
•    Parameterize recorded transactions with test data,
•    Add or remove transactions,
•    Add validation points (called Validation Rules in VSTS lingo)

We can not
•    Perform GUI validation with this tool
•    Perform Database Validation using this tool
•    Use the flow control and looping constructs that are provided by more advanced programming languages.

Pros of Web test

•    Script is more reliable in terms of being oblivious to minor changes in the application user interface
•    VSTS scripts can also detect defects that would be missed with GUI tools.

Cons of Web test

•    Web test is not a classical test automation tool. It does not work on GUI elements
•    Learning curve is significantly higher. You require Coders to do the effective test automation

By Pankaj Goel

While working on performance testing assignments, many a times, I came across situation where following error was reported while applying load on the system
“The Maximum Connection Limit Has Been Reached”
On further analysis, I found that the error was being returned by the database (MS SQL Server in this case). Here is in this post, I have tried to detail what all to do when you get this error
So for MS SQL Server (2000 or 2005), if you have identified that the number of user connections is high, you can control the number of user connections by doing one of the following:
•    Enable Microsoft SQL Server to automatically add user connections as needed. Do this through the Microsoft SQL Server SQL Server Properties window by setting the user connections configuration option to 0. If you specify a value other than 0, Microsoft SQL Server allocates memory for those connections at startup, which burns up portions of the memory pool.

•    Specify a maximum number of user connections. Set this amount to the expected maximum number of concurrent users. Microsoft SQL Server rejects new login requests after reaching the maximum number of user connections when this value is set. After changing the maximum number of user connections, you must restart Microsoft SQL Server.

To re-configure the maximum number of user connection, In SQL Server 2000, you can use the sp_configure stored procedure. When you install SQL Server, the default value for user connections is 0 (32767 concurrent connections).
a)    Start SQL server from command prompt
sqlservr.exe -c -f
b)    Issue the following command
sp_configure ‘user connections’, 0
go
reconfigure with override
This code sets user connections back to the default setting.
c)    Restart SQL server normally

For further details about the optimum SQL server settings, resources @ the following link contain some good information
http://support.microsoft.com/kb/319942/EN-US/
http://support.microsoft.com/kb/224587/

By Pankaj Goel

Performance tuning tips for IIS 6.0
The IIS server tuning is slightly involved in terms of understanding the performance critical parameters and tuning them to meet the performance specifications. Some of the ways to improve performance hosted on dot net platform are

a) Recycling Worker process
b) Reducing thread contention
c) Kernel Mode caching

a) Recycling Worker Processes
If a Web application contains code that causes problems, and you cannot easily rewrite the code, it might be useful to limit the extent of the problems by periodically recycling the worker process that services the application. You can accomplish this by using what is known as Worker Process Recycling. Worker process recycling is the replacing of the instance of the application in memory. IIS 6.0 can automatically recycle worker processes by restarting the worker process, or worker processes, that are assigned to an application pool. This helps keep problematic applications running smoothly, and minimizes problems such as memory leaks. You can trigger the recycling of the worker processes assigned to an application pool by using worker process recycling methods that are based on elapsed time, the number of Hypertext Transfer Protocol (HTTP) requests, a set time of day, and two kinds of memory consumption, in addition to recycling on demand.
To configure all the above settings, go to the Properties window of the application pool in which your Web application is running using the IIS manager. Using the Recycling, Performance, and Heal