If you are trying to automate your web application using selenium, a few things which you should know before you start the quest. For most of the selenium commands a “target” is required which is laymen terms is the object in your web-application on which you wish to perform a desired action.

Selenium follows a location strategy to locate the element in your application which consists of four modes:

1. Locating by Identifier.

2. Locating by XPath/CSS

3. Locating by DOM

4. Location by CSS

You will find detail description of the above locating techniques on the selenium website which are explained well there [http://seleniumhq.org/]. But what not is explained is when to use what and the negatives of each locating strategy.

1. Locating by Identifier: To locate an element using an identifier means that the element must have either an “ID” attribute or “NAME” attribute which should be unique on that page. You may use “type” or “index” property in combination with these identifiers to help selenium uniquely identify the element.

Selenium is right in expecting that if will find an element uniquely by locating using an identifier. The W3-standards also states that the “ID” attribute has to be provided for all elements and it should be unique. But since the HTML has no inherent check on such slip-ups, most of the developers end up doing so in their web applications while developing forgetting that it has to undergo testing also.

So if you know before hand that “Selenium” is the automation tool which suits your web application scenario the best; ensure your developers haven’t made such mistakes.

2. Locating by XPath: To locate an element using an Xpath, is not a very straightforward solution but it helps immensely when you have to use multiple attributes to help locating an element. Xpath is a powerful way of using any attribute by which you would want the tool to find the element and perform action on it. It is useful mostly in the cases when the developers have used same name, ids across the website and didn’t cared for the W3-standards.

But if you can expect your developers to overlook W3 standards for unique IDs you may very well expect them to give you “unclosed” tags in your application! And if your web application has even a single unclosed tag, Selenium won’t be able to detect the element using the “XPath”.

So if the above is the locating method, you expect to find an element ensure you don’t have unclosed tags in the application.

3. Locating by DOM: The last resort to find an element and weakest one to. It is not advisable to use especially if your website undergoes too many changes. And also if the intention of testing your web site is functional/regression.

4. Locating by CSS: If for initial thoughts you are thinking that what if my web-app doesn’t have CSS will this work, then the answer is “Yes”. This locator type works whether you have CSS or don’t have CSS. It is faster than X-Path, and as stated on the Selenium website, experience users like to use the CSS way of locating an element. This also doesn’t works if the DOM is broken, and also is browser depended, as different browsers have different way of handling CSS.

To summarize the above, it is vital to do a W3 standard check on your website so that you can throw all issues back at your developers to fix, before you jump in to testing your website using Selenium. A good starting point is the, W3-validator [http://validator.w3.org/] available freely online. It clearly list down the issues like duplicate ids, unclosed tags and other slip-ups. Selenium is a powerful tool to use to test your web application across operating systems and browsers but if the DOM is broken than none of the locators will work, coz selenium using the browsers java script engine so the results will also depend on which browser you are using for the test.

It is not mentioned clearly on the website www.seleniumhq.org when to use what locator strategy, and it is kind of overwhelming information why they have provided all such locator categories to the users? Couldn’t there have been a simpler and straight forward way of handling elements using just a single locator type which works under any circumstance. I don’t know the answer to this yet, but maybe will have….

Till the next blog happy testing your sites with Selenium.

(Continued from SQL Injection Prevention Techniques (Series-I))

Stored Procedures

Stored procedures have the same effect as the use of prepared statements when implemented safely*. They require the developer to define the SQL code first, and then pass in the parameters. The difference between prepared statements and stored procedures is that the SQL code for a stored procedure is defined and stored in the database itself, and then called from the application. Both of these techniques have the same effects in preventing SQL injection so it’s your choice, which approach makes the most sense for you.

Safe Java Stored Procedure Example

The following code example uses a CallableStatement, Java’s implementation of the stored procedure interface, to execute the same database query. The “sp_getAccountBalance” stored procedure would have to be predefined in the database and implement the same functionality as the query defined above.

String custname = request.getParameter(”customerName “); // This should REALLY be validated
 try {
  CallableStatement cs = connection.prepareCall(”{call sp_getAccountBalance(?)}”);
  cs.setString(1, custname);
  ResultSet results = cs.executeQuery();  
  // … result set handling
 } catch (SQLException se) {   
  // … logging and error handling
 }

Safe VB .NET Stored Procedure Example

The following code example uses a SqlCommand, .NET’s implementation of the stored procedure interface, to execute the same database query. The “sp_getAccountBalance” stored procedure would have to be predefined in the database and implement the same functionality as the query defined above.

 Try
  Dim command As SqlCommand = new SqlCommand(”sp_getAccountBalance”, connection)
  command.CommandType = CommandType.StoredProcedure
  command.Parameters.Add(new SqlParameter(”@CustomerName”, CustomerName.Text))
  Dim reader As SqlDataReader = command.ExecuteReader()
  ‘ …
 Catch se As SqlException
  ‘ error handling
 End Try

There are some additional security and non-security benefits of stored procedures also that are worth considering. One security benefit is that if you make exclusive use of stored procedures for your database, you can restrict all database user accounts to only have access to the stored procedures. This means that database accounts do not have permission to submit dynamic queries to the database, giving you far greater confidence that you do not have any SQL injection vulnerabilities in the applications that access the database. Some non-security benefits include performance benefits (in most situations), and having all the SQL code in one location potentially simplifies maintenance of the code and keeps the SQL code out of the application developers’ hands, leaving it for the database developers to develop and maintain.

Escaping all User Supplied Input

Each DBMS supports a character escaping scheme using which you can escape special characters to indicate to the DBMS that the characters you are providing in the query are intended to be data, and not code. If you escape all user supplied input using the proper escaping scheme for the database you are using, the DBMS will not confuse that input with SQL code written by the developer, thus avoiding any possible SQL injection vulnerabilities.

Additional Defenses

Least Privilege

To minimize the potential damage of a successful SQL injection attack, you should minimize the privileges assigned to every database account in your environment. Do not assign DBA or admin type access rights to your application accounts. We understand that this is easy, and everything just ‘works’ when you do it this way, but it is very dangerous. Start from the ground up to determine what access rights your application accounts require, rather than trying to figure out what access rights you need to take away. Make sure that accounts that only need read access are only granted read access to the tables they need access to. If an account only needs access to portions of a table, consider creating a view that limits access to that portion of the data and assigning the account access to the view instead of the underlying table. Rarely, if ever, grant the create or delete access to database accounts.
If you adopt a policy where you use stored procedures everywhere, and don’t allow application accounts to directly execute their own queries, then restrict those accounts to only be able to execute the stored procedures they need. Don’t grant them any rights directly to the tables in the database.
SQL injection is not the only threat to your database data. Attackers can simply change the parameter values from one of the legal values they are presented with, to a value that is unauthorized for them, but the application itself might be authorized to access. As such, minimizing the privileges granted to your application will reduce the likelihood of such unauthorized access attempts, even when an attacker is not trying to use SQL injection as part of their exploit.

While you are at it, you should minimize the privileges of the operating system account on which the DBMS runs. Don’t run your DBMS as root or system! Most DBMSs run out of the box with a very powerful system account. For example, MySQL runs as system on Windows by default! Change the DBMS’s OS account to something more appropriate, with restricted privileges.

White List Input Validation

It is always recommended to prevent attacks as early as possible in the processing of the user’s (attacker’s) request. Input validation can be used to detect unauthorized input before it is passed to the SQL query. Developers frequently perform black list validation in order to try to detect attack characters and patterns like the ‘ character or the string 1=1, but this is a massively flawed approach as it is typically trivial for an attacker to avoid getting caught by such filters. Moreover, these filters frequently prevent authorized input, like O’Brian, when the ‘ character is being filtered out.
White list validation is appropriate for all input fields provided by the user. White list validation involves defining exactly what IS authorized, and by definition, everything else is not authorized. If it’s well structured data, like dates, social security numbers, zip codes, e-mail addresses, etc. then the developer should be able to define a very strong validation pattern, usually based on regular expressions, for validating such input. If the input field comes from a fixed set of options, like a drop down list or radio buttons, then the input needs to match exactly one of the values offered to the user in the first place. The most difficult fields to validate are so called ‘free text’ fields, like blog entries. However, even those types of fields can be validated to some degree, you can at least exclude all non-printable characters, and define a maximum size for the input field.

Here a set of techniques is mentioned, which provide prevention from SQL Injection, one of the dangerous security vulnerability. This technique is beneficial for the technologies, such as Java, .net, php and so on.

Prevention from SQL Injection requires lot of defensive measures to be taken for your application. The basic defensive measures to be taken are considered as the primary defense that consists of some programming techniques, defined as follows:

Primary defenses:
1. Parameterization(Prepare statement)
2. Stored Procedure
3. Escaping all user supplied input

Although, applying the primary defense techniques, you can secure your application from the basic security vulnerabilities, if you want to secure the application a step ahead, then you need to use extra defensive measures defined as follows:

Extra Defenses:
1. Least Privilege
2. White List Input Validation

Let’s take an example that is unsafe and is vulnerable for SQL Injection.
The following (Java) example is UNSAFE, and would allow an attacker to inject code into the query that would be executed by the database.
 String query = “SELECT account_balance FROM user_data WHERE user_name = “+ request.getParameter(”customerName”);
 
 try {
  Statement statement = connection.createStatement();
  ResultSet results = statement.executeQuery(query);
     }

In the preceding example, the invalidated “customerName” parameter that is simply appended to the query allows an attacker to inject any SQL code they want. Unfortunately, this method for accessing databases is common to be used among programmers.
Considering the preceding example, let’s now discuss the defensive measures that can be used to prevent SQL Injections.

Primary Defenses

Prepared Statement
     Prepared statements ensure that an attacker is not able to change the intent of a query, even if SQL commands are inserted by an attacker. In the safe example below, if an attacker enters the userID of xyz’ or ‘1′=’1, the parameterized query would not be vulnerable and would instead look for a username which literally matched the entire string xyz’ or ‘1′=’1.

Language specific recommendations:
• Java EE – use PreparedStatement() with bind variables
• .NET – use parameterized queries like SqlCommand() or OleDbCommand() with bind variables
• PHP – use PDO with strongly typed parameterized queries (using bindParam())
• Hibernate - use createQuery() with bind variables (called named parameters in Hibernate)

Safe Java Prepared Statement Example
The following code example uses a PreparedStatement, Java’s implementation of a parameterized query, to execute the same database query.
 String custname = request.getParameter(“customerName”); // this should REALLY be validated too

 // perform input validation to detect attacks
 String query = “SELECT account_balance FROM user_data WHERE user_name =?”
 
 PreparedStatement pstmt = connection.prepareStatement(query);
 pstmt.setString(1, custname);
 ResultSet results = pstmt.executeQuery( );

Safe C# .NET Prepared Statement Example
With .NET, it’s even more straightforward. The creation and execution of the query doesn’t change. All you have to do is simply pass the parameters to the query using the Parameters.Add() call as shown here.
 String query =
   “SELECT account_balance FROM user_data WHERE user_name = ?”;
 try {
  OleDbCommand command = new OleDbCommand (query, connection);
  command.Parameters.Add(new OleDbParameter(”customerName”, CustomerName Name.Text));
  OleDbDataReader reader = command.ExecuteReader();
  //
 } catch (OleDbException se) {
  // error handling
 }

Hibernate Query Language (HQL) Prepared Statement (Named Parameters) Examples

 First is an unsafe HQL Statement
 
 Query unsafeHQLQuery = session.createQuery(”from Inventory where productID=’”+userSuppliedParameter+”‘”);
 
 Here is a safe version of the same query using named parameters
 
 Query safeHQLQuery = session.createQuery(”from Inventory where productID=:productid”);
 safeHQLQuery.setParameter(”productid”, userSuppliedParameter);

(to be continued…)

In this series of blog, we will understand what we mean by performance testing of a web service, and with each upcoming series, we will take the various commercial and open source tools available to assist us in this venture. Let us first begin with a very basic question:

What is Web Service?

Web services are an XML based technology that allow applications to communicate with each other, regardless of the environment, by exchanging messages in a standardized format ( XML ) via web interfaces ( SOAP and WSDL APIs ). In simpler terms a Web services is a web enabled API. To read and understand further about web services you can visit, Pallavi’s blog.
Lets us now understand what we mean by the Performance Testing

What is Performance Testing?

Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such as reliability, scalability and interoperability may also be evaluated. Performance testing can verify whether a system or software meets the specifications claimed by its manufacturer or vendor. The process can compare two or more devices or programs in terms of parameters such as speed, data transfer rate, bandwidth, throughput, efficiency or reliability. For further reading, visit Himanshu’s blog.

Why Web Service Needs Performance Testing?

The next question that comes to our mind is why our web services need performance testing. Here I am going to assume that you are either an experienced reader or you had taken my recommendations seriously and visited the blogs mentioned above.
As you can see in the diagram below that, there is common service provider, which is used by all service requestors. These requestors would want assurance that the web service meets acceptable performance criteria when under stress/load. The goal of performance or load testing of a Web service application is to find how the web service scales as the number of clients accessing it increases. Also, to gather the performance and stability information of the server such as throughput, CPU usage, the time taken to get a response from a web method, etc. when there are, say, 5, 50, 200, 500 concurrent users or more.
The service requestors require these statistics to ensure that the provider is meeting the SLA’s set. Whether they can expect a reliable user experience when a web service is used under load, which may be caused due to, increase in the number of users accessing it and under volume stress, which may be caused when large requests and responses are generated.
In addition, the service providers need to ensure that they have enough web server resources where the service is hosted to cater to the expected load and the web service is scalable enough to function rightly under stress.

Web Service Performance Criteria:

There are various performance criteria, which one should consider when testing a web service for performance.

Server Side:

Throughput: It is measured as number of request sent per second.

Latency (Transaction Time): It is the time taken between service request arriving and being serviced per second. We test the response time as we,
a. Increase the size of web service request, b. Increase the number of virtual users

Resource Utilization: We should be able to figure out the resource demand of the web service under various virtual user workloads/requests volume, so that an optimum resource can be provided for the expected load.

Client Side:

Latency: Time taken for service call to return the earliest response bytes, includes network latency.

Throughput: Average bytes flow per unit time including latency.

Conclusion:
Loosely coupled service environments provide unique benefit for the organizations that utilize them but they introduce new challenges. Both the service providers and consumers should be aware of the performance of the service so that they can ensure it meets the SLA’s specified. Many tools and profilers are available to detect and optimize the performance of a web service. In the next series of blogs, I would be taking a few tools and understand how to do performance testing of a web service using the various tools.

(to be continued…)

The Quality Center 10.0 was launched on 14th January 2009 and came into the market finally from Jan 30th, 2009

The new Version 10 is an absolute milestone not only for tests managers. It also makes life easier for Testers, for R&D persons, Test Managers, as well as QC System administrators. The new functions have been anticipated for quite some time and have been implemented in the new product in a well-conceived manner.

•    Editions Of QC 10.0

1.    QC Starter Edition: This Edition is for small teams , managing small teams.

2.    QC Enterprise: Targeted at Mainstream QA Organization that are involved in managing medium to large Releases and are having new capabilities.

3.    QC Premier Edition: Mainly for Large and mature Organization who have either or implemented COE (Center Of Excellence) in their Organization and who are managing enterprise releases.

•    Versioning:-
Initially there were change related problems that the QA people had to face like
1. Frequent changes in Requirements and tests…
2. Repeated copying of Project related assets leads to chaos.
3. There is lack of visibility when priorities change.
For that matter Versioning concept has been introduced in QC 10.0 to improve the chaos, improve the change related problems to Requirements, Tests and Components.
In this manner, the people involved will have latest information related to the Project and there is no need to purchase third party version tools as well as data integrity can be maintained across projects.

•    Baselining:
Capture project related information or assets at key point of time to make it clear what was agreed to and what was delivered. If any problem happens at any particular version of the project baselining helps   you to go to the previous version or rollbacks to previous version of the project assets and also helps in to sign off when a particular milestone is reached.

•    Integrated Dashboard :
Initially in previous version of Quality Center the Dashboard has to be installed separately. It had separate database, complex and hard to use.

Quality Center 10.0 includes a role based Dashboard which enables a QA Manager, R&D Manager, Business analyst and Team Lead to create their personalized Dashboard. This Dashboard is for a single project and replaces the dashboard of previous versions. It is now finally possible, when working on a QC project, to get an overview of all of one’s ongoing projects. These dashboards are freely configurable and can be designated to be personal favorites or publicly accessible. Special Excel reports also enable direct access to the database via SQL. The generated reports can then be graphically processed at the same time by means of VBScript
Helps in Real time progress and status reporting with focus on individual stakeholders in effective release decisions and there is no need to manage Servers.
No linkage between legacy Dashboard and new Dashboard of QC 10.0.

•    Enhanced Risk Based Quality Management with Testing effort:

The QC 10.0 has a strategic test plan that helps to identify the Testing Effort of a particular Requirement of a particular Business Risk. This helps to accurately plan the project deliverables and how much testing effort should be put on Requirement of a particular level.

•    Shared Libraries or Library Management: Sharing tests , requirements and components:– With QC 10.0 a Shared Library can be created for requirements , Tests and Components, can be re-used and distributed

A library represents a collection of entities in a QC project, including their relationships to each other. When dealing with many similar projects, it offers the advantage of not having to repeatedly create entities. Libraries can be imported from project A into project B, compared against each other, or even synchronized. A library also allows one to collect the same entities as in versioning. With Quality Center Premier Edition, Defects are not included, but they can be shared with the new “HP Quality Center Synchronizer” manually among several QC projects.

•    Cross Project Customization

Many organizations have defined standards, such as a uniform defect status field or a standardized priority scale, for their software quality-related areas. However, these fields and lists were often changed or even deleted by QC project administrators. Some companies have even gone to great lengths in using their own programming to define a template that can be distributed to all QC projects, thereby establishing a uniform standard.

Site Administrator now provides a way to link projects with a template. If the template is changed, the data can then be passed on later at the right point in time. This function has been awaited not only by test managers who like to have the same configuration in all their projects, but especially also by the respective operators of QC installations, namely the system administrators. Cross Project Customization is also only available in the Premier Edition.

I wanted to share my understanding of USE CASES and get valuable inputs from you. I found use case very interesting and an easy way to design test cases. The name sounds a bit awkward. Doesn’t it? So then what does it mean?

Use Case is a very effective elicitation technique which can be used throughout SDLC. It describes “who” can do “what” with the system in question. The use case technique is used to capture a system’s behavioral requirements by detailing scenario-driven threads through the functional requirements. It is very effective where multiple user classes are involved. As compared to other approach (Requirement Itemisation) of test case designing from SRS, use case has some advantages:

• Easy to write

• Expressed in language of the user

• Can be visually represented

• Can be used for validation (acceptance testing)

• Tools support

• Ideal for OO development

• Where the system is functionally oriented (different types of users, complex behavioral patterns)

We should not use a Use Case when system has few users and is dominated by Non-Functional requirement. Now let us see the various entities involved in a Use Case.

• Actor: An Actor is someone or something that interacts with the system. Example of actors are People (Users), Devices, Other Systems, Organizations. Actors live outside the boundary of the system being defined and they initiate an action. Types of actors are Primary actor and Supporting actor.

A PRIMARY ACTOR is one who calls on the system to deliver a service. i.e. it has a GOAL with respect to the system. A PRIMARY ACTOR is typically who triggers the use case

Examples: A caller initiating a telephone call, a customer withdrawing money from the ATM

(Note: Use cases help identify Primary actors early in the lifecycle)

A SUPPORTING ACTOR is an external actor that provides a service to the system under design.

Examples: A high-speed printer, a web service

(Note: We can use supporting actors to identify external interfaces and the protocols the system will use. An actor can be primary in one use case and supporting in another).

• A GOAL is:

• What an ACTOR wants to get with the help of the system

• An ACTION is

• what the ACTOR must perform to reach the GOAL

• An INTERACTION is

• a sequence of steps that must be  followed to complete the ACTION

Examples of USE CASEs are:

1. “Purchase Goods” Level: User Goal

1. Visitor enters customer information (name, address, etc.).

2. System retrieves customer’s profile information, and presents product search and selection mechanisms.

3. Visitor selects products until satisfied. After each selection, system adds the selected product to the customer’s shopping cart and presents the ongoing running total of products selected.

4. Visitor selects to purchase the selected items.

5. System presents contents of shopping cart and requests customer’s payment information.

6. Visitor enters manner of payment and other payment details.

7. System presents visitor with final sums, charges credit card, and delivers packing list to the shipping department.

2. “Book flight!” referencing “Find a flight-”

“Book flight!” Level: User Goal

1. The customer contacts the travel agency and requests a flight.

2. The customer finds a flight-.

3. The system presents the travel options to the customer.

4. The customer selects a flight.

5. The system builds and reserves flight itinerary for the customer.

7. The customer provides a credit card number.

8. The system charges the flight against it and issues the ticket.

“Find a flight-” Level: Subfunction

1. The customer requests to find a flight.

2. The travel agent captures the customer’s trip origin and destination.

3. The travel agent looks up the airport codes for the origin and destination.

4. The travel agent captures the preferred departure times for the customer.

5. The travel agent captures the customer’s preferred class of service.

6. The travel agent confirms that the customer’s preferences are correct.

7. The system gets from the airline reservation system and presents a list of the available flights that match the customer’s preferences.

To summarise, Use Cases are very helpful in deriving test cases from specification. Some of the good practices for writing Use Cases are:

• Make the use cases clear, short and easy to read: Use active voice, present tense, make sure actor and actor’s intent are visible in each step.

• Every Use case has two possible endings: Success and Failure / alternate courses. Don’t forget to gather both.

• Create a list of primary actors and their goals (actor-goal list).

• Restrict use cases to capture system behavior….use cases are not suitable for other type of requirements.

• Use a common template.

Borland silk test is a leading tool for functional and regression, cross platform and localization testing. It can test application based on vast set of technologies like Java, .Net , web , active x, adobe Flex ,client server  etc. Silk Test’s powerful testing framework enables high reusability of test scripts across test projects, building and maintaining regression testing suites and thus helps in expanding test coverage and optimize application quality.

Silk Test Features

A single, automated functional testing tool tests a broad range of enterprise technologies and environments—without costly connectors, adaptors or add-ins
•    A wizard-driven toolbar workflow enables novice users to develop useful tests quickly
•    Robust, resilient tests are supported by a flexible, object-oriented, fourth-generation scripting language called 4Test is designed expressly for automated testing
•    Silk Test also offers functionality of test case management, test planning, data base function, date time functions etc. to make your automation more effective.
•    Silk Test offers many features such as basic workflow for recording tests, workflow for linking a single test case to test data values stored in external tables and code completion in the SilkTest IDE .
•    Unicode support allows localization testing of global applications on multiple platforms with a single script
•    A built-in, customizable error-recovery system returns systems to pre-failure state and resumes testing, unlike other tools that leave systems unstable with unexecuted tests
•    TrueLog based visual reports simplify problem diagnosis when you find defects
•    Integration with Borland’s software test management tool delivers enhanced management capabilities that support a mature and complete functional and regression testing process
•    Built-in support for testing Adobe Flex applications using Internet Explorer, Firefox, the Standalone Flex Player, and Adobe AIR
•    Animated Run Mode to provide ‘slow motion’ playback of test scripts

Architecture

Silk Test consists of two distinct components that execute in separate
processes:
• The Silk Test Host software
• The Silk Test Agent software

The SilkTest Host

The host software is the SilkTest component you use to develop, edit, compile, run and debug your test scripts and testplans. The machine that runs this component is often referred to as the host machine.

The SilkTest Agent

The SilkTest Agent is the component of SilkTest that interacts with the GUI of your application. The Agent translates the commands in your 4Test scripts into GUI specific commands, driving and monitoring the application you are testing. The Agent can run locally on the same machine on which the Host is running or, in a networked environment, any number of Agents can run on remote   machines. In a networked environment, the machine that runs the Agent is often referred to as the remote machine.

How SilkTest  Works and records user actions

Applications are composed of graphical user interface (GUI) objects such as windows, menus and buttons that a user manipulates using a keyboard and a mouse to initiate application operations. Silk Test interprets these objects and recognizes them based on the class, properties and methods that uniquely identify them. During testing, Silk Test interacts with the objects to submit operations to the application automatically, simulating the actions of a user, and then verifies the results of each operation. The simulated user, Silk Test, is said to be driving the application.

Before you begin creating and running test scripts, you create a repository of information about your application to be used by Silk Test. This repository includes descriptions of the GUI objects that comprise your application.Based on the properties and methods Silk Test associates with these objects, Silk Test can recognize the actions performed on them and intelligently record those actions into your test script using the 4Test language.

Test automation is normally developed in SilkTest host using either record/playback or by manual scripting using 4Test. Normally, test cases for SilkTest are developed in IDE (Integrated Development Environment) provided by SilkTest host software. Executing automated test suite is the responsibility of SilkTest agents. SilkTest even supports parallel execution of test cases with the help of these agents. SilkTest host can communicate with these agents residing on multiple machines and execute automated test cases on multiple machines simultaneously. Example , if you need to test your application on Windows 2000, Windows XP and Windows 2000 SP 2, you can have SilkTest agent installed on these machines and run automated test suite in parallel on all these machines at once. Feature like this, saves precious execution time for the testers.

In the next series we will discuss the testing process of silk test and will learn to create simple 4test scripts.

(to be continued…)

(Continued from Updates about QTP 10 (I) … )

II. Report improvements

The native QTP report includes several new improvements:

Tracability: Selecting a report-node will automatically focus on the relevant code-line in the script. This may sound like a cool feature at first, but a closer look revels that it only works for actions (not functions); and that in any case, using a custom report function (as most of us do) completely nullifies the feature (since you’re always at the same code line when performing the report).

Exporting: The report now has a two-click export feature, which comes with a built-in ability to export to Word and PDF documents, as well as the ability to export through a custom XSL of your choosing. You can choose between Short and Long formats (corresponding to PShort.XSL and PDetails.XSL) to get a document relevant to your needs. This blessed feature has one major flaw – It has no API support. This means that you cannot export the results automatically at the end of a test run, which is quite a miss, to say the least.

Resource Monitor: QTP can now hook onto the Windows Performance Monitor and present it as part of the test results. You can select several counters to monitor (e.g. GDI objects, memory usage etc.), and the monitor output graph will be available in special tab in the result window. You can set up a “checkpoint” for a counter (e.g. Fail the test if there are more than 500 GDI objects), facilitating a kind of a poor-man’s version of load-testing.

The fact that clicking the graph focuses on the relevant test step (as well as the other way around), provides an effective way to quickly locate problematic actions and resource usage spikes. This feature is well executed, and HP has even went the extra mile and added several unified counters that simplify monitoring the application. However, the fact that you can only monitor one process per test may leave the more advanced users with their own implementation of a resource monitor. 

Native image integration: This is a small, yet long-awaited feature. The ReportEvent command now has a new optional parameter – you can specify a path to a picture file, and it will be attached to the report node of the current event. When used in conjunction with the CaptureBitmap method, this presents a technical, yet revolutionary upgrade to the native QTP report. Finally, users can attach screenshots to their custom report events without any special functions or frameworks.

III. IDE improvements

Intellisense: IDE improvements, and the new intellisense engine in particular, is what got me excited about QTP Atlantis. 

Can create an Excel COM object provided a full intellisense for all its methods and properties, for as many levels as we’d like. Every variable set to this object also presented the same intellisense, and the autocomplete caught every variable we’ve defined or used (yes, there’s autocomplete for variable names!). The autocomplete and intellisense features worked smoothly, and presented no apparent performance issue. It’s still left to be seen how it functions in a real script, with hundreds / thousands code lines.

Tasks and Comment Pane: QTP has a new bottom pane which includes a run-of-the-mill implementation of tasks and comments. Double clicking a comment will take you to the relevant code-line, though strangely enough, you cannot do this with a task (i.e., tasks cannot be linked to specific code lines). It was mentioned that enabling the comments feature for function libraries may sometimes cause performance issues.

Dynamic code-zones: When standing inside a code block like If, While, Do, etc, the IDE will mark the relevant block with blue lines, making it much more easy to make your way inside nested blocks of this sort (somewhat like highlighting left-right bracket pairs). While it will surely make our life easier, a more robust mechanism like collapsible code-regions is still needed.

Custom Toolbars: You can add your own buttons and commands to QTP toolbars and menus. While this does not include inner-QTP macros, you can assign a program / File shortcut to your own button / menu item. It’s nice, but i think it will only gain power once QTP’s inner mechanisms will be bindable to such buttons.

IV. Miscellaneous features

General look and feel: QTP has departed from the old Tab layout, and into the more modern settings-tree layout (similar to Office, EMule, Adobe, and pretty much every other program). It’s nice, but nothing as groundbreaking as the transformation in QTP 9.0.

Bitmap Checkpoint improvements: These include presenting the size of the selected area when choosing to check only a part of an Image, as well as the ability to plug your own custom DLL for comparing images. Another great addition is the ability to see the difference between the expected and actual bitmaps in a separate tab.

API changes: QTP Automation API will receive several upgrades, the most noteworthy of which is the ability to read and write that code of the test you’ve opened. Writing the code will not effect an ongoing run-session (there goes my ambitious try-catch implementation for QTP), but it still opens the door for some creative tweaks and hacks…

Saving a test with all linked resources: For those who’re working with QC, this is a real blessing. Up until QTP 10, copying a QC saved test to your local system was a hellish procedure of changing each and every resource and action reference to your local file-system. QTP and QC Atlantis offer a one-click solution for copying a test and all its resources to your local file-system, and automatically changing all the references accordingly.

Dynamic Action Call: I’ve saved the best for last. QTP 10 presents a new native command – LoadAndRunAction. It allows you to load external actions during the run-session, without having to associate them with your test beforehand. All the run-time debug abilities will be available for these dynamically added actions, so you’re not giving anything up by using this feature. I think it’s a long awaited feature, and a well executed one.

QTP 10 revolves around 3 pivotal features, alongside several minor features (which turned out to be quite revolutionary):

I. QC integration – which (mostly) boils down to Resource Management and Source Control:

Resource Management: Although you could keep saving your resources as attachments (for backward compatibility), you can upgrade to a new, fuller mode of work. This includes a whole new Resource module in QC, and allows for some very neat tricks on Function Libraries, Tests, Object Repositories etc.

It should be noted, though, that other types of files (external excel / XML files, for example), remain as unmanaged attachments.

1. Resources have full meta-data, and have a special view pane – you can view Object-Repositories, data-tables, and function libraries code right from QC.

2. Resources are aware of their dependencies – Who relies on them, and who do they rely on. This enables a very strong warning system – when changing / deleting a resource, you’ll be alerted to the repercussions – namely, which tests, if any, might break. Also, the ability to immediately know who uses a share object repository is very useful, nearly revolutionary.

3. A very neat trick is a live, automatically updated path system – When moving a function library between folders, QC will automatically update all the tests which depend on it, so they will use it at its new location. This makes the once critical problem of hard-path-link a non-issue. Very impressive.

4. A word about the user interface – when opening a QC resource / test from QTP, the file dialog shows the items with large, crisp icons, very similar to Word’s save dialog. Everything is very clear and intuitive, as is the ability to revert back to saving / opening a test from the File-System.

5. And what about your existing projects? Well, when upgrading to QC 10, a wizard will automatically transform all you unmanaged attachments to managed resources (if you’d like it to).

Source Control: This includes a very rich line of features which are very well executed, and effectively allow you to manage a QTP project as any other code project:

1. First, the basics – QTP and QC 10 introduce a new Check-in/Check-out ability. It works similar to what you’d expect – a checked out item will be locked to all other users, and you can immediately know an item’s status by looking at its icon (green/red locks).

2. An interesting twist regards manner in which a test / resource is locked – it’s at the user level (not the local machine level). This means that if you log into QC from a different machine, you’ll have access to all your checked-out items, even if they were originally checked-out on a different local machine. The ability is implemented very well, both from QTP’s end, as well as from QC’s end.

A major enabler for source control is the new versioning feature of QC. It manifests both with a kind of instant versioning for a single resource, and with a project-wide “base-line version”, which allows you to revert your entire test framework to a previous version. Both types of versioning are supported by a surprisingly robust comparison mechanism. You can select two versions of a resource / test, a see a very detailed comparison of their respective changes. For function libraries this amounts to a “simple” text comparison, but this feature truly shines in full test comparisons.

It will present changes in the different actions and their resources (data-table, object repositories, inner code), as well as in the global test-settings, associated function libraries, recovery scenarios, and pretty much anything you could think of. The ability to drill-down into the data is very impressive; and the effective, concise manner in which the data is presented in the top level view is downright unbelievable. A nice touch is a small screen capture of the change, in case you don’t remember what “Run all rows –>Changed into-> Run a single iteration only” means (for example).

Now to the versioning mechanism itself: Whenever you check and item in, a new “version” will be created, and you’ll be able to revert back to it with ease. The snapshots are visible both from QC and QTP, and you can very easily choose which one to open. This allows you a kind of an instant undo to a single file which worked in the past, but is broken in the present.

The second mechanism presents the ability to select several folders, and create a full blown “base-line version” of them and everything they relate to. Defects, inner-connections, tests, history data, resources – all these and more will be “frozen” and preserved as a base-line. You can then choose to revert back to an old baseline, and truly regain all the abilities that were present at that time. As all the resources, attachments tests and reports will be restored, you don’t have to worry about forgetting anything, or leaving some minor resource at the wrong version. This is versioning with a vengeance – it allows you to track the AUT’s versions with your own automation versions, enabling, among other things, running automation efforts on several AUT versions at once.

For conclusion – The new abilities inherit in the connection of QTP and QC Atlantis are (or at least seem to be) revolutionary. At last, QTP projects can be natively managed as code projects; and some of the supporting infrastructure is surprisingly robust and useful.

(to be continued…)

Introduction: This document has been prepared in order to resolve the issue that might occur during Invocation of RPT- Agent Controller process on Linux Machine.

Requirement: Our requirement was to generate the load from Linux machine (client) on the application server

For this, we have installed Load Generating tool (Rational Performance Tool version - 7.0.2) on window machine (OS: Window XP 2000 Profession SP-2) and RPT-Agent controller Process (version-7.0.2.1) on Linux machine (OS-Red Hat Enterprise Linux AS release 4-Nahant).

Below are enlisted some of the issues which were encountered during the load distribution through load generating machine (RPT) to Linux machine.

On executing the performance schedule we were getting the following error “Connection failed on host 172.23.244.207”.

————————————————————

 Security Message

Connection failed on host 172.23.244.207

Reason:
IWAT0284E The agent controller is not available on host 172.23.244.207
Make sure that:
*the agent controller is installed.
*the agent controller is configured to communicate with your machine
*you have the correct host name and port number for the agent controller.

————————————————————–

Possible reason: The above error might have occurred due to Agent Controller is not installed or is not running on Linux machine.
Since, on Linux machine Agent Controller process (RAServer) process is not started automatically. So, we have to start this process manually.

Starting and Stopping Agent Controller on Linux machine:
• To start the Agent Controller process (RAServer) on Linux machine, move to the Installation location’s bin directory (for e.g. /opt/IBM/AgentController/bin). Then execute the following command
./RAStart.sh

• To stop the Agent Controller process (RAServer) on Linux machine, move to the Installation location’s bin directory ( for e.g. /opt/IBM/AgentController/bin) and then execute the following command
 ./RAStop.sh

On trying to start the Agent Controller process on Linux machine we may get the following Errors. (Below section describe the Error description, reason and resolution for the same)

Error:
1) Starting Agent Controller
“RAServer: error while loading shared libraries: libstdc++-libc6.2-2.so.3: cannot open shared object file: Error 40 No such file or directory.
RAServer failed to start.” Error

Possible Reason: Since the Agent Controller is compiled using libstdc++-libc6.2-2.so.3 shared library. Ensure that this shared library exists under the /usr/lib directory. If it does not exist, then you have to install the RPM package compat-libstdc++ that comes with the operating system installation media.
Note: - To make sure that libstdc++-libc6.2-2.so.3 shared library is available in the /usr/lib directory:
Move to the /usr/lib directory and execute the following command at the Shell prompt.
 # ls –l libstdc*

Resolution:
The solution is to install the standard C++ compatibility libraries in order to satisfy this library dependency. The version of Linux on the client machine will determine what RPM or software package needs to be installed.
In our case, since we are using Red Hat Enterprise Linux As Release 4 (Nahant) Operating System on Linux machine, we need to install compat-libstdc++-296-2.96.132.7.2.i386.rpm package that is located on the Red Hat 4.0 Installation Disc 3.
Note: For more which rpm package required installing, browse the following link
http://seer.entsupport.symantec.com/docs/267077.htm

We can also download required rpm package from the following link
http://rpmfind.net/linux/rpm2html/search.php?query=libstdc%2B%2B-libc6.2-2.so.3&submit=Search
http://rpmfind.net/linux/RPM.

How to Install Required RPM Package:
1) Insert the required disc in CD-ROM and change the directory Red Hat/RPMS from Shell command.
  cd media/CDROM/Red Hat/RPMS/
2) Enter the following command and execute
rpm –ivh compat-libstdc++-296-2.96-132.7.2.i386.rpm
If installation is successful, you see the following message:
Preparing…   ########################################### [100%]
1:compat-libstdc++-296   ################################## [100%]
RPM prints out the name of the package and then prints a succession of hash marks as the package is installed as a progress meter.
Note: For more information on RPM package browse the following link
http://www.faqs.org/docs/securing/chap3sec20.html

Now we can start the Agent Controller Process (RAServer) on Linux machine. Following message should be displayed on successfully start the Agent Controller Process.

Starting Agent Controller
RAServer Started Successfully
RPM prints out the name of the package and then prints a succession

2) “RAServer failed to Start” Error
Possible Reason: This failure is usually caused when TCP/IP port 1002 is not free. Agent Controller listens on this port by default. Agent controller was just stopped and restarted before the port could be released.
• If Agent Controller failed to start. You can start it as follows:
If port 10002 is being used by another process, you can change the port number by editing the serviceconfig.xml file. Serviceconfig.xml file is located in Installation Location Config’s directory /opt/IBM/AgentController/Config/

• If Agent Controller was just stopped, wait a few minutes and try to start it again.