What are advantages and disadvantages of using agents in system testing?

Technote (troubleshooting)


Problem(Abstract)

What are advantages and disadvantages of using agents in system testing?

Resolving the problem

QUESTION:
What are advantages and disadvantages of using agents in system testing?

ANSWER:
The purpose of this document is to help you decide whether or not the use of
agents would be a desirable strategy for system testing, using either
functional (GUI) or performance (VU) scripts.

Test Agents are a Rational Software solution for dealing with two distinct
issues:

1. The ability to run large performance tests where the amount of memory
required to run the test would be difficult, if not impossible, to put in the
single system that is running Rational TestManager.

2. The use of agents to run more than one GUI script at a time during a
TestManager playback.

This document will attempt to give the reasons, both for and against, using
Test Agents for playing back scripts.

Running agents when doing performance testing (using VU scripts):

Generally, we would recommend running the tests on the local machine if
possible. There are several reasons for this:.

1. Installing and maintaining Test Agent Software on Test Agent machines. This
is a relatively simple step to go through as the agent software is available on
both cd and via download from the Rational Website (www.rational.com.) and any
available patches which are also downloadable. Though this is simple, it is
still separate software to be maintained. In addition, if you are using Unix
agents, these require a separate software version (and download) which
increases the complexity of maintaining the agent software

2. Networking issues - Having agents mean that you will incur additional
network traffic. The data generated by the agents when the test has been run
will have to be shipped over to the master machine to be used in generated
reports, this will not only increase the amount of traffic that will be sent
over the local network, but will also cause the test to run longer. This
network traffic is in addition to the traffic generated by the test itself to
test the server in question. There is also the additional problem of possible
loss of connectivity to the agents, which would cause a test to fail.

3. Lack of Control of Test Machines - Typically (though not by rule or in all
cases), a test agent machine will not be under the immediate control of the
tester. What this means is that it is much more likely for the test agent
process to be stopped or disrupted before or during a test. This is easy to do
because there is no outward sign a test is running during testing.

Reasons for running with agents with performance tests:

1. The only realistic reason for running with agents on a VU test is that the
there is not enough memory on the TestManager system to run the entire test. A
guideline to use is that is will take between 4 and 6 megs of memory per
virtual user. This is just a guideline. The complete script is loaded down for
each used, so if a script is very long it may take more than 6 megs per user.

The use of agents to run more than one GUI script at a time during a
TestManager playback (Distributed Functional Test or DFT)

Reasons against running with agents for functional testing:

The same three reasons listed as negatives against running a performance test
with agents are also valid as negative points when running DFTs. Issue #3, Lack
of Control of Test Machines is not quite as much of a concern because when
running a DFT, the actual application shows up on the screen of the agent
machine so someone coming up on the machine when a test is running would be
more likely to see something running on the test machine. The same issue of
someone stopping the test agent process or service between tests still is valid.

Reasons for running with agents for functional tests:

1. Being able to test multiple windows Operating systems with the same scripts
simultaneously - This feature is helpful when you are doing regression testing
with the same scripts on different Windows Operating systems, Win98, WinNT,
WinXP and Win2000. A tester should expect the script to function in the same
way on each of the different windows platforms; this will show whether or not
your application is Operating System independent.

2. Speed of testing - GUI script require that the actual scripts run on a
windows system. Running with only one system will require that if multiple
scripts are to be run, they be run sequentially. Running with multiple agents
allows these scripts to run simultaneously on multiple windows systems.

3. Clarity of test monitoring - In this scenario, you would have the machine
running TestManager not have any tests run back directly on it. This would
enable the tester to have the Monitoring tools of TestManager not be obscured
by the GUI script play back.

Historical Number

164277615

Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

Rational TestManager
Integrations: IBM

Software version:

2003.06.00

Operating system(s):

All Windows Platforms

Reference #:

1117523

Modified date:

2010-10-27

Translate my page

Machine Translation

Content navigation