NeXtMidas Performance Testing

Introduction

TBD

Performance testing is a much more involved process than the standard regression tests. This is due to the fact that the performance of an application is greatly affected by external factors that must be accounted for and because specialized third-party tools are required to accurately measure the performance. For these reasons, the running of performance tests should be restricted to people with a good working knowledge of NeXtMidas, performance measurement tools used, and an expert knowledge of Java.

Third-Party Tools Used

TBD

Wireshark (formerly Ethereal)
Wireshark is a network protocol analyzer that can be used to analyze and measure network traffic. Please see http://www.wireshark.org/ for more details.
NetBeans Profiler Module
The Profiler Module for NetBeans can be used to measure CPU and memory usage. Please see the "NetBeans Profilers" section in the NeXtMidas Training and http://profiler.netbeans.org/ for more details.

[July 2005: At this time the NetBeans Profiler Module is incorrectly counting Thread.sleep(..) time as CPU time. This substantially biases the methods in NeXtMidas that wait for piped data to become available and renders the CPU usage measurements inaccurate. The NeXtMidas developers have submitted this as an issue to the NetBeans project (see NetBeans Issue 57639 for details).]

Measuring Execution Speed

TBD

Measuring Memory Usage

TBD

Measuring X11 Traffic for Display Forwarding

A number of NeXtMidas users operate in environments where the core of NeXtMidas runs on a server (called a "display server") with the X11 displays forwarded to a remote system using xhost and setting the environment variable DISPLAY to point to a remote system. This is a special case situation where some of the normal optimizations in Java (specifically those that take advantage of the system's graphics acceleration hardware) can actually degrade overall performance because they cause an increase in X11 network traffic.

Due to this situation there are a number of tests that are run to measure the X11 traffic:

Test Name Graphical Components Tested Test Duration
%test_list_perf LIST 60 seconds
??? STATIC PLOT (static plot) ?? seconds
??? REALTIME PLOT (real-time 1D plot) ?? seconds
??? RTRASTER PLOT (real-time 2D raster) ?? seconds
??? PANEL, GCONTROL ?? seconds
??? PANEL, GCONTROL, LIST ?? seconds
??? CONTROL PANEL, GCONTROL, PLOT
(one static plot, two blank plots)
?? seconds
??? NOTABS PANEL, GCONTROL, PLOT
(one static plot, two real-time plots)
?? seconds
??? RTTABS PANEL, GCONTROL, PLOT
(visible tab with static plot, two hidden tabs with real-time plots)
?? seconds
??? BLANKTABS PANEL, GCONTROL, PLOT
(visible tab with static plot, two hidden tabs with blank plots)
?? seconds

The test macros for the above tests are located in the TEST area of the SYS option tree. The tests are usually performed using three systems (see diagram below):

Display Server
This is the server that NeXtMidas usually runs on.
Client
This is one of the systems to which the Display Server typically forwards its displays. Under normal operating conditions the display server would have many clients receiving displays at any given time.
Developer Workstation
An independent system used by the person running the tests. This is necessary because the developer is unlikely to have direct physical access to the display server. This system is not part of the normal operating configuration and is used only for the purposes of testing.
+-----------------------+  X11 Graphical Displays  +-----------------------+
| DISPLAY SERVER        |------------------------->| CLIENT                |
|  - Runs NeXtMidas     |                          |  - Displays the       |
|                       |<-------------------------|    NeXtMidas GUI      |
+-----------------------+   X11 Control Messages   +-----------------------+
           ^
           |
           |                                       +-----------------------+
           |           SSH Connection              | DEVELOPER WORKSTATION |
           +-------------------------------------->|  - Used to log into   |
                     (Used for Testing)            |    display server to  |
                                                   |    run tests          |
                                                   +-----------------------+