PTLIB:
Evaluation Criteria for Parallel Debuggers
and Performance Analyzers
- Performance
- Performance encompasses accuracy, efficiency, and scalability.
- Accuracy
- A performance monitoring tool is accurate if it doesn't cause too
great a change in the behavior and timing of the program it is monitoring.
- Efficiency
- The software runs fast enough, in that slow speed does not make it
an ineffective tool.
(NHSE Software Review Policy)
- Scalability
- A parallel tool is scalable if its overhead grows in a reasonable
manner with increases in system and problem sizes. In some cases,
linear growth may not be acceptable.
- Capabilities
- The tool has adequate functionality to effectively accomplish its
intended tasks.
- Versatility
- Versatility encompasses heterogeneity, interoperability, portability,
and extensibility.
- Heterogeneity
- A heterogeneous tool can simultaneously
be invoked on and/or have its components running
on all platforms in a heterogeneous system.
- Interoperability
- A parallel tool is interoperable if it is designed based on open interfaces
and if it conforms to applicable standards.
- Portability
- A parallel tool is portable if it works on different parallel platforms
and if platform dependencies have been isolated to specific parts of the
code.
- Extensibility
- A performance analysis tool is extensible if new analysis methods and
views can be added easily.
- Maturity
- Maturity includes robustness, level of support, and size of user
base.
- Robustness
- A parallel tool is robust if it handles error conditions without crashing
and by reporting them and recovering from them appropriately.
- Ease of use
- The software has an understandable user interface and is easy to
use at the level of a typical NHSE client.
(NHSE Software Review Policy)
Last updated October 11, 1996
ptlib_maintainers@nhse.org