LIFETIMEIOSTATS(1)
NAME
LifetimeIOstats - Produce report from I/O File Lifetime
Summary trace records
SYNOPSIS
LifetimeIOstats [ -browserFile browserOut ] tracefile
DESCRIPTION
LifetimeIOstats generates a report of application I/O
activity summarized by file from File Lifetime Summary
trace records in the input SDDF trace file. The necessary
trace event records are produced by the I/O extension to
the Pablo trace library when the File Lifetime Summary
option is enabled.
As LifetimeIOstats is running, it periodically displays
the number of input trace packets (records) processed to
standard error. The report output is directed to standard
out and is 132 characters wide. Several notes regarding
interpretation of reported values appear after the actual
I/O summary information. Those notes are included in a
slightly modified form in the section "THE REPORT" below.
If the input file does not include any File Lifetime Sum-
mary records, the report will contain headings and a mes-
sage that none of the necessary records appear in the
trace file, but no summary information. For the report to
work as intended, there must be File Lifetime Summary
records in the input trace file. If the programmer elects
not to enable the File Lifetime Summaries for the entire
application execution, the report must be interpreted with
that in mind.
OPTIONS
-browserFile browserOut
When specified, this option causes LifetimeIOstats
to produce an SDDF file, browserOut, containing a
subset of the reported information in a format that
can be loaded and viewed with the Pablo Browser
program. See "THE BROWSER DATA" section below for
details on the information written to the SDDF
file.
Note that the Pablo Brower program is currently not
distributed outside the Pablo research group due to
limited portability across platforms.
BEFORE RUNNING THE PROGRAM
The application to be studied must be instrumented with
the I/O extension to the Pablo trace library and call the
function enableLifetimeSummaries() to generate the trace
records needed as input for this program. The
instrumented application is run and one or more trace
files are generated - the number of trace files depends on
the number of processors used to run the application.
If there are multiple trace files, they should be merged
with the MergePabloTraces(1) program to produce a single
trace file for the execution including information from
all processors.
After the trace files have been merged, you should run the
program AdjustTime(1) on the merged trace file if you
linked the application with the the raw clock version of
the Pablo trace library.
THE REPORT
The report generated by LifetimeIOstats is made up of
three major sections. The first section summarizes I/O
activity over the lifetime of files accessed on a per-pro-
cessor basis. The second section provides similar sum-
mary information over all processors and also includes a
statistical analysis of the length of time files were kept
open during execution. The final section reports the sec-
onds spent in any of the traced I/O operations( Read,
Seek, Write, Open, Close ) for each processor, and the
total for all processors. The processors spending the
least and the most time doing I/O are identified with
"Minimum" and "Maximum" labels in the third section of the
report.
Further details on the contents of sections one and two
follow. Unless explicitly noted, the comments apply to
both the "per-processor" and the "all processors" sections
of the report.
The processor number, or all processors, is given. Fol-
lowing that a table containing the headings "Read Count",
"Read Bytes", "Read Time", "Seek Count", "Seek Bytes",
"Seek Time", "Write Count", "Write Bytes", "Write Time",
"File Lifetime", "Opn/Cls Time", "Open Count", and "File-
name" is generated. In section one (per-processor), sum-
mary lines appear in the table for all files opened on the
processor being reported. In section two (all proces-
sors), summary lines appear in the table for all files
opened by any processor that was part of the application
run.
Each file name is given under the heading "Filename". The
number of times each file was opened appears under the
"Open Count" heading. The amount of time spent (in sec-
onds) opening and closing each file is given in the column
"Opn/Cls Time". The total time the file was open during
the execution is listed in the "File Lifetime" column.
Count, Bytes, and Time columns show the event count, bytes
involved, and seconds taken for read, seek, and write
requests. Asynchronous reads and writes (on the Intel
Paragon systems) are included in the read and write
columns, with the reported seconds corresponding to the
duration of the asynchronous call, not to the time
required for the completion of the requested data trans-
fer. The table ends with a summary line for all files
accessed on the processor (or processors if in section
two).
Following the table, the distribution of I/O read, seek,
and write requests is listed. Next, the mean request size
for reads, seeks, and writes is shown.
Bytes/Second values based on individual requests are
reported for reads and writes. These are calculated by
dividing the number of bytes read (written) for all files
by the number of seconds required by the reads (writes).
If your application makes use of asynchronous reads and/or
writes, special care should be taken in interpreting these
numbers since only the time for the asynch I/O request and
not the time for the data transfer is included in the num-
ber of seconds reported.
For section one (per-processor), a set of statistics based
on "Processor Execution Time" is given next. For section
two (all processors), the statistics are given based on
"All Processor Execution Time" and on "Application Execu-
tion Time".
For purposes of this report, "Processor Execution Time"
measures the time from the initialization of the Pablo
Trace library to the time when endIOtrace() is called.
The Pablo Trace library is initialized by the first call
to traceEvent(), countEvent(), or startTimeEvent(); or by
the first call to initialize a trace library extension --
initIOtrace() for example. Provided one of these calls is
made near the beginning of the application execution and
the call to endIOtrace() is made near the end of the
application execution, the "Processor Execution Time"
gives a fairly good measure of the application run time.
Since tracing is often done on multi-processor systems, we
use the term "Processor Execution Time" to refer to the
execution time of an individual processor. We use "All
Processors Execution Time" to refer to the sum of the exe-
cution times of the individual processors. The term
"Application Execution Time" is used to refer to the maxi-
mum of the execution times of the individual processors in
the system. "Application Execution Time" corresponds to
wall clock time -- how long it took the application to run
on the system.
The statistics given show the operation frequency
(operations per second) for reads, seeks, and writes based
on the Execution Time. The operation time as a percentage
of execution time is listed for read, seek, write, and
opens/close operations. The time taken by the
opens/closes can be thought of as overhead since they
accomplish no real work, but are necessary in order to
perform the desired I/O. Finally, a bytes/second figure
based on Execution Time is given for reads and writes.
THE BROWSER DATA
The browser SDDF file contains data on:
o Read Counts per File Open
o Seek Counts per File Open
o Write Counts per File Open
o Read Time per File Open
o Seek Time per File Open
o Write Time per File Open
o Open/Close Time per File Open
o File Lifetimes
o Open/Close Overhead -vs- File Lifetime
KNOWN PROBLEMS
For files accessed in a global mode, the "Seek Bytes" val-
ues will often be incorrect. In particular, on Intel
Paragon systems, these values should not be trusted for
files accessed with an iomode of M_LOG, M_SYNC, M_RECORD,
or M_GLOBAL. The I/O extension to the Pablo instrumenta-
tion library attempts to minimize the overhead incurred in
gathering file pointer information and does not track file
pointer positioning correctly when the activity on one
processor affects the file pointer position on another
processor. An attempt will be made to address this prob-
lem in the next release.
SEE ALSO
AdjustTime(1), FileRegionIOstats(1), IOstats(1),
IOtotalsByPE(1), MergePabloTraces(1),
TimeWindowIOstats(1), runBrowser(1)
Ruth A. Aydt, A User's Guide to Pablo I/O Instrumentation
Ruth A. Aydt, The Pablo Self-Defining Data Format
COPYRIGHT
Copyright 1994, The University of Illinois Board of
Trustees.
AUTHOR
Ruth A. Aydt, University of Illinois
Pablo Environment Oct 16, 1996