23: ilibr librarian

These files have been made freely available by SGS-Thomson and may not be used to generate commercial products without explicit permission and agreed licensing terms OR placed in a public archive or given to third parties without explicit written permission from SGS-Thomson in Bristol.
Tony Debling, SGS-Thomson Microelectronics Ltd, 1000 Aztec West, Almondsbury, Bristol BS12 4SQ, England. June 7, 1995.
Converted to HTML via RTFtoHTML, PERL5 and by hand by Dave Beckett <D.J.Beckett@ukc.ac.uk>.
WARNING: This has been converted from RTF format and may have mistakes compared to the printed version. If you find any in this file, please send them to me, and I will update it.

Dave Beckett <D.J.Beckett@ukc.ac.uk>

This chapter describes the librarian tool ilibr that integrates a group of compiled code files into a single unit that can be referenced by a program. The chapter begins by describing the command line syntax, goes on to describe some aspects of toolset libraries, and ends with some hints about how to build efficient libraries from separate modules.

23.1: Introduction

The librarian builds libraries from one or more separately compiled units supplied as input files. The input files may be any of the following types: The input files for one library must all be of the same type.

The librarian takes a list of compiled files in TCOFF format and integrates them into a single object file that can be used by a program or program module. Each module in the input list becomes a selectively loadable module in the library. Input files can either be specified as a list on the command line or in indirect files.

The library, once built, will contain an index followed by the concatenated modules. The index is generated and sorted by the librarian to facilitate rapid access of the library content by the other tools in the toolset, for example, the linker.

Compiled object files (excluding library files) may be concatenated for convenience before using the librarian. This may prove useful when dealing with a large number of input files.

The operation of the librarian in terms of standard file extensions is shown below.

23.2: Running the librarian

To invoke the librarian use the following command line:
ilibr filenames { options }

filenames is a list of input files separated by spaces.

options is a list of one or more options from Table 8.1.

The number of file names allowed on a command line is system dependent. To avoid overflow, an indirect file may be used or files (other than library files) may be concatenated. It is the user's responsibility to ensure that the concatenation process does not corrupt the modules, for example by omitting to specify that the concatenation is to be done in binary mode.

If no arguments are given on the command line a help page is displayed giving the command syntax.

    Option      Description
  F filename    Specifies a library indirect file.
      I         Displays progress information as the library is built.
  O filename    Specifies an output file. If no output file is specified the
                name is taken from the first input file and a .lib extension
                is added.

Table 8.1 - ilibr command line options


ilibr myprog.t4x myprog.t8x
ilibr myprog.t4x myprog.t8x myprog.t0x

In this example, the compiled object code files myprog.t4x and myprog.t8x (compiled for T4 and T8 transputers respectively) are used to create a library. Because no output file name is specified on the command line, the library will be given the name myprog.lib.

In this example, the compiled object code files myprog.t4x, myprog.t8x and myprog.t0x[1] (compiled for T4, T8 and T9000 transputers respectively) are used to create a library. Because no output file name is specified on the command line, the library will be given the name myprog.lib.

23.2.1: Default command line

A set of default command line options can be defined for the tool using the ILIBRARG environment variable. Options must be specified in the variable using the syntax required by the command line. Options from an environment variable are processed before other options.

23.2.2: Library indirect files

Library indirect files are text files that contain lists of input files, directives to the librarian, and comments. Filenames and directives must appear on different lines. Comments must be preceded by the double dash character sequence `--', which causes the rest of the line to be ignored. By convention indirect files are given the .lbb extension.

Indirect files may be nested within each other, to any level. This is achieved by using the #INCLUDE directive. By convention nested indirect files are also given the extension .lbb.

The following is an example of an indirect file:

-- user's .lbb file

userproc1.tco -- single modules
myconcat.tco -- concatenation of modules
#INCLUDE indirect.lbb -- another indirect file
userproc4.tco -- another single module

The contents of a nested indirect file will effectively be expanded at the position it occurred.

To specify indirect files on the command line each indirect filename must be preceded by the `F' option.

23.2.3: Linked object input files

The librarian will also accept linked object files as input, with certain conditions. The facility to create libraries of linked modules provides an easy method of specifying input to the configurer. Such library files should only be referenced from a configuration description.

The librarian will generate an error if an attempt is made to include both linked units and compiled modules in a single library. In addition, libraries of linked object modules must not be used as input to the linker ilink. This is because the linker does not accept linked units as input files.

23.2.4: Library files as input

Library files can themselves be used as input files to ilibr. When a library file is used as a component of a new library, its index is discarded by ilibr.

Library files may not be concatenated for input to the librarian.

23.3: Library modules

Libraries are made up of one or more selectively loadable modules. A module is the smallest unit of a library that can be loaded separately. Modules are selected via the library index.

23.3.1: Selective loading

Libraries can contain the same routines compiled for different transputer types and (for occam modules) in different error modes.

Selection of library modules for linking in with the program is made on the basis of target processor type and error mode. For example, if the program is compiled for an IMS T414 only modules compiled for this processor type or for processors in a compatible transputer class are loaded. For languages such as FORTRAN and C the error mode is always UNIVERSAL.

For C and FORTRAN modules the linker selects the library modules best suited to the compilation units. For occam the compiler identifies the modules to be selected according to the requirements of the main program. The linker then makes the selection.

The linker also selects library modules for linking on the basis of usage. Only those modules that are actually used by the program are linked into the program.

23.3.2: How the librarian sorts the library index

The librarian creates a library index which is used by the linker to select the required modules. The librarian sorts the index so that for a given processor type, the optimum module is always selected by the linker.

The librarian compares and sorts modules according to a number of factors including attributes set by the compiler options used. These determine for example, the instruction set of the module and influence run-time execution times.

For example, where two library modules were derived from the same source but compiled for classes TA and T8, the librarian would place the T8 module first because it uses a larger instruction set. Modules compiled with interactive debugging disabled are placed earlier in the index than those for which debugging is enabled. This is because disabling debugging causes code to execute faster and the librarian orders the index entries such that the first valid entry is always the `best choice'. If two entries are found to be identical the librarian will issue a warning.

23.4: Library usage files

Library usage files describe the dependencies of a library on other libraries or separately compiled code. They consist of a list of separately compiled units or libraries referenced within a particular library. The .liu files required by the toolset's libraries are supplied by INMOS.

If the imakef tool is used then library usage files should be created for all libraries that are supplied without source. This is to enable the imakef tool to generate the necessary commands for linking. Library usage files are text files. They may be created for a specific library by invoking the imakef tool and specifying a .liu target. See section 26.5.

Such files are given the same name as the library file to which they relate but with an .liu extension.

23.5: Building libraries

This section describes the rules that govern the construction of libraries and contains some hints for building and optimizing libraries.

23.5.1: Rules for constructing libraries

  1. Routines of the same name in a library must be compiled for different transputer types, error modes or debug attributes.
  2. Libraries that contain modules compiled for a transputer class (i.e. TA or TB) are treated as though they contain a copy for each member of the class (using the subset of instructions which are common).
  3. Libraries that contain modules compiled in UNIVERSAL mode are treated as though they contain a copy for each of the two error modes HALT and STOP.
  4. Libraries that contain modules with interactive debugging enabled are treated as though they also contain a copy with interactive debugging disabled. (When interactive debugging is enabled on occam code files, channel input/output is performed via library calls otherwise transputer instructions are used).

23.5.2: General hints for building libraries

Routines that are likely to be used together in a program or procedure (such as routines for accessing the file system) can be incorporated into the same library. At a lower level, routines that will always be used together (such as those for opening and closing files) can be incorporated into the same module.

Libraries can contain the same routines compiled for different transputer types, in different error modes and with different input/output access to channels. Only those modules actually used by the program are incorporated by the compiler and linked in by the linker.

Where possible compile library input files with debugging enabled. This enables the debugger to locate the library source if an error occurs inside the library.

When building C libraries care should be taken if the `FS' or `FC' INMOS C compiler command line options are used, that code compatibility is maintained.

23.5.3: Optimizing libraries

It is possible for the user to optimize the size and content of any libraries which he builds himself, to target appropriate processors, improve the speed of code execution and to provide the best code for a given processor. At the same time is can be desirable to build libraries which are flexible i.e. support different transputer targets.

All libraries

Points to consider when constructing libraries in any language or mixture of languages:

Libraries containing occam modules

When building libraries which include modules written in occam the same considerations apply, but also note the following:
When interactive debugging is enabled, channel input/output will be implemented via library calls. When interactive debugging is disabled using the occam compiler `Y' option, transputer instructions are used for channel input/output. This leads to faster execution times. However, disabling interactive debugging for one module of a program, will disable this facility for the whole program. Code which is compiled to use library calls for channel i/o can be called by code compiled to use either library calls or transputer instructions for channel i/o. The opposite case is not true, therefore, in order to build a library which is flexible in this respect, code should be compiled to use library calls.

For a detailed description of transputer types and error modes, see appendix B.

Outlined below are three different approaches to optimization. The first approach provides the greatest level of flexibility in its application. The experienced user may refine these guidelines to specific requirements.

Semi-optimized library build targeted at all transputer types

This is the simplest way to build a flexible library that covers the full range of transputers.

The user should compile each module separately for the following four cases and incorporate all four versions into the library.

Processor type/class     Error mode       Method of channel I/O
T2                       UNIVERSAL        Via library calls
TA                       UNIVERSAL        Via library calls.
T8                       UNIVERSAL        Via library calls.
T9000                    UNIVERSAL        Via library calls.
Note: Error mode and channel i/o only apply only to modules which employ
them e.g. occam modules compiled by oc.

The resulting library will be small in terms of the number of modules it will contain. Due to their generic nature the modules themselves may be bulky and because they contain only the base set of instructions, the execution time for the program will tend to be slower than a more optimized approach.

Optimized library targeted at all transputer types

In order to build a library which is both generalized enough to work for all 32-bit transputers and is then optimized for modules which require extended instructions sets the following approach is recommended:
  1. Compile all modules for classes TA, T8 and T9000. This will provide modules which can be run on all 32-bit transputers.
  2. If any of the modules perform floating point operations, compile these modules for class TB as well.
For 16-bit transputers it should be sufficient to compile all modules for class T2.

Library build targeted at specific transputer types

This method of building a library will limit the use of the library modules to specific transputer types and error modes. It is recommended as the simplest strategy to use when the following options are known for each module: All modules to be included in the library must be compiled for each target transputer type required and, if appropriate, for the same error mode and method of channel input/output. The resulting library may be large and contain a certain amount of duplication.

For example, for the following options:

each module should be compiled for the following:
Processor type/class     Error mode        Method of channel I/O
T414                     HALT              Via library calls
T425                     HALT              Via library calls.
Note: Error mode and channel i/o only apply only to modules which employ
them e.g. occam modules compiled by oc.

23.6: Error Messages

This section lists each error and warning message which may be generated by the librarian. Messages are in the standard toolset format which is explained in appendix A.

23.6.1: Warning messages

filename bad format: symbol symbol multiply exported
An identical symbol has occurred in the same file. There are three possibilities: The same file has been specified twice. The file was a library where previous warnings have been ignored. A module in the file has been incorrectly generated.

filename1 symbol symbol also exported by filename2

An identical symbol has occurred in more than one module. If the linker requires this symbol, it will never load the second module.

23.6.2: Serious errors

bad format: reason
A module has been supplied to the librarian which does not conform to a recognized INMOS file format or has been corrupted.

filename line number bad format: excessively long line in indirect file

A line is too long. The length is implementation dependent, and is set to 256 characters in this version.

filename line number bad format: file name missing after directive

A directive (such as INCLUDE) has no file name as an argument.

filename line number bad format: non ASCII character in indirect file

The indirect file contains some non printable text. A common mistake is to specify a library or module with the F command line argument or the INCLUDE directive.

bad format: not a TCOFF file

The supplied file is not a library or module of any known type.

filename line number bad format: only single parameter for directive

The directive has been given too many parameters.

command line error token

An unrecognized token was found on the command line.

filename could not open for reading

The named file could not be found/opened for reading.

filename1 line number could not open filename2 for reading

The file specified in an INCLUDE directive could not be opened.

filename could not open for writing

The named file could not be opened for writing.

filename must not mix linked and linkable files

The librarian is capable of creating libraries from compiled modules or linked units, but it is illegal to attempt to create a library from both.

no files supplied

Options have been given to the librarian but no modules or libraries.

filename nothing of importance in file

The file name specified in a library indirect file or in an INCLUDE directive was empty or contained nothing but white space or comments.

filename line number only one file name per line

More than one file name has been placed on a single line within an indirect file.

filename line number unrecognised directive directive

An unrecognized directive has been found in an indirect file.