This section describes the process of installing the Linda software you have purchased through Gaussian, Inc. and building a distributed-memory parallel version of Gaussian. It assumes that you have already built and tested the regular version of the program. It also assumes that you have read the normal installation instructions and also that you have access to the Gaussian 09 User’s Reference.
HF, CIS=Direct, and DFT calculations on molecules are Linda parallel, including energies, optimizations and frequencies. TDDFT energies and gradients and MP2 energies and gradients are also Linda parallel. Portions of MP2 frequency and CCSD calculations are Linda parallel, but others are only SMP-parallel, so they see some speedup from using a few nodes but no further improvement from larger numbers of nodes.
It is always best to use SMP-parallelism within nodes and Linda only between nodes. For example on a cluster of 4 nodes, each with a dual quad-core EM64T, one should use
%NProcShared=8 %LindaWorkers=node1,node2,node3,node4
rather than using more than one Linda worker per node.
If you have purchased Gaussian binaries, Linda is distributed on the same CD as the Gaussian binaries and no additional installation is necessary. Follow the directions in the file README file on the distribution CD.
If you have purchased Gaussian source code, then Linda is distributed on a separate CD. Follow the directions in the file README.source file on the Gaussian source distribution CD to install Linda and compile Gaussian to use Linda.
In either case, you must run the command bsd/install as detailed in the README files and installation instruction sheets.
The Linda parallel programming model involves a master process, which runs on the current processor, and a number of worker processes which can run on other nodes of the network. So a Gaussian 09/Linda run must specify the number of processors to use, the list of processors where the jobs should be run, and occasionally other job parameters. An environment variable is generally the easiest way to specify this information (as we will see).
Each of these nodes needs to have some access to the Gaussian 09 directory tree. The recommended configuration is to have G09 on each system that will be used for the parallel job. Note that the Linda binaries need to have the same path on each machine. If this is not feasible, the G09 tree can be made accessible via an NFS-mounted disk which is mounted in an identical location on all nodes.
For MP2 calculations, each node must also have some local disk where Gaussian 09 can put temporary files. This is defined as usual via the GAUSS_SCRDIR environment variable, which should be set in the .cshrc or .profile for your account on each node.
Gaussian 09 gets configuration information from three primary sources
Details about %Link0 commands and the Default.Route file can also be found in the Gaussian 09 User’s Reference manual. Entries specific to Gaussian 09/Linda are described below.
The %LindaWorkers directive is used to specify computers where Linda worker processes should run. It has the following syntax:
%LindaWorkers=node1[:n1][,node2[:n2]] […]
This lists the TCP node name for each node to use. By default, one Linda worker is started on each node, but the optional value allows this to be varied. A worker is always started on the node where the job is started (the master node) whether or not it appears in the node list. %LindaWorkers may be combined with %NProcShared. In this case, one or more parallel worker processes will be run on each node (the number still determined by the values in %LindaWorkers). The value to %NProcShared specifies the number of SMP processors/cores to use on each system in the worker node list.
Do not use the obsolete %NProcLinda directive. G09 will compute the total number of Linda workers based on the %LindaWorkers input.
The following directive causes a network parallel job to be run across the specified 5 nodes. Nodes hamlet and ophelia will each run two worker processes.
%LindaWorkers=hamlet:2,ophelia:2,laertes,horatio,lear
The following directives specify that a parallel job will be executed on hosts norway, italy and spain. Nodes norway and italy will each run one 4-way SMP parallel worker, and spain will run two such workers:
%NProcShared=4 Specifies four-way SMP parallelism. %LindaWorkers=norway,italy,spain:2
These directives make sense when norway and italy are 4 processor/core computers, and spain is an 8 processor/core computer.
Note that the %NProc directive used in earlier Gaussian versions is obsolete.
You can also specify these items in two other ways. You can define the following environment variables:
$ export GAUSS_WDEF="norway,italy,spain:2" bash $ export GAUSS_PDEF="4" % setenv GAUSS_WDEF "norway,italy,spain:2" tcsh % setenv GAUSS_PDEF 4
Finally, these items can also be specified on the g09 command line:
$ g09 -w="norway,italy,spain:2" -p="4"
The %CPUNumber directive specifies which cores/nodes to use for the calculation by node number (beginning with 0). For example, the following directive specifies that the job should use all four cores on a quad-core system:
%CPUNumber=0,1,2,3
Memory is specified using the %Mem Link0 command, just as for serial calculations.
By default, Linda uses rsh to communicate between nodes. You can use ssh instead by including the following Link 0 option within the job file:
%UseSSH
You can also specify ssh use in the GAUSS_LFLAGS environment variable:
% setenv GAUSS_LFLAGS '… -opt "Tsnet.Node.lindarsharg: ssh"'
Finally, you can create a configuration file on the master node named .tsnet.config which contains the following line:
Tsnet.Node.lindarsharg: ssh
This will cause ssh to be used instead. This file can be placed in your home directory or in the directory from which you launch calculations.
Note that passwordless ssh logins must already be configured from the master to all worker nodes.
A few Linda options that are sometime useful are:
-v Display verbose messages -vv Display very verbose messages
Use the GAUSS_LFLAGS environment variable to set them.
For example, one could turn on very verbose Linda output using:
% setenv GAUSS_LFLAGS -vv
There are many other Linda options but most of them are not used by Gaussian. Check the Linda manual which can be downloaded here. The -opt form can be used in GAUSS_LFLAGS to invoke any valid .tsnet.config file directive. Note that Gaussian 09/Linda does not use the native Linda resources minworker and maxworker.
The g09 command is used as usual to initiate a distributed memory parallel a Gaussian 09 job. For a Linda parallel job to execute successfully, the following conditions must be true:
You have already executed the appropriate Gaussian 09 initialization file ($g09root/g09/bsd/g09.login or $g09root/g09/bsd/g09.profile). Test this by running a serial Gaussian 09 calculation on the master node.
The directory $g09root/g09 is accessible on all nodes.
The LD_LIBRARY_PATH environment variable is set (see the G09 install notes) to locate the Linda shared libraries.
Local scratch space is available on each node if needed (via GAUSS_SCRDIR).
All nodes on which the program will run are trusted by the current host. You should be able to login remotely with the rlogin or ssh command without having to give a password to each of them. Contact your system administrator about configuring security for the nodes in the network.
Calculations can then be started as you would for a serial calculations :
% g09 input &
and Gaussian 09 will start the master and worker processes as needed.
You will see processes started on the worker nodes for those links which have been parallelized, e.g. they will have a *.exel entry in the main G09 directory. Using the top or other commands on worker nodes will let you see lxxx.exel when it starts.
The relevant measure of performance for a parallel calculation is the elapsed or wall clock time. The easiest way to check this is to use an external monitor like time, times, or timex, e.g.
% times g09 input &
which will report elapsed, CPU and system times. Note that the last two are only on the master node and similar amounts of CPU and system time are expended on each node. So the speedup is the ratio of the elapsed time running a serial job and the elapsed time for the parallel job.
The -mp n option can be used to run Gaussian with Linda across multiple power PC-based Mac OS X and other multiprocessor systems. It specifies the maximum number of Linda processes to be scheduled per node. Set it to 2 when all of the individual nodes are dual processors.
Last update: 27 April 2016