How to install Atlas software for a cluster (or a single user machine)

WARNING: This page is obsolete.

This page is obsolete - please see manageTier3SW.

Please check also the software versions page to see what versions to install.

This assumes that there is a local cluster of computers which have been setup and, preferably, i686 nodes have GLite installed with the yum method and available to all users. Also, kerberos needs to be correctly setup so that the CVS repository at CERN can be accessed. The procedure described below can also be used for a single-user machine.

To do all of the following, I recommend an ATLAS administrator account - eg. atlasadmin.

Brief description and usage of the ATLASLocalRootBase package

(If you install this, please consider letting us know (atlas-user-support@triumf.ca) so that we can keep you informed of changes.)

Feedback welcome ! ... There are probably better ways to do this and I will amend this page as needed.

ATLASLocalRootBase is an optional package which can be downloaded

cvs -d:pserver:anonymous@cvs.atlas-canada.ca:/atlas/cvs/cvs/atlasCanada checkout ATLASLocalRootBase

and installed for a cluster of machines at a site where it is desirable for software to be centrally managed. This package allows for multiple versions of various packages to reside in the directories and users to select specific versions when setting up their environment. Hence an administrator (for example, atlasadmin) can install and test new versions for compatibility without affecting users.

From the user prespective, all one has to do is the following (or the equivalent c-shell variant):

export ATLAS_LOCAL_ROOT_BASE=/atlas/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh

The user can also add this in ~/.bash_profile and simply type setupATLAS when needed.

export ATLAS_LOCAL_ROOT_BASE=/atlas/ATLASLocalRootBase
alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'

The result will be that the following environment variabes are setup:

ATLAS_LOCAL_ROOT_BASE=/atlas/ATLASLocalRootBase
ATLAS_LOCAL_ROOT=/atlas/ATLASLocalRootBase/i686  (or x86_64 depending on the platform)

together with the following aliases

...Type localSetupAtlantis to setup (standalone) Atlantis
...Type localSetupDQ2 to use DQ2 enduser tools
...Type localSetupDQ2Client to use DQ2 Client
...Type localSetupGanga to use Ganga
...Type localSetupGLite to use GLite (will show only if glite is not already installed by yum)
...Type localSetupPacman to use Pacman
...Type localSetupPandaClient to use Panda Client
...Type localSetupROOT to setup (standalone) ROOT
...Type saveSnapshot [--help] to save your settings
...Type showVersions to show versions of installed software

Nothing else in the user environment will be changed until the user invokes the aliases. (You will only see the summaries for the packages you have installed at your site.)

In addition, users can specify a version of a package to use during setup. The full options are:

source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh --help

Usage: atlasLocalSetup.sh [options]

    This sets up the ATLAS environment for a cluster user

    You need to set the environment variable ATLAS_LOCAL_ROOT_BASE first.

    Options (to override defaults) are:
     -h  --help             Print this help message
     --atlantisVersion=STRING Version of Atlantis to use 
     --dq2Version=STRING        Version of DQ2 (enduser tools) to use
     --dq2ClientVersion=STRING  Version of DQ2 Client to use
     --gangaVersion=STRING    Version of Ganga to use
     --gLiteVersion=STRING    Version of gLite to use
     --pacmanVersion=STRING   Version of pacman to use
     --rootVersion=STRING       Version of ROOT to use

There is also an example requirements file and .gangarc in $ATLAS_LOCAL_ROOT_BASE/user which the user can copy and use.

Another feature that exists in this package is for saving snapshots - that is, a site manager or a user can save the versions of packages used at a certain time and revert to that at a later date.

If a user prefers not to use the aliases, all one has to do is to define the environment variables

ATLAS_LOCAL_ROOT_BASE=/atlas/ATLASLocalRootBase
ATLAS_LOCAL_ROOT=${ATLAS_LOCAL_ROOT_BASE}/<the appropriate arch directory path>

and continue work as before.

There is also a script to do the installations/removals or updates of the software listed in the rest of this page; to see how to use it, type

$ATLAS_LOCAL_ROOT_BASE/utilities/installSW.sh --help
$ATLAS_LOCAL_ROOT_BASE/utilities/removeSW.sh --help 

(You will also need to add the option --user unless the username is atlasadmin.)

To save you time and minimize errors, all these installations and updates (described in the remainder of this page) are done automatically with one script if you use the manageTier3SW package.

The remainder of this page details various installation steps.

Create the ATLASLocalRootBase directory and links

You can find the list of tags in the cvs viewer or use the recommended software versions.

Go to the area (/atlas in the example below) where you want the software installed and checkout the ATLASLocalRootBase package from cvs;

cd /atlas
cvs -d:pserver:anonymous@cvs.atlas-canada.ca:/atlas/cvs/cvs/atlasCanada checkout -r <tag> ATLASLocalRootBase 

If you nfs export this directory, it must also be mounted with the same path on the other machines (ie /atlas in this example.) You can mount the /atlas directory with a --bind option as we do at TRIUMF but be sure to always access the filesystem as /atlas/ATLASLocalRootBase since some software installations hardcode this path.

Then, setup the infrastructure as follows:

/atlas/ATLASLocalRootBase/utilities/createATLASLocalRootBase.pl --baseDirectory=/atlas

This creates the directory structure for scripts and sub-directories for the platforms i686 and x86_64; some subdirectories in x86_64 are softlinked back to i686 since the applications are not yet available for the native platform (when they are, please remove the soft links and install the application.) Note that createATLASLocalRootBase.pl will not overwrite anything - there will be warning messages if something already exists.

Finally, set up your local site configurations. Make a softlink local to this directory (You can either copy TRIUMF's directory or make a softlink to triumf - the later is shown below).

cd /atlas/ATLASLocalRootBase/packageSetups/sites
ln -s triumf local

Now continue with the rest of the package installations as outlined below. These packages should reside in the various sub-directories of /atlas/ATLASLocalRootBase. In addition, a softlink from a package version named current should be created inside every package directory (except Athena). This will be the default version setup for users when they source the setup script.

(i686 only for now) Instead of going through the instructions below, you may want to use the script $ATLAS_LOCAL_ROOT_BASE/utilities/installSW.sh. (type --help for details.)

Install Pacman

You can source it from cern over afs (may run slowly):

source /afs/cern.ch/atlas/software/pacman/pacman-latest/setup.sh

but it is better to install locally: (reference).

cd /atlas/ATLASLocalRootBase/i686/Pacman
mkdir src
cd src
wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-latest.tar.gz
cd ..
gtar xzf src/pacman-latest.tar.gz

The pacman directory will already have a version, in this example, pacman-3.21. Make a soft link to this as shown below.

ln -s pacman-3.21 current

Next, you will need to log in as in bash and csh shells and do the following to setup the first time:

cd /ATLASLocalRootBase/i686/Pacman/current
source setup.[c]sh

To use this version of Pacman for installing Athena in the next section, source the file as above first.

ATLAS Athena software

The ATLAS software is distributed in the form of Pacman kits. A list with releases status is also available. Note that you do not need to install Pacman - you can use the afs version at CERN instead but it may run slower.

Note that the kits do not have the source code; they contain the shared libraries, header files and configurations that will allow you to link and run an application.

Unfortunately pacman installs the kits or multi-kits with hardcoded information of the installation directory. I have not found a way to change these (without writing a script) and it looks like someone has reported this issue in Savannah. So, it is important that once installed, the Athena software must not be moved. Also, note these paths to avoid link.

Instead of CERN or BNL as in the examples below, you can also use TRIUMF's pacman mirror.

Check if a computer is ATLAS ready

Before you fetch a kit, ensure that the computer ATLAS-ready:

pacman -get am-CERN:Ready

Install a Kit

(See the relevant Workbook section.)

( If you are installing for a site, you may want to also see the sections on Multi-release and Mirroring. Multi-kit is addressed in the next section below.)

For example, to install 13.0.20:

mkdir -p /atlas/ATLASLocalRootBase/i686/Athena/Kits/13.0.20
cd /atlas/ATLASLocalRootBase/i686/Athena/Kits/13.0.20
yes | pacman -get am-BNL:13.0.20+KV >& install.log &

will install the kit for release 13.0.20 and validate it ( that is what +KV means). Check the log file install.log to ensure that validation worked and no errors were found during installation.

Install a Multi-kit

Do the following to install a multi-kit - this will only install the Athena release for a specific architecture (32 or 64 bit).

mkdir -p /atlas/ATLASLocalRootBase/i686/Athena/Multi
cd /atlas/ATLASLocalRootBase/i686/Athena/Multi
pacman -get am-CERN/ATLAS.mirror:Multi  -allow tar-overwrite

Note that validation with +KV will fail with the following error:

  removing Multi+KV as some of the old kit validations no longer work

Remember to repace am-CERN with the nearest mirror since this will be a huge download. For western Canada, you can use TRIUMF's mirror.

Installing a production or point1 cache release

( see this reference)

If you need to install a production or a point1 cache release - ie, releases with 4 fields of numbers seperated by dots, you will first have to install the "base" release. For example, to install 13.0.40.5, you will first have to install a the kit version of 13.0.40 (do not do this in a multi-kit since it can be removed in an update). So the steps to do this are, for example :

mkdir -p /atlas/ATLASLocalRootBase/i686/Athena/Kits_SL4/13.0.40
cd /atlas/ATLASLocalRootBase/i686/Athena/Kits_SL4/13.0.40
 yes | pacman -get am-CERN/ATLAS.mirror/:AtlasProduction_13_0_40_i686_slc4_gcc34_opt

Next, setup the environment with the tag of the base release:

source /atlas/ATLASLocalRootBase/i686/Athena/Kits_SL4/13.0.40/cmtsite/setup.sh -tag=13.0.40,32 

Now that the base environment has been setup, the cache can be installed as follows (you can install this anywhere but this procedure installs it in the release itself):

cd /atlas/ATLASLocalRootBase/i686/Athena/Kits_SL4/13.0.40
 yes | pacman -get http://cern.ch/atlas-computing/links/kitsDirectory/Production/cache:AtlasProduction_13_0_40_5_i686_slc4_gcc34_opt

which will install the cache in /atlas/ATLASLocalRootBase/i686/Athena/Kits_SL4/13.0.40/AtlasProduction/13.0.40.5.

What releases are available inside a Kit or Multi-Kit

You can see a listing of the available releases inside a kit or multi-kit. To do this, go to the kit or multi-kit directory and do

pacman -l -d 2

If you want to know what a pacman mirror contains, you can do:

pacman -d 2 -lc <mirror>

Install java for i686

You will need an i686 version of java to use on the x86_64 platform for Atlantis and gLite. To do this,

cd /atlas/ATLASLocalRootBase/i686/java-i686
mkdir src
(fetch java from http://java.sun.com/)
cd ..
(run the java installation which will create, for example, a directory /atlas/ATLASLocalRootBase/i686/java-i686/jre1.6.0_03)
ln -s  jre1.6.0_03 current

Install gLite

If you are on a i686 platform, you should install gLite with yum.

For x86_64, you will need to fetch the tarballs onto an i686 machine and configure it as a single user as outlined in these instructions. Note that with gLite 3.1 (3.1.2-0), there is a bug which you need to go and fix before running yaim. See this page on what you should change.

Install DQ2 (Obsolete)

(05May08: Note that new versions of DQ2Client include the enduser tools so this is obsolete.)

The HEAD version of DQ2 will be pulled from cvs. I do not know if versions matter so I decided to tag it with the date I installed it as follows. Note: You need to first make sure that you have a valid kerberos ticket, for example,:

  kinit -4 -5 desilva@CERN.CH

and also CVSROOT is setup to do the following;

export CVSROOT=:kserver:atlas-sw.cern.ch:/atlascvs

To install, do

  cd /atlas/ATLASLocalRootBase/i686/DQ2
  mkdir DQ2_HEAD_`date +%Y%m%d-%H%M` (eg, this will create a directory DQ2_HEAD_20071015-1154)
  cd DQ2_HEAD_20071015-1154
  cvs co offline/DataManagement/DQ2_0_2/endusers
  cd /ATLAS_LOCAL_ROOT/DQ2
  ln -s DQ2_HEAD_20071015-1154 current

In the last line above, I have a soft link current to the version I downloaded and tested.

The current link will be moved if/when a newer version of DQ2 user tools is installed. Note that you can also use cvs for updating the existing package you checked out; in that case, you need to keep track of the revisions before an update - so that you can revert if there is a problem. I recommend a fresh install as above so that the existing versions are intact.

Note that by default, setup.[c]sh.any is used which relies on afs access to BNL or CERN for the scripts. If you want to use the local versions you downloaded, copy and edit this file as setup.[c]sh to $ATLAS_LOCAL_ROOT_BASE/packageSetups/sites/local/DQ2.

Install DQ2 Client

(The older method of installing the tarballs are in this link - please use the pacman method outlined below instead.)

You will need to see which version of the DQ2Client is available; you can do this a follows:

mkdir -p /atlas/ATLASLocalRootBase/i686/DQ2Client
cd /atlas/ATLASLocalRootBase/i686/DQ2Client
mkdir workarea
cd workarea
pacman -d version -lc  http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/project/ddm/releases/pacman/cache

Suppose the above shows the latest version to be 0.1.3; then you can install this as follows:

cd /atlas/ATLASLocalRootBase/i686/DQ2Client
mkdir 0.1.3
cd 0.1.3
yes | pacman -get "http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/project/ddm/releases/pacman/cache:DQ2Clients | version('0.1.3')"
ln -s 0.1.3 current (after testing !)

Note that this version of DQ2Client includes the enduser tools but syntax (dq2-ls instead of dq2_ls, etc) options are changed (type --help to see them.)

Installing Atlantis

(This is a standalone version; Atlantis is also available from Athena.) This is optional; you need it to view events (unless you use VP1, which is another event display that runs directly under Athena: just setup Athena and type "vp1 myfavourite_AOD_or_ESD_file.pool.root" at the command line...). Instructions for installing Atlantis are in this link.

Install Ganga

Altough different versions of Ganga should in principle be able to exist in the same tree as installed by default, I found an error when installing an older release after the latest version was installed. Hence, I recommend a new directory structure for each version. Do the following (for example, to install 4.4.1)

mkdir -p /atlas/ATLASLocalRootbase/i686/Ganga/4.4.1
cd /atlas/ATLASLocalRootbase/i686/Ganga
wget http://cern.ch/ganga/download/ganga-install
python ganga-install --extern=GangaAtlas,GangaNG,GangaCronus,GangaGUI,GangaPlotter,GangaPanda --prefix=/atlas/ATLASLocalRootBase/i686/Ganga/4.4.1 4.4.1
ln -s 4.4.1 current

Notice the soft link 4.4.1 from current.

These are some notes about Ganga which may be useful if you are doing your own configuration; these are configured in the example_gangarc file which is available in /atlas/ATLASLocalRootBase/user.

Edit the configuration file ~/.gangarc and change / add the following:

• add the line

RUNTIME_PATH = GangaAtlas

• set the following variables if you had installed gLite with yum. (You should not do a gLite setup prior to running Ganga; however, yum installed gLite puts a setup in /etc/profile.d so this change is needed.)

EDG_SETUP    = /opt/glite/etc/profile.d/grid-env.sh
GLITE_SETUP  = /opt/glite/etc/profile.d/grid-env.sh

• set the ATLAS_SOFTWARE path to point to your kit (this is only needed if you want to use Ganga to run jobs locally rather than on the grid.)

ATLAS_SOFTWARE = /atlas/ATLASLocalRootBase/i686/Athena/Multi

• set the virtual organization to atlas:

VirtualOrganisation = atlas

You can check the LCG configuration by typing

config['LCG']

You can change parameters from the Ganga prompt, for example:

config["LCG_Properties"]["middleware"]='GLITE'

• set the voms:

voms = atlas

• set the /atlas/ca attribute (for Canadians only)

init_opts = -voms atlas:/atlas/ca

Installing another i686 version of python

In case it is needed for debugging or to replace existing versions, python for i686 can be installed in a safe way.

Installing ROOT

ROOT is available within Athena and that is the version which should be used. Sometimes, you may need a standalone version and this is how I installed it:

  mkdir -p /atlas/ATLASLocalRootBase/i686/root
  cd /atlas/ATLASLocalRootBase/i686/root
  # fetch the version of root you want and untar it; eg:
  wget ftp://root.cern.ch/root/root_v5.18.00.Linux.slc4.gcc3.4.tar.gz
  gtar zxf root_v5.18.00.Linux.slc4.gcc3.4.tar.gz
  mv root 5.18.00 (it unpacks as root; just rename it to the version.)
  ln -s 5.18.00 current (if this is the version you want to make current).

Maintaining / Updating the software

Updating ATLASLocalRootBase

You can do a cvs update -d -r <tag> (first cd to /atlas/ATLASLocalRootBase) and then fix any conflicts that arise. (To be safe, do a cvs -n update -d -r <tag> to see what will be changed and then do a difference before applying an update.)

In addition, rerun the script createATLASLocalRootBase.pl to create any new directories that are needed.

Updating Pacman

Repeat the steps above and move the current pointer after testing.

Updating the Athena Multi-Kit

The pacman command below will update the multi-kit periodically if you run it as a cron job.

pacman -update

Updating java-i686

To install a new version, repeat the steps above and then move the pointer to current.

Updating gLite

The tarball installation of gLite will leave a cron job running. To install a new version, just repeat the steps outlined before and change the current pointer when finished testing.

If you have installed this with yum, it should automatically be updated. If not, do a yum update glite to see whether there is an update.

Updating DQ2

Do the same as for installing a new version. After testing, change the soft link "current" to point to the new version.

Updating DQ2 Client

Do the same as for installing a new version. After testing, change the soft link "current" to point to the new version.

Updating Atlantis

To update to a new version, install it as instructed and change the current pointer.

Updating Ganga

Updating Ganga is trivial,; just do as instructed above for a new version and, after testing, change where the current pointer.

-- AsokaDeSilva - 11 Oct 2007