GangaTutorialAtToronto

WARNING: This page is obsolete.

Introduction

Here some specific instructions for the GANGA Tutorial at the University of Toronto (this page is modelled after the Tutorial that was held at Lyon, CCIN2P3 ). More detailed explanations can be found in DistributedAnalysisUsingGanga and GangaTutorial427. Note that previous tutorials relied explicitly on setups from CERN via /afs. For the purposes of the Atlas-Canada tutorial, we have tried to excise as many external dependencies as possible.

Setup Instructions

Setup GANGA

GANGA knows how to setup the grid UI, so there is no need for you to do it.

  source /sps/atlas/g/ganga/setup.sh

You can start GANGA simply by entering ganga. To start the GUI enter ganga --gui.

The first time you will start GANGA a file ~./gangarc will be created. It can be used to customise your ganga session. For example one can move the ganga workspace to another place

   [DefaultJobRepository]
   local_root = /sps/atlas/l/liko/gangadir
   [FileWorkspace]
   top_dir = /sps/atlas/l/liko/gangadir

At CERN you can use

   source /afs/cern.ch/sw/ganga/install/etc/setup-atlas.sh

which will give you always the latest version

Setup the LCG UI

In case you want to use LCG commands, you have to setup the UI.

    source /afs/in2p3.fr/grid/profiles/lcg_env.[c]sh

For detailed explanations you can consult the LCG Users Guide.

Setup DQ2 end user tools

DQ2 end user tool require the LCG UI.

   source /afs/in2p3.fr/grid/profiles/lcg_env.[c]sh
   source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.[z|c]sh.CCIN2P3

Attention zsh users! Use the correct setup file! In case you are behind a firewall its useful to set

   export DQ2_COPY_COMMAND='lcg-cp -v --vo atlas'

The most useful commands for analysis are dq2_ls to list datsets and their content and dq2_get to download data or results from analysis jobs.

   dq2_ls -fp 

For more detailed explanations see UsingDQ2. In case of problems see the troubleshooting section.

Setup DQ2 Client

In some cases itis useful to access directly the central catalogs

   alias dq2=/afs/cern.ch/atlas/offline/external/GRID/ddm/pro02_slc3/dq2

Further instruction to use DQ2 are given in DDMClientDQ2.

Hello World Job on the LCG/EGEE grid

Start GANGA interactively. You will be prompted for the passphrase of your grid certificate.

   [liko@ccali30 ~]$ ganga
   *** Welcome to Ganga ***
   Version: Ganga-4-2-11
   Documentation and support: http://cern.ch/ganga
   Type help() or help('index') for online help.

   ATLAS User Support is provided by the Hypernews Forum Ganga User and Developers
   You find the forum at https://hypernews.cern.ch/HyperNews/Atlas/get/GANGAUserDeveloper.html
   or you can send an email to hn-atlas-GANGAUserDeveloper@cern.ch

  Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 0 jobs in jobs
  Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 1 jobs in templates

  In [1]:
 

Submit a Hello World Job

   j = Job()
   j.application=Executable()
   j.application.exe='/bin/echo'
   j.application.args='Hello World'
   j.backend=LCG()
   j.submit()

Ganga jobs are python objects and can be accordingly manipulated

   j = jobs[1].copy()
   j.backend=Local()
   j.splitter=ExeSplitter()
   j.splitter.apps=10*[Executable()]
   j.submit()

Setup your Athena environment

If you are new to Athena consult first the WorkBook or the pages by the PhysicsAnalysisTools.

Create your ~/cmthome directory according to the instructions given in the local Athena User Guide, but remove

   set   CMTSTRUCTURINGSTYLE "with_version_directory"

(I had to switch to bash shell, as the environemnt was too large for my csh.)

The create your working area and setup your environment

   mkdir -p $HOME/testarea/12.0.6
   cd $HOME/testarea/12.0.6
   source ~/cmthome/setup.sh -tag=12.0.6

Check out your analysis exampe and compile it (example assumes that you have choosen

   export CVSROOT=:ext:atlas-sw.cern.ch:/atlascvs
   export CVS_RSH=ssh
   cmt co  -r UserAnalysis-00-09-10-09 PhysicsAnalysis/AnalysisCommon/UserAnalysis
   cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
   cmt config
   source setup.csh
   cmt broadcast gmake
   cd ../run
   get_files -jo AnalysisSkeleton_topOptions.py

Then set your input file to a file on the dCache storage area by modifying AnalysisSkeleton_topOptions.py

   EventSelector.InputCollections = [   "dcap://ccdcache.in2p3.fr:22125/pnfs/in2p3.fr/data/atlas/disk/dq2/mc12test_misal1_csc11/mc12test_misal1_csc11.005403.SU3_jimmy_susy.recon.AOD.v12000601_tid005575/mc12test_misal1_csc11.005403.SU3_jimmy_susy.recon.AOD.v12000601_tid005575._00400.pool.root.1"
]

Unluckley Athena v12 is distributed with the wrong version of the ROOT dCache plugin, which is not compatibe with most ATLAS dCache installations. For some version the installed software has been patched, some not - for example version 12.0.6 in Lyon.

   curl --insecure https://twiki.cern.ch/twiki/pub/Atlas/IssuesWithPosixIO/libDCache.so-5.10d -o libDCache.so
   chmod +x ./libDCache.so
   export LD_LIBARARY_PATH=$PWD:$LD_LIBRARY_PATH

More details on what can go wrong using PosixIO on various grid sites see IssuesWithPosixIO. Now you should be able to analyse your data locally

   athena AnalysisSkeleton_topOptions.py

Locate your data

This paragraph is not concerned with finding out which dataset you want to use. This is the task of AMI. This paragraph is about finding where your data is in the world.

The ATLAS DDM is based on Datasets. A dataset has a content and one or several locations. On each location the dataset can be complete or incomplete. Currently we are trying to establish the first time a distribution of data on all Tier-1 sites. Tier-2 sites will follow. As long as many datasets are incomplete, automatic site selection does not make too much sense.

At this point users, who want to do something useful, have to select their dataset manually - based on the number of files on a give site. There are several tools that can be used to locate datasets.

Data in the French Cloud are cataloged by Stephane. The Dataset broswer by Torre gives another view.

You can also list datasets on a site by query the central catalog

   dq2 listDatasetsInSite SACLAY | grep AOD\.v120006

You can also list on which sites a dataset can be found

dq2 listDatasetReplicas trig1_misal1_csc11.005022.FJ3_pythia_jetjet.recon.AOD.v12000601_tid006008

And you can list the content of the dataset on each site

   export DQ2_LOCAL_ID=SACLAY
   dq2_ls -fp trig0_calib0_csc11.005310.PythiaH120gamgam.recon.AOD.v12000501_tid005238

Send an AOD analysis job to the Grid

Most easy way to send a job to the grid - similar to pathena

   ganga athena --inDS  trig1_misal1_csc11.005033.Jimmy_jetsJ4.recon.AOD.v12000601 \
               --outputdata AnalysisSkeleton.aan.root --split 3 \
               --maxevt 100 \
               --lcg --ce ce102.cern.ch:2119/jobmanager-lcglsf-grid_2nh_atlas \
              AnalysisSkeleton_topOptions.py

In case of problems with the installation on site you have to manipulate the job in the CLI

j=Job()
j.application=Athena()
j.application.prepare()
j.application.option_file='$HOME/athena/testarea/12.0.6/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run/AnalysisSkeleton_topOptions.py'
j.application.max_events='100'
j.splitter=AthenaSplitterJob()
j.splitter.numsubjobs=3
j.inputdata=DQ2Dataset()
j.inputdata.dataset='misalg_csc11.005300.PythiaH130zz4l.recon.AOD.v12003104'
j.inputdata.type='DQ2_LOCAL'
j.outputdata=DQ2OutputData()
j.outputdata.outputdata=['AnalysisSkeleton.aan.root']
j.backend=LCG()
j.backend.CE='ce102.cern.ch:2119/jobmanager-lcglsf-grid_2nh_atlas'
# Lyon and FZK are using dCache
#j.inputsandbox=['libDCache.so']
#j.backend.CE='ce-fzk.gridka.de:2119/jobmanager-pbspro-atlasS'
#j.backend.CE='cclcgceli02.in2p3.fr:2119/jobmanager-bqs-short'
j.submit()

Follow the Job on the Grid

You can use the CLI to follow the status of your job

bash-3.00$ ganga

*** Welcome to Ganga ***
Version: Ganga-Base-4-15
Documentation and support: http://cern.ch/ganga
Type help() or help('index') for online help.


ATLAS User Support is provided by the Hypernews Forum Ganga User and Developers
You find the forum at https://hypernews.cern.ch/HyperNews/Atlas/get/GANGAUserDeveloper.html
or you can send an email to hn-atlas-GANGAUserDeveloper@cern.ch

Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 11 jobs in jobs
Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 1 jobs in templates


In [1]:jobs
Out[1]: Statistics: 11  jobs
--------------
#   id      status        name   subjobs      application          backend                               backend.actualCE
#    1   completed                             Executable              LCG  ce-fzk.gridka.de:2119/jobmanager-pbspro-atlas
#    2         new                             Executable              LCG                                   
#    3   completed                   100       Executable            Local                                   
#    4         new                             Executable              LCG                                   
#    5         new                                 Athena              LCG                                   
#    6         new                                 Athena              LCG                                   
#    7         new                                 Athena              LCG                                   
#    8         new                                 Athena              LCG                                   
#    9         new                                 Athena              LCG                                   
#   10         new                                 Athena              LCG                                   
#   11     running                     3           Athena              LCG                                   

Ganga.GPIDev.Lib.Job               : INFO     job 11.0 status changed to "running"
Ganga.GPIDev.Lib.Job               : INFO     job 11.2 status changed to "running"
Ganga.GPIDev.Lib.Job               : INFO     job 11.1 status changed to "running"
Ganga.GPIDev.Lib.Job               : INFO     job 11 status changed to "running"

You can also use the GUI ganga --gui.

When the job has finished you can retrieve your Ntuple using dq2_get

  dq2_get -rv user.DietrichLiko.ganga.11

Now you can go on and try other task from GangaTutorial427

• Perform TAG Analysis
• Run the

• Try the GUI at GangaGUITutorial427

  ---

%RESPONSIBLE% LeslieGroer
%REVIEW% Never reviewed

-- LeslieGroer - 20 Mar 2007