Tier1 and Tier-2 Weekly Report 2012-11-13

General News

• ADC and ATLAS Computing News [20121113]
  • CA Site Availability report for October 2012 https://indico.cern.ch/getFile.py/access?subContId=1&contribId=4&resId=0&materialId=slides&confId=214924
    • All sites above 90% except CA-SCINET-T2: 89.81% (including downtime for cooling maintenance and changing subnets)
  • ADC Weekly meeting
  • LHC 0.84 fb^-1 delivered last week. Peak Luminosity 7.24E33. Some cryo problems.
    • Gearing up for 25ns running in Dec before shutdown, then heavy ions in January
  • Reprocessing
    • Almost all period D done. Some intervention needed as T1 diskspace filling up as all four periods were going simultaneously (B,A,C)
    • About 80% done of 1.4M jobs total
  • T1 Disk Space
    • All T1-T1 replication stopped
    • FZK, INFN, SARA, PIC are out of T0 export
    • Victor (deletion service) needed some assistance over the weekend to keep up (deleted about 1PB since end October)
  • AMOD Report
    • 120k jobs running, about 20k reprocessing
    • Rain flooding at CERN over the weekend
    • ND and RAL working on file recoveries due to problems with pool hardware
    • Unplanned power cut at RAL Nov 8
  • PAT Workshop summary https://indico.cern.ch/getFile.py/access?contribId=7&resId=2&materialId=slides&confId=216449
  • Highlights
    • The expertise in atlas software is changing. New physicists start by analysing data with ROOT and not by contributing to Athena.
    • To analyse the Atlas data we have two different solutions using different frameworks: athena-based and root-based. The root based solution is gaining and this may change how the data has to be stored.
    • The D3PDs have been a success and we have to optimize the Atlas analysis model based on them.
    • The atlas software needs to be more portable and capable of making use of new hardware and new infrastructures.
    • A plan must be set for LS1 as soon as possible.
• CA Cloud Squad Actions [20120626]
  • User DaTRI requests approved / processed (last week)
  • Declared Downtimes

• • HammerCloud autoexclusions

ANALY_SCINET Auto-Excluded at Tue Nov 6 07:30:58 2012 CET
ANALY_SCINET reset online at Tue Nov 6 08:00:09 2012 CET
IAAS Auto-Excluded at Wed Nov 7 02:31:58 2012 CET
IAAS reset online at Thu Nov 8 03:31:04 2012 CE
ANALY_VICTORIA-WG1 reset online at Thu Nov 8 04:30:15 2012 CET
CA-VICTORIA-WESTGRID-T2 reset online at Thu Nov 8 04:31:12 2012 CET
ANALY_AUSTRALIA reset online at Fri Nov 9 08:00:11 2012 CET
ANALY_TRIUMF Auto-Excluded at Fri Nov 9 14:31:10 2012 CET
ANALY_AUSTRALIA Auto-Excluded at Fri Nov 9 16:00:54 2012 CET
ANALY_AUSTRALIA reset online at Fri Nov 9 19:00:12 2012 CET
Australia-ATLAS reset online at Fri Nov 9 20:00:55 2012 CET
ANALY_TRIUMF reset online at Fri Nov 9 23:00:12 2012 CET
ANALY_TRIUMF Auto-Excluded at Sat Nov 10 02:31:03 2012 CET
ANALY_TRIUMF reset online at Sat Nov 10 04:00:12 2012 CET
Australia-ATLAS Auto-Excluded at Sat Nov 10 04:32:01 2012 CET
ANALY_TRIUMF Auto-Excluded at Sat Nov 10 09:31:02 2012 CET
ANALY_TRIUMF_CVMFS Auto-Excluded at Sat Nov 10 14:01:11 2012 CET
ANALY_TRIUMF reset online at Sat Nov 10 15:00:13 2012 CET
ANALY_TRIUMF_CVMFS reset online at Sat Nov 10 15:30:13 2012 CET
ANALY_TRIUMF Auto-Excluded at Sat Nov 10 19:00:48 2012 CET
ANALY_TRIUMF reset online at Sat Nov 10 19:30:12 2012 CET
ANALY_TRIUMF_CVMFS Auto-Excluded at Sat Nov 10 20:30:54 2012 CET
ANALY_TRIUMF_CVMFS reset online at Sat Nov 10 21:30:09 2012 CET
CA-VICTORIA-WESTGRID-T2 Auto-Excluded at Sun Nov 11 00:31:53 2012 CET
ANALY_TRIUMF Auto-Excluded at Sun Nov 11 01:00:55 2012 CET
ANALY_TRIUMF reset online at Sun Nov 11 01:30:09 2012 CET
CA-VICTORIA-WESTGRID-T2 reset online at Sun Nov 11 01:30:52 2012 CET
ANALY_TRIUMF_CVMFS Auto-Excluded at Sun Nov 11 04:31:15 2012 CET
ANALY_TRIUMF_CVMFS reset online at Sun Nov 11 06:00:19 2012 CET
CA-VICTORIA-WESTGRID-T2 Auto-Excluded at Sun Nov 11 08:32:14 2012 CET
CA-VICTORIA-WESTGRID-T2 reset online at Sun Nov 11 09:31:12 2012 CET
ANALY_AUSTRALIA Auto-Excluded at Sun Nov 11 18:01:22 2012 CET
Australia-ATLAS Auto-Excluded at Sun Nov 11 20:31:51 2012 CET
ANALY_AUSTRALIA reset online at Mon Nov 12 00:00:10 2012 CET
Australia-ATLAS reset online at Mon Nov 12 00:01:02 2012 CET
ANALY_AUSTRALIA Auto-Excluded at Mon Nov 12 15:31:24 2012 CET
Australia-ATLAS Auto-Excluded at Mon Nov 12 16:32:11 2012 CET
ANALY_AUSTRALIA reset online at Mon Nov 12 18:00:32 2012 CET
Australia-ATLAS reset online at Mon Nov 12 18:01:42 2012 CET
ANALY_VICTORIA-WG1 Auto-Excluded at Mon Nov 12 18:30:58 2012 CET
CA-VICTORIA-WESTGRID-T2 Auto-Excluded at Mon Nov 12 19:01:50 2012 CE
CA-VICTORIA-WESTGRID-T2 reset online at Mon Nov 12 20:31:10 2012 CET

• • other cloud squad related savannah and / or ROC ggus tickets [20110705]
    • panda monitor for PD2P activity within clouds (default view (last 7 days) units = datasets)
• Open Issues and Ongoing Discussions
  • Sites should be providing the April 2012 storage pledge.
    • http://gstat-wlcg.cern.ch/apps/capacities/sites/ http://gstat-wlcg.cern.ch/apps/pledges/resources
  • HammerCloud [20120214]
    • [20120214] Ongoing tests with McGill https://twiki.atlas-canada.ca/bin/view/AtlasCanada/McGill_Feb2012
    • [20110531] Ongoing tests with Victoria https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CA_Cloud_Apr2011#Additional_Victoria_WG1_testing
    • HammerCloud tests of TRIUMF completed via panda to compare direct i/o via dcap, FileStager and full file copy
      • https://twiki.atlas-canada.ca/bin/view/AtlasCanada/TRIUMF_Aug2010
      • follow-up required to determine which modifications can be added to Panda schedconfig so as to enable the more efficient FileStager method of data input
    • HammerCloud tests of CA clouds sites is planned to be resumed soon to help measure site performance and identify any site setup bottlenecks [2010803]
      • TWiki Link: https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CACloudOct2010

• Security News [20120801]

• gLite/EMI News [20121030]
  • [20121029] UMD v.2.2.1 has been released
  • [20121029] UMD v.1.9.0 has been released
  • [20120925] EGI Trust Anchor release 1.50-1 - https://wiki.egi.eu/wiki/EGI_IGTF_Release
  • [20120924] Retirement calendar of gLite 3.1 and currently unsupported gLite 3.2 products
  • [20120903] extended security support calendar of gLite 3.2 UI, WN, DPM and LFC - extended to 30/11/2012
  • [20120911] UMD 2.1.1 now available - update to lcg-info-dynamic-software v.1.0.7-2
  • [20120911] EMI-2 WN & UI rpms (SL5/x86_64) beta testing version available

• ROC_Canada News [20121030]
  • Current shift - Neil Knecht November 5-9, 2012
  • Next Week - Simon Nderitu

• WLCG and General Computing News [20120911]

SSB Results

• Critical tests for each site in the Site Status dashBoard are available at the links below
  • Australia-ATLAS
  • CA-MCGILL-CLUMEQ-T2
  • CA-SCINET-T2
  • CA-ALBERTA-WESTGRID-T2
  • CA-VICTORIA-WESTGRID-T2
  • SFU-LCG2
  • TRIUMF-LCG2

SSB PanDA Test Queue Availability (month) - this metric, measured for the analysis queues, defines the site status for data preplacement as alpha (T2D and avail > 90%), bravo (T2 and avail > 90%) or charlie (< 90%).

SAM Results

• site service status via Gridview - Please check your site

• April 2012 Detailed Site Information (valid until 20120430):
  • TRIUMF-LCG2
    • OPS (this month)
    • ATLAS (this month)
  • Australia-ATLAS
    • OPS (this month)
    • ATLAS (this month)
  • CA-ALBERTA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-SCINET-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-VICTORIA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • SFU-LCG2
    • OPS (this month)
    • ATLAS (this month)

APEL Accounting

• ACTION ITEM FOR SITES: Sites please regularly check and confirm that your historical APEL data has been published and looks correct when comparing with batch system utilization
  • See example steps circulated by D. Qing (20100406)
    • account09.c
  • See EGEE Accounting Portal (Canada)
  • See EGEE Accounting Portal (Australia)

• • Links
    • [1] http://wiki.hellasgrid.gr/wiki/bin/view/HellasGrid/GOC/MONToAPEL
    • [2] http://goc.grid.sinica.edu.tw/gocwiki/glite-APEL

APEL Publishing Status

APEL Accounting Results

APEL Synchronization Test Links

• http://goc-accounting.grid-support.ac.uk/rss/TRIUMF-LCG2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/Australia-ATLAS_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-ALBERTA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-VICTORIA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-MCGILL-CLUMEQ-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-SCINET-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/SFU-LCG2_Sync.html

Site Reports

Alberta

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

SFU

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Victoria

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Toronto

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

McGill

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

TRIUMF

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Australia-ATLAS

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Production

Summary

• NOTE: ARDA ProdSys Dashboard has stopped being used by ADCoS team due to information updating issues with the interface. The historical job usage information is to appear separately within ARDA and is being validated. This will be used in the future for information presented here.

• Within the CA Cloud and Worldwide (Panglia View):

• Alternate Views (Source link for last 7 days):
• http://dashb-atlas-prodsys.cern.ch/dashboard/request.py/summary?cloud=TRIUMF&grouping=site&grouping=cloud&end-date=2012-04-01%2020:59:59&start-date=2012-04-07%2020:00:00

-- LeslieGroer - 2012-11-13