Tier1 and Tier-2 Weekly Report 2013-11-12

General News

• ADC and ATLAS Computing News [20131112]
  • ADC Weekly meeting
  • Analysis Default limit - set default to 12 hours in PanDA next week.
    • Average length of job 1-2 hours so fine for 99.99%
    • Users can request longer via --maxCpuCount
  • TAIWAN-LCG2 File loss
    • ~ 1M files lost (800k on DATADISK, 450k logs)
    • Recovery effort underway (high priority already handled)
  • AMOD Report
    • Sudden restart of the ss-Triumf on (Sunday Nov 3
    • SAAB failure : SAM test were not running (Nov 5)
    • AGIS web interface problems, one machine failure (Nov 6)
    • CC machines were successfully upgraded (Monday Nov 4)
    • ATLAS-AMI-CERN problems with SLS monitoring (Nov 6)
  • DPD Production: volume estimate
    • MC requests usually forseen about 1 month ahead
    • 1.6PB of data12 (includes physics an delayed streams)
    • 3.6PB of mc12 (about 250TB for 14TeV studies)
    • Data volume estimate on NTUP_COMMON output for upcoming productions underway (about a month)
    • JetMet request for data11 had about 30TB and mc11 10TB
    • Might be able to reduce processing from 10s/event to 5s if do not need to rerun jet collections
    • Other collections smaller - few TB-10TB each
  • MC Production Report
    • Low fraction (10%) of MC11 production still running
    • Limited scope MC11d production undeway (~150M events) with updated geometry. Validated and about to launch.
    • Some changes to output file locations for MC12 work flow due to transient files
    • Volume estimates:
    • Total merged AOD: ~300 TB/month
      • MC12 8 TeV: 600M -> 264 TB
      • MC11 7 TeV: 60M -> 15 TB
      • MC12 14 TeV: 11M -> 20 TB
    • Total ESD/RDO: ~60/10 TB/month
      • MC12 14 TeV: 11M -> 54/10 TB
    • Next big campaign will be DC14 starting early next year
  • HammerCloud and AGIS
    • Autosynching HammerCloud state for queues from AGIS, once a day instead of via human intervention
    • Three possible values for \hc_param (case sensitive!):
      • \False: HC will ignore this queue
      • \OnlyTest: HC will send functional tests, but not act on the results
      • \AutoExclusion: HC will send functional test and autoexclude based on them
    • Set desired value in AGIS panda resource table
    • MCORE queues blacklisted based on corresponding single core queue separate set of low frequency tests
    • If something strange happens, please complain at atlas-adc-hammercloud-support@cern.ch

• CA Cloud Squad Actions [20120626]
  • User DaTRI requests approved / processed (last week)
  • Declared Downtimes

• • HammerCloud autoexclusions

• • other cloud squad related savannah and / or ROC ggus tickets [20110705]
    • panda monitor for PD2P activity within clouds (default view (last 7 days) units = datasets)
• Open Issues and Ongoing Discussions
  • Sites should be providing the 2013 storage pledge.
    • http://gstat-wlcg.cern.ch/apps/capacities/sites/ http://gstat-wlcg.cern.ch/apps/pledges/resources
  • HammerCloud [20120214]
    • [20120214] Ongoing tests with McGill https://twiki.atlas-canada.ca/bin/view/AtlasCanada/McGill_Feb2012
    • [20110531] Ongoing tests with Victoria https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CA_Cloud_Apr2011#Additional_Victoria_WG1_testing
    • HammerCloud tests of TRIUMF completed via panda to compare direct i/o via dcap, FileStager and full file copy
      • https://twiki.atlas-canada.ca/bin/view/AtlasCanada/TRIUMF_Aug2010
      • follow-up required to determine which modifications can be added to Panda schedconfig so as to enable the more efficient FileStager method of data input
    • HammerCloud tests of CA clouds sites is planned to be resumed soon to help measure site performance and identify any site setup bottlenecks [2010803]
      • TWiki Link: https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CACloudOct2010

• Security News [20120801]

• gLite/EMI News [20121127]
  • [20121029] UMD v.2.2.1 has been released
  • [20120924] Retirement calendar of gLite 3.1 and currently unsupported gLite 3.2 products
  • [20120911] EMI-2 WN & UI rpms (SL5/x86_64) beta testing version available

• ROC_Canada News [20130625]
  • Current shift - Erming Pei June 24-28, 2013
  • Next Week - Leslie Groer

• WLCG and General Computing News [20120911]

SSB Results

• Critical tests for each site in the Site Status dashBoard are available at the links below
  • Australia-ATLAS
  • CA-MCGILL-CLUMEQ-T2
  • CA-SCINET-T2
  • CA-ALBERTA-WESTGRID-T2
  • CA-VICTORIA-WESTGRID-T2
  • SFU-LCG2
  • TRIUMF-LCG2

SSB PanDA Test Queue Availability (month) - this metric, measured for the analysis queues, defines the site status for data preplacement as alpha (T2D and avail > 90%), bravo (T2 and avail > 90%) or charlie (< 90%).

SAM Results

• site service status via Gridview - Please check your site

• April 2012 Detailed Site Information (valid until 20120430):
  • TRIUMF-LCG2
    • OPS (this month)
    • ATLAS (this month)
  • Australia-ATLAS
    • OPS (this month)
    • ATLAS (this month)
  • CA-ALBERTA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-SCINET-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-VICTORIA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • SFU-LCG2
    • OPS (this month)
    • ATLAS (this month)

APEL Accounting

• ACTION ITEM FOR SITES: Sites please regularly check and confirm that your historical APEL data has been published and looks correct when comparing with batch system utilization
  • See example steps circulated by D. Qing (20100406)
    • account09.c
  • See EGEE Accounting Portal (Canada)
  • See EGEE Accounting Portal (Australia)

• • Links
    • [1] http://wiki.hellasgrid.gr/wiki/bin/view/HellasGrid/GOC/MONToAPEL
    • [2] http://goc.grid.sinica.edu.tw/gocwiki/glite-APEL

APEL Publishing Status

APEL Accounting Results

APEL Synchronization Test Links

• http://goc-accounting.grid-support.ac.uk/rss/TRIUMF-LCG2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/Australia-ATLAS_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-ALBERTA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-VICTORIA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-MCGILL-CLUMEQ-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-SCINET-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/SFU-LCG2_Sync.html

Site Reports

Alberta

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

SFU

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Victoria

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Toronto

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

McGill

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

TRIUMF

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Australia-ATLAS

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Production

Summary

• NOTE: ARDA ProdSys Dashboard has stopped being used by ADCoS team due to information updating issues with the interface. The historical job usage information is to appear separately within ARDA and is being validated. This will be used in the future for information presented here.

• Within the CA Cloud and Worldwide (Panglia View):

• Alternate Views (Source link for last 7 days):
• http://dashb-atlas-prodsys.cern.ch/dashboard/request.py/summary?cloud=TRIUMF&grouping=site&grouping=cloud&end-date=2013-02-01%2020:59:59&start-date=2013-02-07%2020:00:00

-- LeslieGroer - 2013-11-12