Tier-2 Weekly Report 2008-12-02

SAM Results

• availability-ops-20081202.png:
  

• reliability-ops-20081202.png:
  

• availability-atlas-20081202.png:
  

• reliability-atlas-20081202.png:
  

• Source URL: http://lcg-sam.cern.ch:8080/reports/t2/site_avail.xsql

APEL Accounting Publishing Status

• apel-gant-20081202.png:
  

APEL Accounting Results

• apel-chart-20081202.png:
  

General News

• Connectivity to CERN issues [2008-11-26/27]
  • infrastructure problems with USLHCnet with whom CANARIE peers; resulted in varying levels of packet loss and therefore failed SAM tests for T2s
  • resolved 2008-11-28

• Distributed Analysis Challenges
  • preparations ongoing; targetting Wednesday 2008-12-03 for challenge tests in CA cloud
  • DA challenge twiki page created by A. de Silva with link to mini-exercise tests results from the DA team

• Site Test Package (A. de Silva)
  • site admins are requested to install the following package to aid in the debugging of job submission
  • https://twiki.atlas-canada.ca/bin/view/AtlasCanada/SiteTests

• PRODDISK space token to be increased *sites should increase the size of their PRODDISK space token (reservation) in dcache to ~ 1 TB. This is needed to accommodate an increased size requirement for the weekly functional tests run by ADC operations
  • current sizes and occupancies can be seen at: http://atlddm02.cern.ch/dq2/accounting/cloud_view/CANADASITES/30/

Site Reports

Alberta

Outstanding and Recent Issues

• failed SAM tests and alarms coincident with network connectivity troubles to CERN - CERN ROC created ticket to site
• created unscheduled downtime in GOCDB to account for SAM problems - [2007-11-27 to 2007-11-29]
• corrected site published information in BDII to correctly refect membership in CERN ROC [2008-12-02]
• long running pilot jobs (> 48 hours) from general user type production blocking all regular panda job slots; thus failing ganga robots (unexecuted) and non-running Asoka tests [2008-12-02]
• restarted APEL parser and publisher to update site statistics to the end of November - should appear shortly on APEL statistics pages [2008-12-02]

• installing secondary site infrastructure using vmware for gLite updates deployment testing prior to moving any service changes into production
• stderr/stdout direction with a handful of remaining WNs
• various gLite updates to CE, SE and MON pending (see above)

SFU

Outstanding and Recent Issues

• similar SAM test results during period of network connectivity troubles to CERN [2008-11-26/27]
• added new pool to increase MCDISK to 15 TB [2008-11-25]
• SFU panda production queue offline [2008-11-26] - reason?
• job failures reported [2008-11-27] - hardware failure on s4?
• activated but no running pilots/jobs [2008-11-28]
• ggus ticket re: acl problem in SFU dcache [2008-11-29]
• job failures reported [2008-12-02]

Victoria

Outstanding and Recent Issues

• SAM test failures as per the rest of the sites during network difficulties
• /atlas/ca group mappings incorrect to be resolved
• missing valid proxy reported [2008-11-29]
• apel accounting statistics investigation - why are they so low? cronjob error is believed to be the reason. [2008-11-28]
• release 14.2.24 installation check - missing tag?

Toronto

Outstanding and Recent Issues

• SAM test failures as per the rest of the sites
• missing valid proxy reported [2008-11-29]
• restarted dcache [2008-11-26]

Production

Summary

site	success	failure	success (walltime)	failure (walltime)	efficiency	efficiency (walltime)
TRIUMF-LCG2	11942	293	58886001	3255733	97.6%	94.8%
SFU-LCG2	2517	307	12852583	448289	89.1%	96.6%
TORONTO-LCG2	1274	485	20876782	10640844	72.4%	66.2%
ALBERTA-LCG2	931	231	5952546	647915	80.1%	90.2%
VICTORIA-LCG2	922	65	9061391	4996944	93.4%	64.5%
total	17586	1381	107629303	19989725	92.7%	84.3%

dashboard_summary.pdf

Failed Jobs

ProductionFailedJobs-02Dec08.pdf

Notes

-- AsokaDeSilva - 02 Dec 2008