Tier1 and Tier-2 Weekly Report 2013-11-19

General News

• ADC and ATLAS Computing News [20131119]
  • ROC SAM tests
  • 
  • ADC Weekly meeting
  • DDM Ops: Renaming status
    • CERN deployed WebDAV support for EOS
    • T1s almost done (1 eveluating StoRM update, 2 DPMs to go, others done except LOCALGROUPDISK)
    • T2s about halfway
      • CA Cloud: 18 endpoints Done + 9 To go
      • Add endpoints for a given WebDAV endpoint to savannah and create AGIS endpoint
      • https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/WebDAVSetupForATLAS#How_to_add_the_WebDAV_endpoint_i (includes instructions on testing endpoints for renaming)
  • AMOD/ADCoS Report
    • Cleaning up from data loss at TAIWAN-LCG2
    • SAAB for a while did not act (~3.5h Sunday early morning). All the auto-blacklistings were off. Under investigation.
  • PanDA Pilot Update
    • Next version 58g - still in final tests
    • Updates for FAX
    • Now assuming SITEROOT to follow convention software_path/cmtconfig/release
      • E.g. /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.2.11
      • Previous pilot version would source setup file to get SITEROOT
    • Fixed a problem with core file removal (leading to core file ending up in the log)
    • Displaying the IP number of the WN in the log
    • Now allowing for JEM monitoring if schedconfig.allowjem exists and is set to True
    • Added '2> /dev/null' to all uuidgen usages, since command threw stderr on some NG analysis jobs on slc6
    • Getting appdir from new env variable VO_ATLAS_RELEASE_DIR for AtlasP1HLT and AtlasHLT jobs
    • Simplification of production payload setup [similar to NG job setup]
      • First removed python localization (previously command contained full path to python executable), then removed explicit python altogether
      • Removed explicit path to trf
      • Not tested systematically yet
  • Switcher2 commissioning
    • Switcher1 acts on PanDA queues for downtimes of CE/SE in case of production queue, and on SE for analysis queue – Currently running
    • Switcher2 acts on PanDA resources for downtimes of SE, and acts on downtime of CEs only if all CEs which support that PanDA resource are in downtime at the same time – In commissioning
    • Query AGIS for current and upcoming downtimes
    • Upcoming downtime timeline: setbrokeroff, setoffline, downtime, settest, let HC recover to online
    • 
    • Switcher2 is running in dry-run mode, serving all PanDA resources
      • Will add filter for test resources etc
    • Publishing to SSB http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistory?columnid=10100
    • Plan to move a few sites from Switcher1 to 2, then move rest
    • Timescale: end of the year
  • 100Gb Trans-Atlantic test
    • Provision between BNL and CERN. Test current DDM if can utilize full bandwidth
    • Destination: BNL-OSG2_DDMTEST (a part of the production dCache)
    • BNL added 10 additional gridftp doors on top of the existing 10 doors with >=10Gb/s each (some 40Gbps)
    • BNL Storage: 5 test storage hosts (~1.2PB), small SSDs storage (~30TB) + Existing storage
    • Duration expected: A few to several hours: 1 hour at 100Gb/s ~ 45TB per hour
    • Was supposed to be Nov 12-15 but delayed by fibre cut (now repaired)
  • ATLAS WLCG Availability/Reliability reports with SAM
    • "..The WLCG MB agreed that OPS based Availability Reports will be terminated by the end of 2013 at the latest..."
    • The future T0/T1/T2 reports will based on org.VO.* probe sets
    • Infrasctucture and probes been in place for a few years, but some interoperability with test framework and GOCDB, BDII and AGIS needs to be sorted out
    • Questions being posed to operations team to get this finalized

• CA Cloud Squad Actions [20120626]
  • User DaTRI requests approved / processed (last week)
  • Declared Downtimes

• • HammerCloud autoexclusions

• • other cloud squad related savannah and / or ROC ggus tickets [20110705]
    • panda monitor for PD2P activity within clouds (default view (last 7 days) units = datasets)
• Open Issues and Ongoing Discussions
  • Sites should be providing the 2013 storage pledge.
    • http://gstat-wlcg.cern.ch/apps/capacities/sites/ http://gstat-wlcg.cern.ch/apps/pledges/resources
  • HammerCloud [20120214]
    • [20120214] Ongoing tests with McGill https://twiki.atlas-canada.ca/bin/view/AtlasCanada/McGill_Feb2012
    • [20110531] Ongoing tests with Victoria https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CA_Cloud_Apr2011#Additional_Victoria_WG1_testing
    • HammerCloud tests of TRIUMF completed via panda to compare direct i/o via dcap, FileStager and full file copy
      • https://twiki.atlas-canada.ca/bin/view/AtlasCanada/TRIUMF_Aug2010
      • follow-up required to determine which modifications can be added to Panda schedconfig so as to enable the more efficient FileStager method of data input
    • HammerCloud tests of CA clouds sites is planned to be resumed soon to help measure site performance and identify any site setup bottlenecks [2010803]
      • TWiki Link: https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CACloudOct2010

• Security News [20120801]

• gLite/EMI News [20121127]
  • [20121029] UMD v.2.2.1 has been released
  • [20120924] Retirement calendar of gLite 3.1 and currently unsupported gLite 3.2 products
  • [20120911] EMI-2 WN & UI rpms (SL5/x86_64) beta testing version available

• ROC_Canada News [20130625]
  • Current shift - Erming Pei June 24-28, 2013
  • Next Week - Leslie Groer

• WLCG and General Computing News [20120911]

SSB Results

• Critical tests for each site in the Site Status dashBoard are available at the links below
  • Australia-ATLAS
  • CA-MCGILL-CLUMEQ-T2
  • CA-SCINET-T2
  • CA-ALBERTA-WESTGRID-T2
  • CA-VICTORIA-WESTGRID-T2
  • SFU-LCG2
  • TRIUMF-LCG2

SSB PanDA Test Queue Availability (month) - this metric, measured for the analysis queues, defines the site status for data preplacement as alpha (T2D and avail > 90%), bravo (T2 and avail > 90%) or charlie (< 90%).

SAM Results

• site service status via Gridview - Please check your site

• April 2012 Detailed Site Information (valid until 20120430):
  • TRIUMF-LCG2
    • OPS (this month)
    • ATLAS (this month)
  • Australia-ATLAS
    • OPS (this month)
    • ATLAS (this month)
  • CA-ALBERTA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-SCINET-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-VICTORIA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • SFU-LCG2
    • OPS (this month)
    • ATLAS (this month)

APEL Accounting

• ACTION ITEM FOR SITES: Sites please regularly check and confirm that your historical APEL data has been published and looks correct when comparing with batch system utilization
  • See example steps circulated by D. Qing (20100406)
    • account09.c
  • See EGEE Accounting Portal (Canada)
  • See EGEE Accounting Portal (Australia)

• • Links
    • [1] http://wiki.hellasgrid.gr/wiki/bin/view/HellasGrid/GOC/MONToAPEL
    • [2] http://goc.grid.sinica.edu.tw/gocwiki/glite-APEL

APEL Publishing Status

APEL Accounting Results

APEL Synchronization Test Links

• http://goc-accounting.grid-support.ac.uk/rss/TRIUMF-LCG2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/Australia-ATLAS_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-ALBERTA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-VICTORIA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-MCGILL-CLUMEQ-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-SCINET-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/SFU-LCG2_Sync.html

Site Reports

Alberta

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

SFU

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Victoria

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Toronto

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

McGill

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

TRIUMF

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Australia-ATLAS

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Production

Summary

• NOTE: ARDA ProdSys Dashboard has stopped being used by ADCoS team due to information updating issues with the interface. The historical job usage information is to appear separately within ARDA and is being validated. This will be used in the future for information presented here.

• Within the CA Cloud and Worldwide (Panglia View):

• Alternate Views (Source link for last 7 days):
• http://dashb-atlas-prodsys.cern.ch/dashboard/request.py/summary?cloud=TRIUMF&grouping=site&grouping=cloud&end-date=2013-02-01%2020:59:59&start-date=2013-02-07%2020:00:00

-- LeslieGroer - 2013-11-19