Tier1 and Tier-2 Weekly Report 2013-05-07

General News

• ADC and ATLAS Computing News [20130507]
  • ADC Weekly meeting
  • DQ2 client update
    • Version 2.4.0 (released April 11)
      • New Rucio naming convention in dq2-put
      • dq2-get automatically skips the LFC lookup for deterministic endpoints
      • Use of httplib as default instead of curl (for SLC6 support)
      • Reports exact version of client to the server
      • Many bug fixes
    • Plan for support for old versions to be suspended in June (refuse client access by Cental Catalogs)
    • Fair number of clients out there to be migrated
      • Pilot running on EGI M/W uses 2.4.0
      • Many central tools uses older versions
        • e.g. Functional Tests datasets are created with 2.3.0
      • Many users use: /cvmfs/atlas.cern.ch/repo/sw/ddm/latest/setup.sh which links to version 0.1.35
      • Will be relinked to 2.4.0 when US sites tests finish
      • /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/DQ2Client/current/setup.sh links to version 2.4.0
      • Recommended for Tier3s at https://twiki.atlas-canada.ca/bin/view/AtlasCanada/ManageTier3SW
    • Sites and users will be asked to upgrade during May and beginning of June (issue GGUS tickets)
      • Will wait until after the latest symlink in /cvmfs/atlas.cern.ch/repo/sw/ddm/ is switched to the current version
      • Cannot tell now how many people use this symlink and how many directly an old version
  • ATLAS LJSFI Installation System 2.0
    • Designed around HA Percona XtraDB cluster (extension of MySQL engine)
    • 6 DB machines in Rome (at least 3 for quorum)
    • Plan to add others at other sites for redundancy (Naples and CERN)
    • Hosting the main DBs used by LJSFi
      • The Installation DB: restyled/extended version of the old database
      • The InfoSys DB: the new database used for resource discovery and matchmaking
    • Used for resource discovery and matchmaking, replacing the WMS one, based on information from AGIS (updated hourly)
    • Several installation agents can run in the same site or in different sites
      • Each agent is linked to an LJSFi server, but if using an HA alias it will be delocalized
      • Each server redirect the DB calls via haproxy
      • Taking advantage of the WAN Multi-Master HA properties of the DB cluster
      • Improved efficiency of the installation agents (>10 times faster than the old system)
    • Redesigned web interface
      • Input boxes with hints rather than combo boxes
      • Links to AGIS and Panda for the output resources
      • More user friendly page navigation (HTML5)
      • Online Help
    • New system is running in shadow mode and can tag Analysis and Production resources separately
    • Will keep compatibility with WMS version for the short term
    • Overriding the queue parameters for production queues
      • The jobs are considered analysis jobs, so will always use the analysis parameters, even for production queues
      • The submission agents try to override the analysis parameters of the prod queues with the production parameters, but still we have some failures
      • The ideal solution would be to have the correct parameters in AGIS
    • Priority set to 4100 (higher than HC)
    • Other issues being debugged
      • The new system will not write in ATLAS_LOCAL_AREA
      • Need to test the auto-setup
      • Need to have SITE_NAME or ATLAS_SITE_NAME defined in the environment
      • source /cvmfs/atlas.cern.ch/repo/sw/local/bin/auto-setup which set FRONTIER_SERVER and DQ2_LOCAL_SITE_ID based on AGIS (fallback to local CVMFS static files)
      • The auto-setup will set the following from AGIS (fallback to local CVMFS static files)
      • Some resources set as analysis but in fact are production (e.g. the MCORE resources at BNL)
      • Variety of lost heartbeats, and other pilot errors to be understood
    • Keep the old infrastructure in place for the PFC updates/Frontier settings updates while testing
    • Expect to be debugged end of May and hope to revalidate all sites then
  • WLCG Squid Monitoring
    • Some sites have found that http://wlcg-squid-monitor.cern.ch/squid_monitors.txt doesn't work properly.
    • Check your site on http://wlcg-squid-monitor.cern.ch/snmpstats/indexatlas.html to make sure it is reporting.
    • All Clouds have some sites that are yet to migrate.

• CA Cloud Squad Actions [20120626]
  • User DaTRI requests approved / processed (last week)
  • Declared Downtimes

• • HammerCloud autoexclusions

• • other cloud squad related savannah and / or ROC ggus tickets [20110705]
    • panda monitor for PD2P activity within clouds (default view (last 7 days) units = datasets)
• Open Issues and Ongoing Discussions
  • Sites should be providing the 2013 storage pledge.
    • http://gstat-wlcg.cern.ch/apps/capacities/sites/ http://gstat-wlcg.cern.ch/apps/pledges/resources
  • HammerCloud [20120214]
    • [20120214] Ongoing tests with McGill https://twiki.atlas-canada.ca/bin/view/AtlasCanada/McGill_Feb2012
    • [20110531] Ongoing tests with Victoria https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CA_Cloud_Apr2011#Additional_Victoria_WG1_testing
    • HammerCloud tests of TRIUMF completed via panda to compare direct i/o via dcap, FileStager and full file copy
      • https://twiki.atlas-canada.ca/bin/view/AtlasCanada/TRIUMF_Aug2010
      • follow-up required to determine which modifications can be added to Panda schedconfig so as to enable the more efficient FileStager method of data input
    • HammerCloud tests of CA clouds sites is planned to be resumed soon to help measure site performance and identify any site setup bottlenecks [2010803]
      • TWiki Link: https://twiki.atlas-canada.ca/bin/view/AtlasCanada/CACloudOct2010

• Security News [20120801]

• gLite/EMI News [20121127]
  • [20121029] UMD v.2.2.1 has been released
  • [20120924] Retirement calendar of gLite 3.1 and currently unsupported gLite 3.2 products
  • [20120911] EMI-2 WN & UI rpms (SL5/x86_64) beta testing version available

• ROC_Canada News [20130423]
  • Current shift - Sergey Chelsky April 22-26, 2012
  • Next Week - Erming Pei

• WLCG and General Computing News [20120911]

SSB Results

• Critical tests for each site in the Site Status dashBoard are available at the links below
  • Australia-ATLAS
  • CA-MCGILL-CLUMEQ-T2
  • CA-SCINET-T2
  • CA-ALBERTA-WESTGRID-T2
  • CA-VICTORIA-WESTGRID-T2
  • SFU-LCG2
  • TRIUMF-LCG2

SSB PanDA Test Queue Availability (month) - this metric, measured for the analysis queues, defines the site status for data preplacement as alpha (T2D and avail > 90%), bravo (T2 and avail > 90%) or charlie (< 90%).

SAM Results

• site service status via Gridview - Please check your site

• April 2012 Detailed Site Information (valid until 20120430):
  • TRIUMF-LCG2
    • OPS (this month)
    • ATLAS (this month)
  • Australia-ATLAS
    • OPS (this month)
    • ATLAS (this month)
  • CA-ALBERTA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-SCINET-T2
    • OPS (this month)
    • ATLAS (this month)
  • CA-VICTORIA-WESTGRID-T2
    • OPS (this month)
    • ATLAS (this month)
  • SFU-LCG2
    • OPS (this month)
    • ATLAS (this month)

APEL Accounting

• ACTION ITEM FOR SITES: Sites please regularly check and confirm that your historical APEL data has been published and looks correct when comparing with batch system utilization
  • See example steps circulated by D. Qing (20100406)
    • account09.c
  • See EGEE Accounting Portal (Canada)
  • See EGEE Accounting Portal (Australia)

• • Links
    • [1] http://wiki.hellasgrid.gr/wiki/bin/view/HellasGrid/GOC/MONToAPEL
    • [2] http://goc.grid.sinica.edu.tw/gocwiki/glite-APEL

APEL Publishing Status

APEL Accounting Results

APEL Synchronization Test Links

• http://goc-accounting.grid-support.ac.uk/rss/TRIUMF-LCG2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/Australia-ATLAS_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-ALBERTA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-VICTORIA-WESTGRID-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-MCGILL-CLUMEQ-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/CA-SCINET-T2_Sync.html
• http://goc-accounting.grid-support.ac.uk/rss/SFU-LCG2_Sync.html

Site Reports

Alberta

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

SFU

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Victoria

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Toronto

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

McGill

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

TRIUMF

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Australia-ATLAS

Recent Issues

• summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Production

Summary

• NOTE: ARDA ProdSys Dashboard has stopped being used by ADCoS team due to information updating issues with the interface. The historical job usage information is to appear separately within ARDA and is being validated. This will be used in the future for information presented here.

• Within the CA Cloud and Worldwide (Panglia View):

• Alternate Views (Source link for last 7 days):
• http://dashb-atlas-prodsys.cern.ch/dashboard/request.py/summary?cloud=TRIUMF&grouping=site&grouping=cloud&end-date=2013-02-01%2020:59:59&start-date=2013-02-07%2020:00:00

-- LeslieGroer - 2013-05-07