Tier1 and Tier-2 Weekly Report 2017-04-04

General News

• ADC Tech Coord Meeting [20170403]
  • Discussion about dispatch datasets
• ADC Weekly Meeting [20170404]
  • MC Production
    • No major issues. Running 100k-150k slots due to reprocessing
    • 95% MC16, 5% MC15 eventhough global shares should be 1:1
    • About 930M events in production queue (800M MC16, 130M MC15)
  • Group Derivation Production
    • Built 20.7.8.15 earlier this week (bug-fix)
      • Use for full processing of data15+16 for one format
      • Limited MC prod
    • Built first rel 21 cache (21.0.19.1) for data
      • Submitted trains on all 5 reprocessed runs for checking
      • Fast merge enabled
      • No MC production yet
  • Reprocessing
    • 58 runs submitted - all period A+L (20+24), period I (13) one period F
    • About half are done
    • Most DRAW reco and merges paused due to Frontier issues
    • DESD file size is under examination now
    • 1:1 merging workflow could be simplified to "cp"
  • Pilot Update (68.2)
    • Benchmarks - send dictionary to intermediary server before elastic search
    • Standard payload cleanup - much simplified setup function
      • Pilot uses info from job.release and job.homePackage to determine the asetup options
    • Fixes for Event Service
  • CRC/ADCoS Report
    • Storage at some T1's and T2's filling up - rebalancing
    • Jobs from later physics period have somewhat larger memory requirements than 2 GB/core - running only on HIMEM queues
    • To get more slot some tasks were fixed at 2 GB/core - causing increased number of lost heartbeats at some sites
    • Frontiers at IN2P3-CC and RAL under heavy load
      • Many sites blacklisted because of these problems
      • Many tasks paused to ease the situation, currently starting again
      • Under investigation
  • Reproducibility of MC production steps on the grid - Knut Vadla
    • Compare data produced at different sites and also INTEL vs AMD
    • Use physics validation framework to investigate statistical variations
    • Also do bit-wise comparisons
    • Details in slides
• CA Cloud Squad Actions
  • User DaTRI requests approved / processed (last week)
• Open Issues and Ongoing Discussions
  • Availability & Reliability Reports [20170328]
    • 2017 Reports - click down to find reports of format WLCG_All_Sites_ATLAS_MMMYYYY.(html|pdf)
  • Sites should be providing the 2016 storage pledge.
    • REBUS Site Capacities
    • REBUS Federation Resource Pledges
  • ATLAS Canada HammerCloud Tests
• WLCG and General Computing News
  • Security News
  • gLite/EMI News
• ROC_Canada News
  • Current shift

ABCD Determination and Monitoring

• SSB Qualification and ABCD Determination Wiki
• CA Cloud - Panda Resource Status
• CA Cloud - Switcher Actions
• CA Cloud - ABCD Determination

SSB Results

• Critical tests for each site in the Site Status dashBoard are available at the links below
  • Australia-ATLAS
  • CA-MCGILL-CLUMEQ-T2
  • CA-SCINET-T2
  • CA-ALBERTA-WESTGRID-T2
  • CA-VICTORIA-WESTGRID-T2
  • SFU-LCG2
  • TRIUMF-LCG2

SAM Results (based on VO tests)

• ATLAS Site Availability for last two weeks - CA Cloud
• ROC_Canada
• Australia_ATLAS

APEL Accounting

• ACTION ITEM FOR SITES: Sites please regularly check and confirm that your historical APEL data has been published and looks correct when comparing with batch system utilization
  • See example steps circulated by D. Qing (20100406)
    • account09.c
  • See EGI Accounting Portal (Canada)
  • See EGI Accounting Portal (Australia)
  • Links
    • [1] http://wiki.hellasgrid.gr/wiki/bin/view/HellasGrid/GOC/MONToAPEL
    • [2] http://goc.grid.sinica.edu.tw/gocwiki/glite-APEL

APEL Synchronization Test Links

• TRIUMF-LCG2
• Australia-ATLAS
• CA-VICTORIA-WESTGRID-T2
• CA-MCGILL-CLUMEQ-T2
• CA-SCINET-T2
• SFU-LCG2

Production and Usage Status

• ADC Job Summaries Dashboard
• Within the CA Cloud and Worldwide ( Panglia View):

-- %USERSIG{LeslieGroer - 2017-04-04%

Comments