Tier1 and Tier-2 Weekly Report 2017-06-20

Tier1 and Tier-2 Weekly Report 2017-06-20

General News

ATLAS Software and Computing Workshop, Valencia last week
WLCG Workshop - Manchester this week
No ADC Technical Coordination Meeting this week
ADC Weekly Meeting [20170620]
- MC Production
  - About 150-200k running slots since weekend as rel21 derivation reprocessing started
  - Agreed to keep global shares in PC+PMG+MC meeting last week
  - Still have URGENT requests of MC15c for coming summer conferences (not conf-notes but papers)
  - To increase MC15's "used share", paused EVNT and MC16c simulation, but did not have desired effect yet due to priority
  - About 1100 M events in the production queue (MC16:950M, MC15:150M)
- Group Derivation Production
  - 21.0.19.4 built last week
    - Validation running on 5 runs
    - CP-trains (9 trains, 49 formats) launched on Saturday
    - Some issues identified
      - Something seem to go wrong in the merging of MC derivations in production
      - Some urgent TAUP MC production - disable internal merge for now (order 100 data sets, 3 formats)
      - Problem with electron trigger matching in MC
      - Some formats report on big drop in events in validation runs compared with earlier rel21 caches
      - Problem with FTAG3, already fixed
    - Short term plans
      - Start processing of the remaining derivations on data15 and 16 this week
      - Use 21.0.19.4 for first open-ended production
- CRC/ADCoS Report
  - Short DNS outage at CERN on Sunday - low impact on production; Kibana needed a kick
  - > 2.5M pending jobs (one user tried submitting 1M jobs, twice!)
  - Data backlog at beginning of week due to rebalancing (2-3 days)
  - Increasing data consolidation requests in rucio (1M) due to DAOD data replication from derivation
  - Couple of monitoring issues with CVMFS stratum 1's
    - Removed ASGC stratum 1 from atlas-condb and atlas monitoring pages
  - Long list of sites with failing pilots from some factory issued GGUS tickets
  - Looking at sites with low walltime efficiencies
    - Sites killing jobs that have already used non-negligible amount of CPU
    - May not rise to level of normal ADCoS alerting or affect HC tests
    - SFU-LCG2 appears on list with 0.1 efficiency
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Availability & Reliability Reports [20170328]
  - 2017 Reports - click down to find reports of format WLCG_All_Sites_ATLAS_MMMYYYY.(html|pdf)
- Sites should be providing the 2016 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift