Tier1 and Tier-2 Weekly Report 2016-06-28

Tier1 and Tier-2 Weekly Report 2016-06-28

General News

ADC and ATLAS Computing News [20160628]
ADC Technical Coordination Meeting [20160627]
- DDM Status Update
  - Been having minor release of rucio every 1-2 months
  - Boost button for admins
  - Upload/download/delete on SRM-less sites. Working on space reporting.
  - Dark Data cleanup working well, but relies on sites providing data dumps. 4PB cleaned. TRIUMF SCRATCHDISK 9 of 10 (1PB cleaned)
  - Working on scheme to allow obsoleted data to remain on LOCALGROUPDISK but not be discoverable
  - Progress on writing logs to OS (S3). Test at large scale
  - BB8 for data rebalancing should be in automated production in October
  - C3PO for data pre-placement needs better metrics. In progress
  - Useful analysis of state of rucio rule being added to interface
  - All sites will be asked top open xrootd doors with direct pfn access - GGUS ticket out soon
    - Can still use n2n if want
    - Xroot doors made read/write
  - HTTP task force semi revived - still about 30 sites without
  - Testing SRM write-to-tape with SRM-less source sites
- Cloud Benchmarking
  - CPU Benchmark in Clouds - Randy Sobie
    - Use DIRAC and Belle II jobs
    - Compare with condor_mips and DIRAC HS06
  - CERN Cloud Benchmarking Suite
    - Collect and measure performance metrics from benchmarks on any IaaS
    - Use Elastic Storage and Kibana for reporting/analysis
ADC Weekly Meeting [20160628]
- MC Production
  - Still some issues keeping grid full as transition from MC15 to MC16
    - "panic sessions" to ensure don't empty over weekend, but concern that can't do all necessary checks and scrutiny
  - Proposed change last week for unmerged AODs - keep for 1 month if >10% failure in merge step, just in case
- Group Production
  - Switched to Open-ended derivation production on 20.7.6.5 last week
    - Completed data15 and up-to-date data16 with 20.7.6.4
  - MC15c production with 20.7.6.4 completed last week (took about 2 weeks)
    - Marked previous 20.7.5 production for deletion (3.4PB) barring exceptions
- Reprocessing
- CRC/ADCoS report
  - Passed 2015 delivered luminosity (~4 fb^-1) last week. Now at almost 8 fb^-1
  - Still testing spillover jobs - have had issues
  - Few issues with both CASTOR and EOS last week
  - RRC-KI-T1 lost dcache DB on June 8. Last backup April 19! 131k files lost, 37k unique
  - Problem getting data from some sites after globus switch to new mode "STRICT RFC2818" GGUS:122250
  - Missing entries in dashboards due to ActiveMQ server disks full
    - Correlated with huge increase in messages from Event Index
- "New" Hammercloud blacklisting is commissioned
  - Topology and status from AGIS
  - Aggregated notifications: master vs. slave resources
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Availability & Reliability Report [20160216]
  - January 2016 - All Sites Report ATLAS
- Sites should be providing the 2015 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift