Tier1 and Tier-2 Weekly Report 2018-02-20

Tier1 and Tier-2 Weekly Report 2018-02-20

General News

ADC Weekly Meeting [20180220]
- INFN Status
  - Power restored on one line (1.4MW) - provides half the conditioned power. Other line ~month
  - Cabling for 2017 storage tender going on now
  - All ATLAS disk-only systems recovered - ready for production
  - ATLAS tape buffer already in production
    - Tape recovery by Oracle ~1 a day - few months
    - ATLAS RAW data already replicated
  - Farm services on CNAF (50 kHS06), 24 kHS06 in Bari
  - Installing WNs at CINECA next week
    - 170 kHS06, CentOS7, Singularity
  - Start working on getting all services tested and online
- Group Derivation Production
  - SharedWriter is being tested - at least one format has issues with metadata
  - MemoryMonitor issue still unresolved
  - Latest cache 21.2.16.0 built yesterday only for TAU group
  - 3.5PB marked for deletion very soon
  - Discussion of shares - current: GP_PHYS 10% (mostly data), GP_GROUP 3% only MC. Maybe go 50/50
- MC Production
  - MC16d bulk production as complete as can be at this point (except INFN inputs)
  - Recovering from rucio issues still
  - Fed two more large MC16e simul to P1
  - Remainder of large BPHYS sample to come (several 100M) - could do as ES, clarify with PMG
  - Not much pile up running, some waiting on INFN inputs
  - List of user requests slowly going down
- Reprocessing
  - First part of data17 reprocessing
    - Handful of jobs (~12) in period D and E needed to be obsoleted and retried (mainly merge steps)
  - [RUNNING] Period B 15+1 runs to fix alignment errors in first reprocessing pass (no fork after first event)
  - [RUNNING] Reprocessing of period F, H, I and K queued/started.
- CRC/ADCoS Report
  - Running usual 250-300k slots
  - Had low number of jobs due to less MC
  - Issue with unacknowledged transfers in rucio for a period
  - Heavy Ion overlay jobs overloading CERN and Lyon frontiers
  - Discussion about CRC time spent on inefficient sites and investigation
  - Some T0 issues with CERN-PROD - being followed up
  - Still investigating issues with INFN-T1 transfer failures via RAL FTS
  - No CRC shifter for next 3 months - encourage signup
- ATLAS Data Ocean Project
  - Industry R&D Collaboration with Google Cloud Platform
  - Integrate GCP with ADC
  - Allow ATLAS to explore different computing models for HL-LHC
  - Allow ATLAS user analysis to benfit from the Google infrastructure
  - Give Google real science use cases to improve their cloud platform
  - whitepaper: https://cds.cern.ch/record/2299146/files/ATL-SOFT-PUB-2017-002.pdf
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Availability & Reliability Reports [20170328]
  - 2018 Reports - click down to find reports of format WLCG_All_Sites_ATLAS_MMMYYYY.(html|pdf)
- Sites should be providing the 2017 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift