Tier1 and Tier-2 Weekly Report 2017-05-30

Tier1 and Tier-2 Weekly Report 2017-05-30

General News

ADC Tech Coord Meeting - CANCELLED
ADC Weekly Meeting [20170530]
- MC Production
  - Steadily running ~250k slots. MC 80% global share
  - Essentially all MC16c, small amount of MC15c evgen (500M waiting, 200M to be submitted)
  - Well balanced between evgen and simul (~1:3)
    - MC16c evgen is in project mc15_13TeV but has campaign m16:mc16c
- Group Derivation Production
  - Issue with bigpanda showing new prod step derivation
  - 21.0.19.3 built last week
    - Launched full processing of data15 and 16 on STDM2
    - Some MC16 production for TOPQ1
    - Otherwise only minor validation production
    - Issue with muon trigger matching found (being investigated...)
  - Short term plans
    - Would like to build 21.0.19.4 tomorrow
      - Will most probably be the cache for first collision data
    - Then plan is then to move to git for AtlasDerivation
  - Containers to be used:
    - data17_13TeV.physics_Main.OpenEnded.AOD.t0prod21_v01
    - data17_13TeV.physics_ZeroBias.OpenEnded.AOD.t0prod21_v01
  - Updated the train config based on size estimates from the validation runs - 99 formats in 15 trains
- Pilot Update v69.0
  - Memory monitoring fix
  - Zip maps - create archive of output files
    - Unzip still to be implemented
  - Support for simplified payload setup (jobpars)
  - Initial support for containers
    - Pilot can execute payload and stage-in/out with Singularity if activated via schedconfig.catchall by specifying singularity options
    - Pilot uses cmtconfig/platform info to determine image file name (located in cvmfs)
    - Stage-in/out currently only with rucio and gfal_copy so far
    - To be discussed further in Valencia
  - Bug fixes for testing new versions of asetup and xrootd
  - Event Service - support for prefetcher, allow for remote input files if local SE down
  - Release asap after final tests
- CRC/ADCoS Report
  - ATLAS data taking started - accumulate 70 pb^-1 since last Friday
  - Still few transfer spikes from data consolidation
  - Remove panda logger from monitoring (to be decommissioned)
  - Some tape buffers filling up at Tier-1's already
  - Reevaluate buffering rules for T1 tape for reprocessing
- Implementing WLCG diskless sites for production
  - Based on "ATLAS recommends sites with small storage (< 400 TB) to focus hardware investment in CPUs instead of storage" ==> diskless sites
  - Still could have LOCALGROUPDISK
  - Use remote SE's for input/output
  - Similar approach to cloud sites
  - Another approach would be ARC-CE caching mechanism
  - Looking into technical feasability with couple of Romanian sites
  - These sites would focus on evgen and simul production. No analysis
  - See slides for details
    - Very small sites (<100TB): Probably affects 8 ATLAS T2 sites, but only 0.5% of available storage
    - Small sites (100-400TB): 10 sites, 3% of ATLAS storage
    - Will also look into T3 sites, especially not linked to T1/T2
  - To be reviewed at ICB
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Availability & Reliability Reports [20170328]
  - 2017 Reports - click down to find reports of format WLCG_All_Sites_ATLAS_MMMYYYY.(html|pdf)
- Sites should be providing the 2016 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift