Tier1 and Tier-2 Weekly Report 2016-04-19

Tier1 and Tier-2 Weekly Report 2016-04-19

General News

ADC and ATLAS Computing News [20160419]
- ADC Technical Coordination Meeting [20160418]
  - Minutes are live, in google docs: Minutes
- ADC Weekly meeting
- MC Production
  - MC15c submission is almost finished, about 3B events
  - Have "buffer requests" that can be run when no other higher priority tasks in system
    - Some confusion at moment as still have tasks not in WORLD queue that run evern though have lower priority - can these be moved?
    - Would like to have a "suspend/resume task" functionality
    - Still hard to get list of error for failed jobs - need to work with Panda monitor folks
- Derivations
  - 62 derivations with full 20.7 reprocessed data submitted last week - mostly done
  - 43 derivations on mc15c submitted as well
    - Many tasks in registered state - waiting for xAOD to become available
    - Many tasks in ready and submitting states - waiting for free slots
  - MC jobs filling up slots in last couple of days
- Reprocessing
  - 2015 HI reprocessing
    - 8 runs completed for HardProbes and MinBias with original cutLevel config
    - All 35 CosmicCalo runs done
    - ESDs also ready for all three streams
    - New strategy, using cutLevel=2 tasks submitted last Friday - done in 3 days
      - Seem to reduce memory footprint of jobs by 5GB (20%)
- New PanDA Logger (or PinkPanDA monitoring decommissioning
  - Migrate from Oracle with python scripts to Elastic Search and FLUME (noSQL)
  - Use Kibana for access/visualization http://bigpanda.cern.ch/eslogger/
    - Links provided in PanDA monitoring to ES
  - Producing 10M records daily
- CRC/ADCoS report
  - Steady state 200-250k job slots used. Saw spikes up to 300k with HPC resources
  - Routinely transferring 1PB/day now
  - All sites assigned to WORLD cloud - smaller sites were left in original clouds and got no production jobs since mid-March
  - Ongoing discussion on in-file metadata that refer to task configuration parameters and how to reduce them in each file
  - On-going discussion on queue attributes needed for job matching (especially memory). To round or not to round? Allow 10% overbooking of memory? To be followed up and tried on HI tasks.
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Availability & Reliability Report [20160216]
  - January 2016 - All Sites Report ATLAS
- Sites should be providing the 2015 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift