Tier1 and Tier-2 Weekly Report 2014-12-09

Tier1 and Tier-2 Weekly Report 2014-12-09

General News

ADC and ATLAS Computing News [20141209]
- ADC Weekly meeting
- MC Production
  - ProdSys-2 new validation samples finished - waiting for plots
  - Checking some configurations (mc12 AFII tags, mc14 evgen transforms)
  - Definite improvement in getting samples through the sytems but fair number of open issues in getting tasks defined, extending a task if necessary, reconstruction for container with more than one dataset, etc.
- Derivation Production
  - 13 TeV alomost completed
    - Duplicated tasks aborted, dataset containers cleaned up, actual deletion on tid datasets have not started yet.
    - AMI is not receiving activeMQ messages from Rucio yet, but should be soon
    - Need to rely on what dq2 shows for this production
    - Difficult to follow the tail of the production - did all tasks complete, which need retries etc.
    - Eight derivations (HIGGS and SUSY) had a sample-dependent software error so resubmitted
  - 8 TeV Derivation - Second round of production is basically done, tails to be taken care of
    - Fairly long discussion about the issues with the new tools and checking how production is going and where failures coming from
- Some highlights/notes from the ATLAS Facilities Jamboree
  - Rely on the fact that networks are vastly improved globally
  - Many T2 and opportunistic T3 resources can be as large as T1 (tape is the only differentiator)
  - Break the coupling between cpu and storage (or CE and SE)
  - Define ATLAS Storage pool as:
    - TAPE
    - STABLE disk storage – T1 + reliable T2 (former T2Ds) - use for primary, custodial data (including production final output)
    - UNSTABLE disk storage – less reliable T2s - use for secondary data
    - VOLATILE disk storage – unreliable T2s, T3s, opportunistic storage - don't replicate datasets, but can host LOCALGROUPDISK, Rucio cache etc
  - Run 1 had prod/analy ratio about 75/25, expect Run 2 to be more like 90/10
    - Reduce or avoid merging if possible - jobs are inherently bigger
  - Define an ATLAS Site Availablity Performance for analysis (ASAP) metric (starting in January)
    - Replace the ABCD definition
    - T2S: STABLE, ASAP>90%; T2U: UNSTABLE 90%>ASAP>80%; T2V: VOLATILE 80%>ASAP in last 3 months (to be tuned)
    - T2V may be demoted to T3 by ICB
    - Above metric may be too simple (e.g. no explicit network measurement)
  - Adding the T2 STABLE space (about 50 of current 80 T2 SE's would qualify) doubles space available for production
    - Also should reduce ANALY pressure at T1s
  - Also try and keep jobs within < 1day CPU (especially with mcore); preferably 6-12 hours, tune with JEDI and scouts - much easier to schedule
  - Eventual goal may be global "cloud" where tasks are not brokered to a particular cloud/site
    - can specialize site base on resources available
  - Could be useful to look at https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/ADCOperationsDailyReports2014 each day (written after 9am operations meeting)
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Sites should be providing the 2014 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift