Tier1 and Tier-2 Weekly Report 2014-10-07

Tier1 and Tier-2 Weekly Report 2014-10-07

General News

ADC and ATLAS Computing News [20141007]
- ADC Weekly meeting
- Rucio Status
  - Panda integration
    - First successful job was completed last Friday (all data in/out handled by rucio)
    - http://bigpanda.cern.ch/job?pandaid=2277925230
    - After a lot of iteration between Panda and Rucio developers
    - Repeat the exercise for several thousand jobs using a validation dataset which only exists in Rucio: mc14_valid.110401.PowhegPythia_P2012_ttbar_nonallhad.evgen.EVNT.e2928_tid01592496_00
  - Stress test
    - Now using RAL and CERN FTS3 pilot services - transfers split randomly 50/50
    - Interaction with FTS3 has been improved a lot - sustained ~1.5M files/day over last 5 days
    - Deletion rate also sustained
    - Before that problems with FTS3/slow polling for transfer status
    - Aim to run for 10 days continuously at this rate
- T1 HOTDISK decommissionng
  - T2 HOTDISKs are long gone
  - Datasets in T1 HOTDISK are not really hot
    - Most of the files are also stored and accessed via CVMFS
    - If necessary, can serve data from DATADISK
    - Progress being recorded in https://its.cern.ch/jira/browse/ATLDDMOPS-4741
    - CERN-PROD_HOTDISK will be kept - has been moved from CASTOR to EOS
  - Plan
    - Replicate ddo and cond data on DATADISK - DONE
    - Change AK47 to use DATADISK - DONE
    - Blacklist HOTDISK for read/write - DONE in AGIS
    - Wipe two endpoints and check it does not cause a problem - will be done today for FZK-LCG2 and IN2P3-CC
    - Wipe all T1 HOTDISKs
    - Disable endpoints in AGIS and mark Rucio RSEs as deleted
    - Remove the endpoints from site services and deletion services
    - Ask T1s to remove ATLASHOTDISK space token
- ProdSys2 Commissioning from DPD Production
  - Derivation production new in ProdSys2: AtlasDerivation-19.1.1.5
  - Major milestone achieved last week with full chain from AOD->DAOD making then followed by JEDI-internal merging in the same task (single output merging, namely single carriage trains)
  - Multiple-output merging with DAODMerge_tf.py (new trf) is being tested. Next round of derivation will be done with trains.
  - Using DEFT for standalone productions (not train): https://prodtask-dev.cern.ch/prodtask/dpd_request_create/
    - Interface is ready except for "destination" part for replicating output to group disk
    - The new pilot supporting the dst: syntax is ready and running in US cloud for testing
      - Group contacts will only specify "Destination token" and DEFT will look up the "Destination siteID" and set accordingly so that group contacts do not worry about siteID
    - Goal is to get replication part done/tested this week
    - Working with JIRA team on ambitious goal to migrate from Savannah within next ~week
  - New cache AtlasDerivation-19.1.1.6 today for process period-B data and Run1 MC samples
- MC Production - ProdSys2 Report
  - All sim (last month): 341M events (108M in past week, 15M/day)
    - Not enough simulation for a while, new medium requests arrived recently
  - All digi+reco (last month): 842M events (177M in past week, 25M/day)
  - About 100M(sim)/200M(digi+reco) events to be processed
  - About 150M(full)/100M(AFII) sim events to be submitted
  - DC-14 Sim 13 TeV
    - Simulation with new geometry and using mcore
    - 670M of 780M finished. Requests growing
  - DC-14 Digi+Reco 13 TeV
    - 19.1.1.5 being used for reconstruction (mostly rtag r5787)
      - Still no merging tag until 19.1.1.4 fully deployed on grid
      - Existing caches have a memory leak, fixed in 19.1.4.X
  - Estimate current requests will keep the grid quite busy for 7-10 days
- Connectivity performance of T2 sites
  - Use DDM SSB large and medium-file sonar metrics for input: (485, 484)
    - medium https://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistory?columnid=10103
    - large https://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistory?columnid=10097
  - Merged into single metric for monitoring and ABCD decisions: https://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistory?columnid=10104
    - green - T2D with sufficient connectivity
    - blue - T2 with sufficient connectivity
    - yellow - T2 with not sufficient conn.
    - red - T2D with not sufficient conn.
  - Optimization and documentation still necessary
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Sites should be providing the 2014 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift