Tier1 and Tier-2 Weekly Report 2014-11-11

Tier1 and Tier-2 Weekly Report 2014-11-11

General News

ADC and ATLAS Computing News [20141111]
- ADC Weekly meeting
- Rucio Migration
  - Still planning switchover for production system Nov 24-29 - planning 12 hour downtime
  - Some metadata changes needed beforehand - been agreed to by ProdSysII and PanDA
    - Requires database intervention
    - Planned for Thursday morning - only Rucio datasets will be affected (unavailable) for approx. 30mins
  - DQ2 client testing
    - Few backward incompatibilities already reported by HC team and fixed
    - Doug Benjamin is now testing DQ2 client from end user POV
    - If an incompatibility cannot be fixed it will be mentioned in migration FAQ
    - The migration FAQ will be linked from the top of DQ2ClientsHowTo
  - DQ2 to Rucio migration
    - Implemented on DB level (PLSQL procedure)
    - Moves dataset definition from DQ2 (central catalog) to Rucio, including metadata
    - Creates rules+locks in Rucio for DQ2 replicas
    - Being tested on intergration instance of Rucio
    - First real datasets will be migrated Wednesday morning (old logs)
    - Ops will check that migrated datasets are visible and have proper replicas/rules/attributes
    - Batch migration will start Thursday or Friday
    - Further migration steps in https://docs.google.com/document/d/1CKeF0QWN01fAttzrcbHqwlL4wEjTyEZ82Us9ejakOKY
  - TAPE staging
    - Transferring to T1 tape (with FTS3) is tested (with PIC)
    - Stage-in requests + transfers fixed last week
    - More testing needed - Vincenzo will ask T1s to remove few files from disk cache, so we are sure the whole chain from tape to disk is tested for all T1s
  - Accounting
    - The dumps from Rucio to accounting dashboard are almost ready
    - Still some hadoop technical issues to resolve
    - Plan to run the dumps from DQ2 and Rucio in parallel for one week to compare the results
  - Popularity
    - Trace records are now reported from pilot to Rucio and also DQ2 - pilot not in production yet
- Status of Derivation Production and ProdSys2 Commissioning
  - 8 TeV production launched October 20th. Still active tasks in system after 3 weeks
    - Were some grid issues in the first week but even with single carriage train, this seems too long - some discussion and investigation by GDP
    - 13 TeV production stopped after very slow read of xAODs (about a factor 20-100 slower than for 8 TeV files, for both ROOT and athena) due to the ROOT version in 19.1.4.2, merging took very long (up to 24 hours for 1 GB of output) in validation tasks.
      - Users have been informed to use newer ROOT releases (Root-5.34.22 or newer) as provided in the latest Analysis releases starting from 2.0.16 in analyzing xAODs directly and not wait for derivations for the ATLAS Physics Workshop next week
      - Tasks were slow to start, one problem identified, missing installations in ND cloud. GDP and the cloud support are investigating.
      - Will need a new cache for the ROOT patch. AODFix+trigger fix is also planned to be put in for the upcoming production - to be launched this week.
    - Last week of November is fine for decommissioning ProdSys1
      - Derivation production only implemented in ProdSys2
      - Legacy group production moved to ProdSys2 on Oct. 24th, no problems reported so far
CA Cloud Squad Actions
- User DaTRI requests approved / processed (last week)
Open Issues and Ongoing Discussions
- Sites should be providing the 2014 storage pledge.
  - REBUS Site Capacities
  - REBUS Federation Resource Pledges
- ATLAS Canada HammerCloud Tests
WLCG and General Computing News
- Security News
- gLite/EMI News
ROC_Canada News
- Current shift