TWiki> AtlasCanada Web>GridActivities>ATLASCanadaTier-2Operations>T2WeeklyReport20131217 (2013-12-17, LeslieGroer) EditAttach

Tier1 and Tier-2 Weekly Report 2013-12-17

General News

  • ADC and ATLAS Computing News [20131217]
    • ADC Weekly meeting
    • Rucio renaming deadline
      • Looking to switch from LFC to Rucio Feb 1
      • 64% endpoints done with 15% more "soon" (mostly US)
      • On 1st of February, all sites that have not been "renamed" won't be included into DDM/Rucio
        • Exceptions can be discussed for sites with migration in progress with clear plans agreed beforehand with DDM/Rucio teams
        • For StoRM sites, we notice that the performance of the current implementation of WebDAV is not good enough. StoRM developers are working on an improved version, but it might be tight to have it deployed on all StoRM sites by February 1st. Those sites will be contacted directly by DDM ops
    • AMOD Report
    • MC Production Plans
      • Dips in production again this week due to lack of requests
      • Xmas Break
        • Deadline Thursday for Standard Requests. Do not expect large requests
          • Expect about 70-100M events
        • pre-DC14 campaign
          • Validation still ongoing (including multicore). Decision Thursday.
          • Expect 80M signal samples, 70M single particles
          • Also need to generate pileup (10M + 10M) samples
          • Typical production rate is around 20M/day (sim) and 20 M/day (pileup)
      • If the total amount is around 250 M, we would be finished in less than couple of weeks. We will know more details on Friday
    • Reprocessing Plans
      • Reprocess periods B and D
        • TRIUMF has 130 datasets (51TB)
        • Plan to run in automatic mode with MCP, no data preplacement
      • Multi-core
        • 200M events for DC14 preparation/validation - 2M jobs
        • Previous testing: ~2500 job slots (20k cores)
        • Message to sites: Configure MCORE queues fully dynamically if experienced with it
          • Static allocation otherwise: 50% of production resources for T1 and big T2s
        • No massive single-core production (apart from reprocessing). Single-core only sites will be mostly empty.
        • https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/AtlasMulticore
          • Procedures described for SGE and SLURM. Torque/Maui sites not as clear
    • Group Production Plans
      • Plan to launch NTUP_IDVTXLUM production today/tomorrow (needed urgently for the 2012 luminosity paper)
        • input data: data12_8TeV.*.calibration_PixelBeam.merge.AOD located at CERN-PROD_DATAPREP, CERN-PROD_TZERO
        • 703M events, no more than 2.5 TB output, to go to CERN-PROD_DATAPREP, expect to complete in couple of day
      • Plan to launch NTUP_COMMON v2 production on Thursday
        • Cache is being built today for bug fixes, submit few validation samples later today
        • 270kb/evt for data and 340kb/evt for MC.
        • No partial processing of AOD to preserve lumiblock boundaries. Jobs run times are long, some exceeding 1+ days
        • Run on data12 for Egamma/Muons/jetTauEtmiss, expect ~same data volume as in v1 production
          • 1.6 PB for two copies on datadisk. For MC it will be much reduced from v1 production (3.6 PB for 2 copies)
      • Train Production
        • Still evolving. Testing in ProdSys1 is postponed to January as a new AtlasProduction cache is needed to produce output file names compatible with ProdSys1
    • DDM Space Overview
      • Free space in DATADISKs: T1: 3.5 PB (1.3 PB secondary), T2: 3.7 PB (5.4 PB secondary)
      • From planned activities
        • MC production: 250M events (100TB)
        • mc11d: output ~70 TB
        • Reprocessing step B: output ~50 TB
        • FTK: < 40TB
        • NTUP_COMMON v2 (< v1: 5.2 PB in total, 1.7 PB for T1, 3.5 PB for T2)
      • Need to clean up some primary datasets
        • Unused datasets (for 6 months on T1, 12 months on T2)
          • Datasets untouched for 6 months and with TAPE copies are periodically removed
        • Datasets without TAPE copies: 2.5PB on T1, 1.8PB on T2
      • NTUP_COMMON on disks:
        • Policy is to have 1 T1 and 2 T2 replicas
        • These are not cleaned with the 6-months untouched policy
        • Some reduction in v1 will need to be done as v2 produced
          • Comment: v2 mainly for b-group so v1 still used by most physics groups
        • NTUP_COMMON migrated to TAPE after 30 days
      • Currently planned outputs fit in the free space (i.e. no crisis over Xmas)
      • If possible: Production of NTUP_COMMON v2 should go in parallel with relaxing replication policies of NTUP_COMMON v1

  • Security News [20120801]

  • ROC_Canada News [20130625]
    • Current shift - Erming Pei June 24-28, 2013
    • Next Week - Leslie Groer

  • WLCG and General Computing News [20120911]

SSB Results

SSB PanDA Test Queue Availability (month) - this metric, measured for the analysis queues, defines the site status for data preplacement as alpha (T2D and avail > 90%), bravo (T2 and avail > 90%) or charlie (< 90%).

SAM Results

  • site service status via Gridview - Please check your site

APEL Accounting

APEL Publishing Status

APEL Accounting Results

APEL Synchronization Test Links

Site Reports

Alberta

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

SFU

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Victoria

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Toronto

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

McGill

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

TRIUMF

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Australia-ATLAS

Recent Issues

  • summary of this weeks Savannah bugs and GGUS tickets for site:

Outstanding Issues

Production

Summary

  • NOTE: ARDA ProdSys Dashboard has stopped being used by ADCoS team due to information updating issues with the interface. The historical job usage information is to appear separately within ARDA and is being validated. This will be used in the future for information presented here.

  • Within the CA Cloud and Worldwide (Panglia View):

-- Leslie Groer - 2013-12-17

Comments

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2013-12-17 - LeslieGroer
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2026 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback