TWiki> AtlasCanada Web>GridActivities>ATLASCanadaTier-2Operations>T2WeeklyReport20161122 (2016-11-22, LeslieGroer) EditAttach

Tier1 and Tier-2 Weekly Report 2016-11-22

General News

  • ADC Technical Coordination Meeting [20161121]
  • ADC Weekly Meeting [20161122]
    • MC Production
      • Contention with data reprocessing this week
      • 360M events in queue, about 10 requests in approval step
      • Still expect many MC15c requests for paper preparation
      • Issues with current method of "boosting" nearly completed tasks (90% done) by giving them the highest priority of 900
        • Interfering with real urgent tasks and about 40% of ~500 tasks have this priority set
        • Some suggestions discussed: wait until 95% (or more done) before boosting and don't boost so high
      • Meeting next week about MC16 workflow
      • Requests for Titan being prepared
    • Group Derivation Production
      • Number of issues
      • LAr error flags missing in 2015 data - full processing of data15 for all formats launched
      • Muon TRT problem - full rerun of data15/16 for HIGGxDx, JETM3, SUSY2 and SUSY7 launched last week, EXOxx to be launched today
        • There will need to be MC for these samples too
      • PC decided not to go back and fix issue with silently skipped files
      • Better isolation for photons and muons proposal needs an AODFix
        • Ready to go in
        • After validation, plan a full scale production over Xmas
      • Test of fast merging - need new test as arguments were not being passed to the pmerge job (fixed)
    • Reprocessing
      • 2015/16 ALFA runs (25 total) rel 20.7.8.6
        • physics_MinBias stream completed
        • calibration stream being produced by splitting into 0.5M event chunks
      • 2016 physics_BphysDelayed stream, 117 runs, rel 20.7.8.9
        • Submitted last week, 77 done, 59 merged AOD datasets available
        • Fixing a few issues along the way
      • DESDM_ZMUMU/EGZ production for validation of release 21 alignment underway, 5 runs rel 21.0.8
      • Release 21 validation
        • Large ESD still biggest problem
        • Memory leak (1MB/event) in ESDMerge halted 21.0.9 tasks for now - ticket open
        • Looking into Run 1 crashes in same release
    • CRC/ADCoS report
      • Problems with access to data on tape last week
        • Reprocessing of bphysdata running directly from tape endpoints without prestaging
        • Tier0 staging at CERN for 2015/2016 rreprocessing campaign
        • Users submitted large download requests using r2d2. Approval step needs to be added in r2d2 service.
        • Users running analysis jobs with large amount of input on tape. A cap of 1 TB/day per user has been put temporarily.
        • Low staging efficiency at CERN due to large backlog. Fixed by deletion of obsolete requests today.
        • Additional action to decrease weight for selecting CERN as a source of data on tape.
      • HC has been modified to use direct file specification and not ask rucio
        • During configuration there was a period of HC tests failing on access to input files (Nov 17)
      • Transfers to TAIWAN have been not running for >3 days.
        • Sites with large numerof jobs in transferring state reported by ADCoSshifters,as well as some stuck files in export from Tier0 to Grid were all suffering from transfers to TAIWAN stopped
      • Reported at WLCG meeting that CVMFS service at TAIWAN is unreliable. Concurred by other experiments. To be followed by WLCG.
    • Pilot News (67.0) - release tomorrow?
      • Verifying remaining proxy life time
        • Currently every 5 minutes, req. minimum 10 minutes remaining
        • NB. Switched off in case proxy verification is not used.
      • Bug fix for nightlies in 'new' setup function
      • Event service
        • Setting 'pilot_killed' substatus error instead of 'pilot_failed' for kill signals
        • Other AthenaMP fixes
      • Updates to old site movers
        • OS site mover correction - closing S3 connection
        • GFAL2 site mover fix - Explicitly adding double slashes to "file:/" to avoid issues on one (CC7) site
      • Updates to new site movers
        • Several updates to new mv site mover, including creation of output.list, mv instead of copy, code cleanup
        • Storm site mover using webdav etags instead of broken lcg-gt command
        • Extended xrdcp stage-out error handling
          • Now considering 'Unable to checksum' error (as well as 'query cksum is not supported') for sites that do not support CRC calculation on the fly
        • Added gsiftp to the list of supported protocol schemes in lcgcp site mover
    • CRC/ADCoS report
  • CA Cloud Squad Actions
  • Open Issues and Ongoing Discussions
  • WLCG and General Computing News
    • Security News
    • gLite/EMI News
  • ROC_Canada News

ABCD Determination and Monitoring

SSB Results

SAM Results (based on VO tests)

APEL Accounting

APEL Synchronization Test Links

Production and Usage Status

-- %USERSIG{LeslieGroer - 2016-11-22%

Comments

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2016-11-22 - LeslieGroer
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2026 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback