CA Cloud Site Network Information

Please document your site network configuration as to how it relates to FAX and WebDAV. Useful information that belongs here should trace traffic limitations from a WN requesting data from local or remote sites or remote sites requesting data from your SE through xrootd or http via proxy servers.

Please keep in mind that FAX and WebDAV, if one includes overflows, can be 2 way traffic so it is important to understand what the WNs face when they go off-site to fetch data via xrdcp, root-open, curl or davie-http (i.e. copy to scratch or streaming).

Summary

FAX:

TRIUMF

To access storage over WebDAV and FAX protocols at TRIUMF ATLAS T1 data center we setup dedicated proxy server with 2G network connection. Proxy server is accessible from the following WAN connections: LHCOPN, LHCONE, ORAN and WestGrid. GPN (General Public Network) access to the proxy is not allowed.

In case TRIUMF worker nodes need to access external data over any protocols traffic will be sent though the NAT server which has limited 1G connection.

Australia-ATLAS

Most worker nodes have only 1Gb connections, but some have 10Gb. Worker nodes have direct access to hosts on Research network (no commercial network access though). All disk servers have at least 1x 10Gb ethernet connection, with most having 2x 10Gb in LACP bonded mode. WAN connection to site is 10Gb. No proxy server set up for WebDAV or FAX.

Australia-NECTAR

This is a cloud computing site. Australia-NECTAR runs on the same NREN network as Australia-ATLAS (i.e. 10Gb connectivity to the world), so yes, it could technically do overflow jobs. Only reason why I hesitate is that only the Australia-ATLAS network is tuned for accessing data from the LHC sites, and only the Australia-ATLAS network will eventually end up connected to LHCONE.

CA-MCGILL-CLUMEQ-T2

The Woker-Nodes connect to the outside via NAT, which has a 1GB connection to WAN connection and can reach both the LHCONE and general Calcul Quebec network link to RISQ and CANARIE for the general network . The Wokernodes mount GPFS thus file access is direct. The xrootd/cmsd/WebDav all run on the stoRM node which has a 10GE connection to WAN and also directly mounting GPFS. File fetches are direct from the filesystem unless a grid enabled protocol is defined where then file fetches are via Woker-Nodes to stoRM-node's 10GE external but within site routing. Each Woker-Node has a GigE connection. For the NAT we have not seen any network saturation issues looking at our monitoring and well able to accommodate more traffic at the current configuration.

CA-SCINET-T2

The WN's connect via DDR IB to a NAT server, but that NAT server only has a 1 GbE connection to the WAN.

The xrootd dCache server and the xrootd/cmsd run on dCache door nodes which have 10GbE connections to the WAN, and are connected via DDR IB to the internal network to access the dCache pool servers. The xrootd/cmsd run on the same node as the webDAV door. Transfers should be redirected to the 6 dCache pool servers, each with a 10GbE connection to the WAN.

The site is connected to the WAN via 2 x 10GbE connections. There are two 10GbE uplinks for the switch that serves most of the dCache components.

CA-VICTORIA-WESTGRID-T2

Worker nodes have 4X QDR Infiniband connections with a 10:1 blocking factor. Worker nodes have internet access via a 10Gb gateway server.

The pool nodes have 10Gb ethernet and Infiniband. The SE head node has 1 Gb ethernet, and xroot and webdav doors. The xroot and webdav transfers are redirected to pool nodes.

SFU-LCG2

All servers and nodes are interconnected by the DDR Infiniband (except the old 31 HP worker nodes and the xrootd downstream redirection server). There is also a 1 Gb internal ethernet connecting everything. All servers' external network interfaces are 1 Gb. The core IB switch has a 10 Gb connection to LHCONE. No NAT service on the core switch.

Worker nodes have internet access via a NAT server on the CREAM-CE bugaboo-hep (only 1Gb external interface). The SE pool nodes (12 of them) have 1Gb external interfaces and the IB. Each pool node is running a gridftp door. A Webdav door is running on the SE headnode wormhole. A Xrootd door is running on the other SE headnode bugaboo-se2. A xrootd downstream redirection server (snowpatch) has only a 1 Gb external interface, 1 Gb internal ethernet interface and no IB.