Large Data Transfer for Helmholtz
The HIFIS Transfer Service (HTS) enables the users at the Helmholtz centres to easily share large data sets in collaborations.
How is this implemented?
To achieve this, WebFTS is used as graphical frontend. It enables the user to commission data transfers with CERN’s backend service FTS3.
FTS3 controls transfers between an active and a passive endpoint at the involved centres. The main advantage of using FTS3 compared to commonly used FTP or SCP based transfers is that no active involvement by the user is needed after commissioning the transfer. Hence, users can schedule the transfer, e.g. overnight, and have the data ready for use later.
What are the preconditions to use HTS?
In order to participate in HTS, a centre needs to:
- be connected to Helmholtz AAI (see list),
- install a dedicated endpoint that is capable of communicating with FTS3 either actively or passively, and
- of course, provide sufficiently large, performant and available storage to which the endpoint has access.
At the time of writing, HIFIS offers a passive endpoint solution in form of an Apache webserver for installation at a centre. Configuration and installation instructions can be found in the Technical Documentation.
An active endpoint that can be used as a relay point between two passive endpoints is also available at DESY.
How to use HTS?
If the preconditions are fulfilled, you need to do the following steps to use HTS:
- know the URLs of the endpoints you want to use (available from the centre if they are participating in HTS),
- access WebFTS in your browser,
- log in via Helmholtz AAI (upper left corner),
- click on ‘Submit a transfer’,
- enter the URLs of the endpoints between you want to transfer data,
- select the data you want to transfer on one endpoint,
- click on one of the arrows between the directory views according to the direction of the transfer.
After submitting, you should see a banner stating that your transfer has been submitted successfully. You can check the transfer status by clicking on ‘My jobs’.
Support
If you want to set up data transfers for your needs, or in case of any questions and queries, please don’t hesitate to write us.