Data transfers - How to move data to / from the dedicated data transfer server

Firstly and independent of technical options: make sure you are familiar with the code of conduct / terms and conditions / license or whatever it is called and that you are allowed to upload / download a data set! When in doubt contact your supervisor / principal investigator and the group / institute that created the data set.

The Nibbler HPC cluster features a dedicated data transfer server nb-transfer.hpc.rug.nl, which can be used to exchange data with external collaborators, that do not have a regular cluster account with full shell access. This dedicated data transfer server can only be used with guest accounts, which can transfer data using

  • SFTP protocol on port 22
  • rsync protocol on port 443

data-transfers

  • R1: Cluster user uses their regular account with SSH key forwarding enabled to login to user interface server nibbler via jumphost tunnel
  • R2: Cluster user uses guest account to transfer data from nibbler to nb-transfer.hpc.rug.nl or vice versa.
  • G1: External collaborator uses guest account to transfer data to/from nb-transfer.hpc.rug.nl.

Arranging a guest account for data transfers

Procedure for cluster users who already have a regular account

  • Ask your external collaborator(s) to create a public-private key pair using these instructions for Windows clients or for macOS/Linux/Unix clients.
  • Ask your external collaborator(s) to send only their public key to our helpdesk.
  • Contact our helpdesk and request a temporary guest account for you and your external collaborator.
  • You do not need to include your own public key(s) as we already have those.
  • Motivate your request and specify
    • the name and affiliation of your collaborator,
    • the name of the project for which data will be exchanged and
    • for how long you will need the guest account.
  • We will assign a temporary guest account for your data transfer and link both your public key(s) as well as the public key(s) of your collaborator(s) to the same guest account.
  • You can now transfer data from/to nb-transfer.hpc.rug.nl using the guest account and your private key.

Procedure for external collaborators

  • Your contact (with a regular cluster account) will request a guest account from our helpdesk.
  • Your contact will ask you to create a public-private key pair using these instructions for Windows clients or for macOS/Linux/Unix clients.
  • You will send only your public key to our helpdesk.
  • We will link your public key to a guest account and notify you when the guest account is ready.
  • You can now transfer data from/to nb-transfer.hpc.rug.nl using the guest account and your private key.

Using the guest account to transfer data to/from nb-transfer.hpc.rug.nl