Data transfers - Push to or pull from Betabarrel User Interface via Jumphost with GUI on macOS

On macOS you can use

either SSHFS (SSH File System) to browse the file systems of the Betabarrel cluster directly in the Finder
or a dedicated SFTP client app like ForkLift2.

Both use the Secure File Transfer Protocol, which is basically FTP tunneled over an SSH connection.

SFTP with SSHFS in the Finder

SSHFS simulates a file system over a normal SSH connection and depends on the FUSE libraries, which extend the native macOS file handling capabilities via third-party file systems. Important: SSHFS is great for browsing file systems and transferring data. You may also open a remote file read-only in an application on your Mac for a quick peak at it's content, but you should not open a remote file in read-write mode and start to make changes unless you want to get surprising results; When you loose the network connection, for example due to unstable WiFi or when you put your Mac to sleep, you may loose unsaved changes or worse end up with "half modified" corrupt files. So when you want to modify a remote file, transfer a copy from the remote server to your local Mac, make changes to the copy, save them and transfer the updated file to the server.

1. Install FUSE for macOS

Download and install FUSE for macOS from https://osxfuse.github.io/
Note: when you are on macOS Catalina or newer, you may get a security warning like this:

Open the Security Preferences pane from System Preferences and

1: Open the lock to make changes and
2: Allow the system to load the Fuse for macOS kernel extensions developed by Benjamin Fleischer

2. Install SSHFS for macOS

Download and install SSHFS from https://osxfuse.github.io/

3. Download and run mount-cluster-drives app

Download and unzip the mount-cluster-drives AppleScript application.
Locate the mount-cluster-drives app, right click in the Finder application and select Open from the contextual pop-up menu:
Note: when you are on macOS Catalina or newer, you may get a security warning like this:

Select Open to continue. (If you started the app by double clicking as opposed to choosing Open from the contextual pop-up menu, the window will look similar, but will lack the Open button allowing you only to Cancel or Move to Bin)
Depending on your macOS version, you may receive a pop-up requesting permission to allow access to the Finder application:

Click Ok to allow access to the Finder.
If you want to revoke this permission or change it back to allow later on, you can do so in System Preferences -> Security & Privacy prefs -> Privacy tab -> Automation
The mount-cluster-drives app will mount the file systems of all configured clusters in a sub directory of your home dir named ClusterDrives.

You can now drag and drop files in the Finder to transfer to / from Betabarrel.
To unmount the SSHFS shares click the eject button behind the name of the share.

Technical Details

Some technical details for the curious who like to know how this works or need to debug connection issues:

The mount-cluster-drives app parses special comment lines like this:

#
# Special comment lines parsed by our mount-cluster-drives script to create sshfs mounts.
# (Will be ignored by OpenSSH.)
# 
# 
#SSHFS betabarrel_groups=bb-porch+betabarrel:/groups/
#SSHFS betabarrel_home=bb-porch+betabarrel:/home/<youraccount>/
#

in the OpenSSH client configuration file for Betabarrel ${HOME}/.ssh/conf.d/betabarrel, which was created by the ssh-client-config-for-betabarrel app from the OpenSSH client configuration instructions for macOS clients page.

The parsed comment lines result in the following mount commands:

sshfs -o "defer_permissions,follow_symlinks,noappledouble,noapplexattr,reconnect,auto_xattr,auto_cache,connecttimeout=10,volname=betabarrel_groups" \
         "bb-porch+betabarrel:/groups/" \
         "~/ClusterDrives/betabarrel_groups"
sshfs -o "defer_permissions,follow_symlinks,noappledouble,noapplexattr,reconnect,auto_xattr,auto_cache,connecttimeout=10,volname=betabarrel_home" \
         "bb-porch+betabarrel:/home/<youraccount>"
         "~/ClusterDrives/betabarrel_home"

If you have access to multiple clusters, which were configured in a similar way, you may have multiple SSHFS mounts, which are all mounted with the same mount-cluster-drives app.

SFTP with dedicated ForkLift2 client

If you prefer a dedicated Graphical User Interface that is both free and supports multi-hop SSH via a jumphost using your OpenSSH config, we suggest you give ForkLift 2 a try. You can get ForkLift 2 from the App store. Please note that there is a newer version ForkLift 3, but this one is not available from the App store neither is it free. There are various other options, but those are either paid apps or they don't support multi-hop SSH using your OpenSSH config.

To start a session with ForkLift 2:

Launch the app; You will see two file browser columns next to each other.
Both will initially show the same contents of your local home dir.

To configure one of the columns to show the contents of the cluster, click on the star symbol at the beginning of the path at the top of a column.
Click the + button to create a new favorite
Provide the connection details:
- Protocol: SFTP
- Name: bb-porch+betabarrel
- Server: bb-porch+betabarrel
- Username: your account name as you received it from the helpdesk
- Leave the Password field empty.
- Optionally you can specify a default encoding and remote path to start browsing on the cluster.
  Click the Save button to store the new favorite.
Your favorite should now be listed under Favorites.
Click on your new favorite to connect to the server and start a session.
Note that if you did not specify an explicit remote path you will start by default in your remote home dir on the cluster, which may be empty.

Back to operating system independent instructions for data transfers