You are here: CLASSE Wiki>Computing Web>GlobusDataTransfer (22 Nov 2022, dab66)Edit Attach
Tags

Globus @ CLASSE

Globus is a high-speed file transfer system that was originally developed for Grid computing applications. Please note that Globus is our recommended data transfer solution for both on and off-site visitors and users. However, note that Globus will not work from the Cornell-Visitor wireless network. Please see our wireless recommendations for more information.

Two separate login steps will be required:
  • Log into Globus (see below) to access Globus endpoints around the world (not only at CLASSE).
  • Log into your CLASSE account to access CLASSE's Globus endpoint.
    • To request a CLASSE account, please ask your supervisor to fill out our NewUserForm.
    • If you are a CHESS user, please contact your beamline scientist or the CHESS Users' Office.

You can use globus to copy data remotely from CLASSE central storage to your local computer, following the instructions below.

1. Log into Globus

If your home institution is Globus member, you can use your credentials with that institution. Otherwise, you can create a personal Globus ID:
  1. Browse to https://www.globus.org
  2. Click on "Log In" in the upper-right corner.
    • If your home institution is a Globus member, use the scroll-down menu to find your organization.
      • For example, if you have a Cornell NetID, you can select "Cornell University", click on Continue, and then login with your Cornell NetID.
      • If you have an account with some other institution, like Argonne National Lab, you can select the entry for that institution.
    • If your home institution is not a Globus member, you can create your own personal account.
      • Click on "Globus ID" and sign up for a new Globus ID, or use one that you created previously.

2. Install Globus Connect Personal (if necessary)

To set up your computer as a Globus endpoint, download and install Global Connect Personal. This needs to be done only once:
  1. After logging into https://www.globus.org, click on "Endpoints" in the left sidebar.
  2. Click on "Create a personal endpoint" and complete the two steps described on that page.
  3. Put somewhere safe the multi-character Globus Setup Key which Globus generates for your Personal Endpoint. It'll be needed whenever you restart the Globus Connect Personal software.

For detailed instructions with screenshots see here: https://www.globus.org/globus-connect-personal.

3. Connect to a CLASSE endpoint / Collection

  1. Click on "File Manager" in the left sidebar.
  2. Enter classe in the Collection search box. You should see the endpoints in the table below.
  3. Select a CLASSE collection.
  4. Click on "Continue" if prompted to authenticate with the globus.classe.cornell.edu domain.
  5. Click on "Link an identity from CLASSE Globus Server (globus.classe.cornell.edu)"
  6. Login using your CLASSE username and password
  7. Click on "Continue" to link your username@globus.classe.cornell.edu identity to your Globus account.

Collection Name Path Usage
CLASSE /home/username/ Defaults to your home directory, but from there you can browse to or type in the path to any /nfs/ filesystem.
CHESS Raw /nfs/chess/raw/ All raw data from CHESS data transfers
CHESS Current Data /nfs/chess/raw/current/ All raw CHESS data from the current run.
CHESS Aux /nfs/chess/aux/ CHESS data transfers
CHESS Test /nfs/chess/raw/test-download/ Testing data transfers

4. Connect to your personal endpoint

  1. Click on "Transfer or Sync ..." in the right-hand column.
  2. Click the other "Collections" box (the second box to the right of “Collection”. It probably just says “Search”), and then the "Your Collections" tab.
  3. Select the personal endpoint you set up in Step II above.
    • Note that Globus Connect Personal must be running on your computer.

5. Transfer files

  1. Navigate through the folders/directories on both endpoints, and use the "Start" buttons to transfer data back and forth as desired.
  2. To monitor the progress of a data transfer, click on "Activity".
  3. You will receive an email after each transfer has completed.

The CHESS Test endpoint contains two folders with sample files for testing downloads:
  • small contains three small text files.
  • large contains one binary file with a size of 2 GB, which may be used to measure transfer speed.


Notes

  • Only Globus Plus users may transfer files between two personal endpoints: https://www.globus.org/providers/provider-plans#globus-plus.
  • In the web interface, "drag and drop" doesn't work. One must use the transfer arrows to put the files into a transmission queue.
  • The web interface does not automatically update the contents of the source and destination directories. One must manually click on the "refresh" buttons to see the results of a transfer.
  • The Globus command-line interface (CLI) allows you to write scripts to automatically transfer any new or updated files at specified intervals, script file transfers as part of your data collection, etc. Instructions on enabling and using the Globus CLI are available at https://docs.globus.org/cli/. Note that to enable ssh access, you must login to https://www.globusid.org (not globus.org).
  • Please report any problems by opening a CLASSE-IT Service Request.
Topic revision: r32 - 22 Nov 2022, dab66
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding CLASSE Wiki? Send feedback