Account Setup

Web Browser

When interacting with the POD user portal use a modern up-to-date web browser like Mozilla Firefox or Google Chrome. These web browsers can be installed on most desktop Operating System environments including: Windows, Linux and macOS. Please Note: Some features in the web portal will not work with Microsoft Internet Explorer.

Authentication using SSH Keys

You should have already created a POD portal account using your e-mail address and a password, you will also now need to setup SSH keys to access the compute resources on POD. The documentation on this page will help you with this task. Once completed you will have two sets of access credentials for POD:

  1. E-mail address and password used for logging in with a Web Browser
  2. Username and SSH key-pair used for logging in with an SSH client

SSH keys are a useful method of authentication. Instead of proving your identity with something you know, like a memorized password, you must provide something you have, a private SSH key file. Typically this is done automatically by an SSH Agent process running in the background. Some clients also support setting a path or directory location to the private file directly.

SSH Keys on Linux and macOS

As a Linux user you may already have an existing SSH key pair that you can use on POD. If so, you can easily add your public key to the POD portal Add an SSH Key form. By default, an existing public key should be located in your home directory: $HOME/.ssh/id_rsa.pub.

Create a new Key Pair

If you don’t have a key pair you would like to use on POD you can generate a new one. The -t rsa option specifies the type of key to create and the -C option is a comment field. When you manage more than one key this field can help you identify your keys. Follow the prompts, taking note of the install location of the key, usually $HOME/.ssh. When asked to enter and confirm a passphrase, choose a strong and well-secured passphrase. You will need to enter this passphrase to use your key.

$ ssh-keygen -t rsa -C "penguin@mylaptop"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/penguin/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in id_rsa.
Your public key has been saved in id_rsa.pub.
The key fingerprint is:
SHA256:sc+GdfQhQqICl5WjRKsyIjxliJuwy3u23iXpnsCRuQM penguin@mylaptop
The key's randomart image is:
+---[RSA 2048]----+
|  ..oo... .      |
|. .oo.o. o       |
|o. +o.... . o .  |
|o+o.+.   o o o . |
|OE.+    S . . .  |
|+++ o .  = .     |
|.. = o .. +      |
|  .o= +  .       |
| .+oo=           |
+----[SHA256]-----+
$ cat /home/penguin/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsvctNQODHWNCAPdONfibjIW6YcMA1xvk/rmevRiTX
s1/QD68DVQWHyO5agOMwH4ve09QezcwMSanIuhn/rGVOd/XYoDd9AnMe2t0PFRl7ubPL2wswBUeVeryU
1HtF+nLpcGSBrzvrwP/UjcpctACUjajFBtsxS/R4n/tg6rLG3u2epCi9igUeUnfsBgR+r88wVTab3eNk
zNAXyzLJJZgulWXeShwFQYddecOTCdw9XXKwS9UQzuS8XCW0KgN02eNC/YlTvJITzm+eqktgVZKoW2zJ
v9iwcUIhXyWzJEUI7rPz7Pb1gXBp3Z/gJWQYc+81TDu9BMx0oIp7KWOB7rtf penguin@mylaptop

Add new Public Key to the POD Portal

The public portion of the newly created SSH key pair is now ready to be uploaded into the Add an SSH Key form.

  1. Enter a name for your new public key in the Name textbox.
  2. Select and copy the entire contents of the ~/.ssh/id_rsa.pub file as a single line.
  3. Paste the public key string into the Key textbox.
  4. Select the checkboxes for POD MT1 and POD MT2 to install your public key on both cluster locations.
  5. Click the blue Upload New Key button to publish your private key
_images/add-an-ssh-key-form.png

Load your Private Key and Connect

To make sure that your private key is loaded into your local authentication agent please use the ssh-add -l command to list the fingerprints for each identity loaded. If you don’t see your key listed then add it using the add-add command specifying the path to your private key. If your key is protected with a passphrase you will need to enter it now.

$ ssh-add -l
$ ssh-add ~/.ssh/id_rsa
Enter passphrase for /home/penguin/.ssh/id_rsa:
Identity added: /home/penguin/.ssh/id_rsa (/home/penguin/.ssh/.id_rsa)
$ ssh-add -l
2048 SHA256:sc+GdfQhQqICl5WjRKsyIjxliJuwy3u23iXpnsCRuQM penguin@mylaptop (RSA)

Now that your public key is shared with the POD user portal and your private key is loaded into your local authentication agent, you should be able to login and authenticate onto your POD login node without entering a password. Just make sure you are using your POD username and the hostname or IP address of your login node. These will be different for each user. The first time you try to access your login node you may need to add the host to your known hosts list. Respond yes to continue connecting.

$ ssh penguin@login-12-34.pod.penguincomputing.com
The authenticity of host 'login-12-34.pod.penguincomputing.com (192.41.12.34)' can't be established.
ECDSA key fingerprint is SHA256:+1uEXIaP3SxgRNsq5qGUxBEHomh18g3H6ZakmWozeBo.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'login-12-34.pod.penguincomputing.com,192.41.12.34' (ECDSA) to the list of known hosts.

                       Welcome to Penguin Computing's POD

          Use the 'podstatus' command to show available cores per queue

                See our documentation for examples and templates
                 https://pod.penguincomputing.com/documentation

            Queue    Compute Nodes             Cores/Node  RAM/Node
            -------------------------------------------------------
            B30      2.4GHz Intel Broadwell        28       256GB
            S30      2.4GHz Intel Skylake          40       384GB

              Contact Support for Help: pod@penguincomputing.com

   * Use 'podstatus -v' to show available cores per queue based on duration
   * Use 'podstatus --help' for options to filter by queue, nodes, cores, duration

[penguin@podmt2-e0e974e ~]$

SSH Keys on Windows using PuTTY

We will walk through the steps required for Windows users to generate and install a SSH key using PuTTYgen. PuTTY is a widely used suite of ssh client programs for Windows. The private portion of the key pair is stored on the client Windows machine and the public part is uploaded to the POD cluster from the portal. For a more thorough introduction to SSH or public/private key concepts, visit OpenSSH.org. Use the following links to download and install any required software for Windows.

Creating a new Key Pair

  1. Start PuTTYgen and generate a new public/private key pair or import an OpenSSH key by selecting Conversions/Import key. When generating the key you will have to move your mouse cursor inside the window to help generate some entropy:
_images/Section_1_Image1.png
  1. After you generate or import an OpenSSH key, enter and confirm a passphrase before saving the private key:
_images/Section_1_Image2.png
  1. Save the key in a safe but accessible folder.
_images/Section_1_Image3.png

Add the Public Key to the POD Portal

  1. Inside the PuTTYgen window, right-click and Select All to highlight all of the public key. Make sure you copy the entire string including the beginning that starts with ssh-rsa and ends and the key-comment that may include the date by default:
_images/Section_2_Image1.png
  1. The public key is can now be pasted into the Key textbox on the Add an SSH Key form. Make sure to select the checkboxes for POD MT1 and POD MT2 to install your public key on both cluster locations:
_images/Section_2_Image2.png
  1. Once uploaded, you should then see the ssh key added to your account with a Success message:
_images/Section_2_Image3.png

Start Pageant (Authentication Agent)

  1. From your POD account, click Manage My Login Nodes and get the IP address:
_images/Section_3_Image1.png
  1. Start the Pageant program using the Start Menu. It should be installed as part of the PuTTY suite of applications. Once running it will appear in your Windows system dock located near the clock on the desktop. You may need to right click the Pageant icon and select Manage Keys to open the Pageant Key List window:
_images/Section_3_Image2.png
  1. Add your new private key that you just saved to Pageant:
_images/Section_3_Image3.png
  1. You should see your key in the Pageant Key List window. Please Note: Make sure this matches the fingerprint of the key listed in the POD Portal.
_images/Section_3_Image4.png

Connect to your Login Node with PuTTY

Please Note: Make sure Pageant is running!

  1. Run PuTTY using the Start Menu and enter your login node host name or IP address into the Host Name textbox. Make sure to save the session before opening.
_images/Section_4_Image1.png
  1. The first time you connect to a new login node, a security alert will pop up, click yes to accept the host key:
_images/Section_4_Image2.png
  1. A console window will open prompting you for your POD username to login as. Enter in your POD System Account username. It should then authenticate you using the private key loaded into Pageant:
_images/Section_4_Image3.png

SSH Keys on Windows Using PowerShell

Microsoft announced it was bringing an integrated OpenSSH client to Windows back in 2015. In the April 2018 Windows 10 update, the built-in SSH client is now enabled in the optional feature apps section of Windows 10. This documentation will guide you though the steps to install a native OpenSSH client to your Windows OS.

Installing Windows 10 SSH Client

The SSH client is a part of Windows 10, but it is an “optional feature” that is not installed by default. You can either install it with the Graphical User Interface (GUI) or with the Command Line Interface (CLI).

GUI Install: Go to Settings > Apps and click “Manage optional features” under Apps & features:

_images/Windows_10_SSH_Client_Image1.png

Now click “Add a feature” at the top of the list of installed features. If you already have the SSH client installed, it will appear in the list:

_images/Windows_10_SSH_Client_Image2.png

Scroll down, click the “OpenSSH Client” option, and click “Install”:

_images/Windows_10_SSH_Client_Image3.png

Restart your machine. Once you have restarted you will have a native OpenSSH client installed on your system.

CLI Install: Open an Administrative PowerShell window and run the following commands:

PS C:\Users\johnnyappleseed> Get-WindowsCapability -Online | ? Name -like 'OpenSSH.Client*'

Name  : OpenSSH.Client~~~~0.0.1.0
State : NotPresent

PS C:\Users\johnnyappleseed> Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

Path          :
Online        : True
RestartNeeded : False

Please Note: You may need to confirm that the ssh-agent is running in your services. Please follow the directions below to check.

Open a PowerShell windows and run the following:

PS C:\Users\johnnyappleseed> Get-Service ssh-agent

Status   Name               DisplayName
------   ----               -----------
Stopped  ssh-agent          OpenSSH Authentication Agent

As you can see under the “Status” the agent is not running. Run the following commands in the PowerShell windows:

PS C:\Users\johnnyappleseed> Set-Service 'ssh-agent' -StartupType Automatic
PS C:\Users\johnnyappleseed> Start-Service 'ssh-agent'
PS C:\Users\johnnyappleseed> Get-Service ssh-agent

Status   Name               DisplayName
------   ----               -----------
Running  ssh-agent          OpenSSH Authentication Agent

The “Status” is now running and moving forward the service will start automatically with windows. Restart your machine. Once you have restarted you will have a native OpenSSH client install on your system.

How to Setup SSH Keys

Once OpenSSH Client has been installed, you should have access to the following ssh commands from any Command Prompt:

ssh.exe
scp.exe
sftp.exe
ssh-add.exe
ssh-agent.exe
ssh-keygen.exe
ssh-keyscan.exe

The above OpenSSH binaries are located in C:\Windows\System32\OpenSSH\. SSH related files such as “know_hosts” and any user generated key pairs are located here is the relative %USERPROFILE%\.ssh\ or the absolute C:\Users\johnnyappleseed\.ssh\ path. We will be using the ssh-keygen.exe to create your public/private SSH key pair. This works in either a PowerShell window or a Command Prompt window, so use whichever you prefer. All the code examples from this section on are using PowerShell.

Please Note: It is recommended for security reasons that you NEVER share your private key with anyone and use a passphrase that only you know.

From a PowerShell window type:

PS C:\Users\johnnyappleseed\.ssh> ssh-keygen.exe -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (C:\Users\johnnyappleseed/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\Users\johnnyappleseed/.ssh/id_rsa.
Your public key has been saved in C:\Users\johnnyappleseed/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:VW+V0gj2XvPCwuUbT402SZZja0I8d2+AbA1+5JDcO4k johnnyappleseed@DESKTOP-16DRCKD
The key's randomart image is:
+---[RSA 2048]----+
|          7      |
|  S      4       |
|      X          |
|    l            |
|       B         |
|                 |
|     1           |
|        3        |
|           8     |
+----[SHA256]-----+
PS C:\Users\johnnyappleseed\.ssh> explorer .

A file explorer window opened up from the PowerShell prompt. You should see two file:

_images/Windows_10_SSH_Client_Image4.png

The first file (id_rsa) is your private key. The second file (id_rsa.pub) is your public key. You will need to open the public key with Notepad or some other text editor. Now copy all of the public key and go to the POD Portal and sign in. Then navigate to and select “SSH Keys”. Under the section “Add an SSH Key” you will give the “Name” of the public key and paste the public key you copied into the “Key” section. Make sure to select which cluster you want the newly generated SSH key to be added too. Then select the “Upload New Key”. You will see the new key with a banner saying something like “Success Key successfully uploaded.” Now you are ready to use SSH!

Please Note: If you use a passphrase and forget it, then you will have to create a new key pair. There is no way to recover a lost passphrase.

How to Use Windows 10 SSH Client

You can now use the SSH client by running the ssh command. This works in either a PowerShell window or a Command Prompt window, so use whichever you prefer.

To open a PowerShell window, right-click the Start button or press Windows+X and choose “Windows PowerShell” from the menu. To view the syntax of the ssh command run as seen below:

PS C:\Users\johnnyappleseed> ssh
usage: ssh [-46AaCfGgKkMNnqsTtVvXxYy] [-B bind_interface]
           [-b bind_address] [-c cipher_spec] [-D [bind_address:]port]
           [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11]
           [-i identity_file] [-J [user@]host[:port]] [-L address]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-Q query_option] [-R address] [-S ctl_path] [-W host:port]
           [-w local_tun[:remote_tun]] destination [command]

This command works the same as connecting to an SSH server by the way of the ssh command on other operating systems like macOS or Linux. The syntax or command line options, are the same.

Now before we connect to one of the POD nodes, we want to make sure your SSH key is added. Run the following command ssh-add $Your_Priavte_SSH_Key in the directory where your private SSH key is located:

PS C:\Users\johnnyappleseed\.ssh> ssh-add .\johnnyappleseed
Identity added: .\johnnyappleseed (.\johnnyappleseed)
PS C:\Users\johnnyappleseed\.ssh> ls

    Directory: C:\Users\johnnyappleseed\.ssh

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----         3/9/2019   9:18 PM            408 authorized_keys
-a----        7/29/2019  11:09 AM           1679 johnnyappleseed
-a----        7/29/2019  11:22 AM            414 johnnyappleseed.pub
-a----        7/29/2019  11:23 AM            384 known_hosts

You can see that your Private key Identity has now been added. We can also check that your identity has been added by running:

PS C:\Users\johnnyappleseed\.ssh> ssh-add -l
2048 SHA256:uDQpHUr/zF0G7P38BgU2yWMSlviPVBtX4O+QEy5PKaw .\johnnyappleseed (RSA)

We are now ready to connect into a POD node.

Please Note: Make sure the username used when connecting to any POD node, is the one you created when first setting up your POD account. Be aware that the user name is case sensitive.

We are now ready to connect into a POD node using SSH with the username johnnyappleseed. To do so, run the following:

PS C:\Users\johnnyappleseed> ssh johnnyappleseed@login-12-34.pod.penguincomputing.com

By default, the command attempts to connect to an SSH server running on port 22.

Please Note: Before you try to SSH into one of your login nodes, make sure you have setup your SSH public/private keys setup, otherwise you will not be able to connect into any of your POD nodes.

The first time you connect to any system with SSH, you will be prompted to accept the hosts key as seen below:

PS C:\Users\johnnyappleseed> ssh johnnyappleseed@login-12-34.pod.penguincomputing.com
The authenticity of host 'login-12-34.pod.penguincomputing.com (192.41.12.34)' can't be established.
ECDSA key fingerprint is SHA256:+1uEXIaP3SxgRNsq5qGUxBEHomh18g3H6ZakmWozeBo.
Are you sure you want to continue connecting (yes/no)?

You have established a connect with POD and it is asking you “Are you sure you want to continue connecting (yes/no)?”. Type “yes” to continue accessing your node as seen below:

Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'login-12-34.pod.penguincomputing.com,192.41.12.34' (ECDSA) to the list of known hosts.
                       Welcome to Penguin Computing's POD

          Use the 'podstatus' command to show available cores per queue

                See our documentation for examples and templates
                 https://pod.penguincomputing.com/documentation

            Queue    Compute Nodes             Cores/Node  RAM/Node
            -------------------------------------------------------
            B30      2.4GHz Intel Broadwell        28       256GB
            S30      2.4GHz Intel Skylake          40       384GB

              Contact Support for Help: pod@penguincomputing.com

   * Use 'podstatus -v' to show available cores per queue based on duration
   * Use 'podstatus --help' for options to filter by queue, nodes, cores, duration

[johnnyappleseed@podmt2-e0e974e ~]$

You are now connected to the remote node using SSH!

Remote File Management

Typically users will need to upload input and application files to their POD login nodes and download results and output files after a job completes. There are many ways to manage your files on POD, however we recommend using a desktop application like WinSCP or FileZilla. These applications can be installed locally, connect to your POD login node. This section of the documentation will describe how to configure and run these application on a Windows desktop machine. Use the following links to download and install any required software for Windows.

Managing Files on Your Login Node With WinSCP

Please Note: Make sure Pageant is running!

  1. Open WinSCP and put your login node into the Host name section. For File protocol, use either SCP or SFTP. Type in your username from your POD System Account and leave password blank. You can also save this login:
_images/Section_5a_Image1.png
  1. The first time you connect, a security alert will pop up, click yes to accept the host key:
_images/Section_5a_Image2.png
  1. It should then log in and authenticate you via pageant:
_images/Section_5a_Image3.png
  1. Your computer files are on the left and your POD files are on the right:
_images/Section_5a_Image4.png

Managing Files on Your Login Node With FileZilla

Please Note: Make sure Pageant is running!

  1. Open FileZilla and put your login node into the Host and your username from your POD System Account. Leave password blank and make sure port is set to 22. You can use Quickconnect or optionally save this login:
_images/Section_5b_Image1.png
  1. Your computer files are on the left and your POD files are on the right:
_images/Section_5b_Image2.png

Transferring Files Between MT2 and MT1 Storage

There are three tools for file transfer between MT2 and MT1 clusters. These tools are:

  1. SCP
  2. SFTP
  3. Rsync

The node mt2-xfer1 is used as a gateway between the locations.

Using SCP to copy files and directories

This example copies a directory called test and it’s contents from my MT2 storage to my MT1 storage:

$ scp -r mt2-xfer1:test ./
nest7grid.parms                        100%  270     0.3KB/s   00:00
namelist.input                         100% 3821     3.7KB/s   00:00
projection.jpg                         100%   42KB  41.8KB/s   00:00
namelist.wps                           100% 1738     1.7KB/s   00:00

The -r switch recursively copies the entire directory. To copy a single file, just specify that file (or a use a wildcard [*] specification)

$ scp mt2-xfer1:*.sub ./
ls-dyna.sub                            100% 1473     1.4KB/s   00:00
mpi-ring.sub                           100%  153     0.2KB/s   00:00
n.sub                                  100%   68     0.1KB/s   00:00
sing.sub                               100%  190     0.2KB/s   00:00
t.sub                                  100%  101     0.1KB/s   00:00

Note that SCP will overwrite existing files without warning. One option is to make a MT1 directory to store transferred files and then proceed with the SCP transfer.

$ mkdir mt2-files
$ cd mt2-files

Using SFTP

SFTP can be used to copy files, but not whole directories. It is useful for viewing the contents of MT1 or MT2 storage.

$ sftp mt2-xfer1
Connecting to mt2-xfer1...
sftp> ls *.sub
ls-dyna.sub                  mpi-ring.sub                 n.sub
sing.sub                     t.sub
sftp> quit

When connected to mt2-xfer1 with SFTP, use the question mark ? to get help with usage.

Please Note: See the section SSH Keys for Windows Users and use either WinSCP or FileZilla as the SFTP client on your local host to transfer files if needed (local <—> remote host).

Using Rsync

Rsync is a versatile file-copying tool. It can be used much like SCP, but it first checks to see if the file is available locally and skips the copy of the file if it is already there. This is useful to keep directories in sync in different locations.

$ rsync -v mt2-xfer1:*.sub ./
ls-dyna.sub
mpi-ring.sub
n.sub
sing.sub
t.sub

sent 106 bytes  received 2307 bytes  1608.67 bytes/sec
total size is 1985  speedup is 0.82

$ rsync -v mt2-xfer1:*.sub ./
ls-dyna.sub
mpi-ring.sub
n.sub
sing.sub
t.sub

sent 148 bytes  received 330 bytes  318.67 bytes/sec
total size is 1985  speedup is 4.15

Please Note: The second time was much faster as the files already existed on MT1. For most users SCP and SFTP should be sufficient. Know that POD support will be happy to help with Rsync if needed as it has many options.

There are manual (man) pages available for all of these commands as seen below:

$ man scp
$ man sftp
$ man rsync

Please let us know if you need assistance with any transfers at POD Support: pod@penguincomputing.com