When interacting with the POD user portal use a modern up-to-date web browser like Mozilla Firefox or Google Chrome. These web browsers can be installed on most desktop Operating System environments including: Windows, Linux and macOS. Please Note: Some features in the web portal will not work with Microsoft Internet Explorer.
Authentication using SSH Keys¶
You should have already created a POD portal account using your e-mail address and a password, you will also now need to setup SSH keys to access the compute resources on POD. The documentation on this page will help you with this task. Once completed you will have two sets of access credentials for POD:
- E-mail address and password used for logging in with a Web Browser
- Username and SSH key-pair used for logging in with an SSH client
SSH keys are a useful method of authentication. Instead of proving your identity with something you know, like a memorized password, you must provide something you have, a private SSH key file. Typically this is done automatically by an SSH Agent process running in the background. Some clients also support setting a path or directory location to the private file directly.
SSH Keys on Linux and macOS¶
As a Linux user you may already have an existing SSH key pair that you can use on POD. If so, you can easily add your public key to the POD portal Add an SSH Key form. By default, an existing public key should be located in your home directory:
Create a new Key Pair¶
If you don’t have a key pair you would like to use on POD you can generate a new one. The
-t rsa option specifies the type of key to create and the
-C option is a comment field. When you manage more than one key this field can help you identify your keys. Follow the prompts, taking note of the install location of the key, usually
$HOME/.ssh. When asked to enter and confirm a passphrase, choose a strong and well-secured passphrase. You will need to enter this passphrase to use your key.
$ ssh-keygen -t rsa -C "penguin@mylaptop" Generating public/private rsa key pair. Enter file in which to save the key (/home/penguin/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in id_rsa. Your public key has been saved in id_rsa.pub. The key fingerprint is: SHA256:sc+GdfQhQqICl5WjRKsyIjxliJuwy3u23iXpnsCRuQM penguin@mylaptop The key's randomart image is: +---[RSA 2048]----+ | ..oo... . | |. .oo.o. o | |o. +o.... . o . | |o+o.+. o o o . | |OE.+ S . . . | |+++ o . = . | |.. = o .. + | | .o= + . | | .+oo= | +----[SHA256]-----+ $ cat /home/penguin/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsvctNQODHWNCAPdONfibjIW6YcMA1xvk/rmevRiTX s1/QD68DVQWHyO5agOMwH4ve09QezcwMSanIuhn/rGVOd/XYoDd9AnMe2t0PFRl7ubPL2wswBUeVeryU 1HtF+nLpcGSBrzvrwP/UjcpctACUjajFBtsxS/R4n/tg6rLG3u2epCi9igUeUnfsBgR+r88wVTab3eNk zNAXyzLJJZgulWXeShwFQYddecOTCdw9XXKwS9UQzuS8XCW0KgN02eNC/YlTvJITzm+eqktgVZKoW2zJ v9iwcUIhXyWzJEUI7rPz7Pb1gXBp3Z/gJWQYc+81TDu9BMx0oIp7KWOB7rtf penguin@mylaptop
Add new Public Key to the POD Portal¶
The public portion of the newly created SSH key pair is now ready to be uploaded into the Add an SSH Key form.
- Enter a name for your new public key in the Name textbox.
- Select and copy the entire contents of the
~/.ssh/id_rsa.pubfile as a single line.
- Paste the public key string into the Key textbox.
- Select the checkboxes for POD MT1 and POD MT2 to install your public key on both cluster locations.
- Click the blue Upload New Key button to publish your private key
Load your Private Key and Connect¶
To make sure that your private key is loaded into your local authentication agent please use the
ssh-add -l command to list the fingerprints for each identity loaded. If you don’t see your key listed then add it using the
add-add command specifying the path to your private key. If your key is protected with a passphrase you will need to enter it now.
$ ssh-add -l $ ssh-add ~/.ssh/id_rsa Enter passphrase for /home/penguin/.ssh/id_rsa: Identity added: /home/penguin/.ssh/id_rsa (/home/penguin/.ssh/.id_rsa) $ ssh-add -l 2048 SHA256:sc+GdfQhQqICl5WjRKsyIjxliJuwy3u23iXpnsCRuQM penguin@mylaptop (RSA)
Now that your public key is shared with the POD user portal and your private key is loaded into your local authentication agent, you should be able to login and authenticate onto your POD login node without entering a password. Just make sure you are using your POD username and the hostname or IP address of your login node. These will be different for each user. The first time you try to access your login node you may need to add the host to your known hosts list. Respond
yes to continue connecting.
$ ssh firstname.lastname@example.org The authenticity of host 'login-12-34.pod.penguincomputing.com (126.96.36.199)' can't be established. ECDSA key fingerprint is SHA256:+1uEXIaP3SxgRNsq5qGUxBEHomh18g3H6ZakmWozeBo. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'login-12-34.pod.penguincomputing.com,188.8.131.52' (ECDSA) to the list of known hosts. Welcome to Penguin Computing's POD Use the 'podstatus' command to show available cores per queue See our documentation for examples and templates https://pod.penguincomputing.com/documentation Queue Compute Nodes Cores/Node RAM/Node ------------------------------------------------------- B30 2.4GHz Intel Broadwell 28 256GB S30 2.4GHz Intel Skylake 40 384GB Contact Support for Help: email@example.com * Use 'podstatus -v' to show available cores per queue based on duration * Use 'podstatus --help' for options to filter by queue, nodes, cores, duration [penguin@podmt2-e0e974e ~]$
SSH Keys on Windows using PuTTY¶
We will walk through the steps required for Windows users to generate and install a SSH key using PuTTYgen. PuTTY is a widely used suite of ssh client programs for Windows. The private portion of the key pair is stored on the client Windows machine and the public part is uploaded to the POD cluster from the portal. For a more thorough introduction to SSH or public/private key concepts, visit OpenSSH.org. Use the following links to download and install any required software for Windows.
Creating a new Key Pair¶
- Start PuTTYgen and generate a new public/private key pair or import an OpenSSH key by selecting Conversions/Import key. When generating the key you will have to move your mouse cursor inside the window to help generate some entropy:
- After you generate or import an OpenSSH key, enter and confirm a passphrase before saving the private key:
- Save the key in a safe but accessible folder.
Add the Public Key to the POD Portal¶
- Inside the PuTTYgen window, right-click and Select All to highlight all of the public key. Make sure you copy the entire string including the beginning that starts with
ssh-rsaand ends and the key-comment that may include the date by default:
- The public key is can now be pasted into the Key textbox on the Add an SSH Key form. Make sure to select the checkboxes for POD MT1 and POD MT2 to install your public key on both cluster locations:
- Once uploaded, you should then see the ssh key added to your account with a Success message:
Start Pageant (Authentication Agent)¶
- From your POD account, click Manage My Login Nodes and get the IP address:
- Start the Pageant program using the Start Menu. It should be installed as part of the PuTTY suite of applications. Once running it will appear in your Windows system dock located near the clock on the desktop. You may need to right click the Pageant icon and select Manage Keys to open the Pageant Key List window:
- Add your new private key that you just saved to Pageant:
- You should see your key in the Pageant Key List window. Please Note: Make sure this matches the fingerprint of the key listed in the POD Portal.
Connect to your Login Node with PuTTY¶
Please Note: Make sure Pageant is running!
- Run PuTTY using the Start Menu and enter your login node host name or IP address into the Host Name textbox. Make sure to save the session before opening.
- The first time you connect to a new login node, a security alert will pop up, click yes to accept the host key:
- A console window will open prompting you for your POD username to login as. Enter in your POD System Account username. It should then authenticate you using the private key loaded into Pageant:
SSH Keys on Windows Using PowerShell¶
Microsoft announced it was bringing an integrated OpenSSH client to Windows back in 2015. In the April 2018 Windows 10 update, the built-in SSH client is now enabled in the optional feature apps section of Windows 10. This documentation will guide you though the steps to install a native OpenSSH client to your Windows OS.
Installing Windows 10 SSH Client¶
The SSH client is a part of Windows 10, but it is an “optional feature” that is not installed by default. You can either install it with the Graphical User Interface (GUI) or with the Command Line Interface (CLI).
GUI Install: Go to Settings > Apps and click “Manage optional features” under Apps & features:
Now click “Add a feature” at the top of the list of installed features. If you already have the SSH client installed, it will appear in the list:
Scroll down, click the “OpenSSH Client” option, and click “Install”:
Restart your machine. Once you have restarted you will have a native OpenSSH client installed on your system.
CLI Install: Open an Administrative PowerShell window and run the following commands:
PS C:\Users\johnnyappleseed> Get-WindowsCapability -Online | ? Name -like 'OpenSSH.Client*' Name : OpenSSH.Client~~~~0.0.1.0 State : NotPresent PS C:\Users\johnnyappleseed> Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0 Path : Online : True RestartNeeded : False
Please Note: You may need to confirm that the
ssh-agent is running in your services. Please follow the directions below to check.
Open a PowerShell windows and run the following:
PS C:\Users\johnnyappleseed> Get-Service ssh-agent Status Name DisplayName ------ ---- ----------- Stopped ssh-agent OpenSSH Authentication Agent
As you can see under the “Status” the agent is not running. Run the following commands in the PowerShell windows:
PS C:\Users\johnnyappleseed> Set-Service 'ssh-agent' -StartupType Automatic PS C:\Users\johnnyappleseed> Start-Service 'ssh-agent' PS C:\Users\johnnyappleseed> Get-Service ssh-agent Status Name DisplayName ------ ---- ----------- Running ssh-agent OpenSSH Authentication Agent
The “Status” is now running and moving forward the service will start automatically with windows. Restart your machine. Once you have restarted you will have a native OpenSSH client install on your system.
How to Setup SSH Keys¶
Once OpenSSH Client has been installed, you should have access to the following
ssh commands from any Command Prompt:
The above OpenSSH binaries are located in
C:\Windows\System32\OpenSSH\. SSH related files such as “know_hosts” and any user generated key pairs are located here is the relative
%USERPROFILE%\.ssh\ or the absolute
C:\Users\johnnyappleseed\.ssh\ path. We will be using the
ssh-keygen.exe to create your public/private SSH key pair. This works in either a PowerShell window or a Command Prompt window, so use whichever you prefer. All the code examples from this section on are using PowerShell.
Please Note: It is recommended for security reasons that you NEVER share your private key with anyone and use a passphrase that only you know.
From a PowerShell window type:
PS C:\Users\johnnyappleseed\.ssh> ssh-keygen.exe -t rsa Generating public/private rsa key pair. Enter file in which to save the key (C:\Users\johnnyappleseed/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in C:\Users\johnnyappleseed/.ssh/id_rsa. Your public key has been saved in C:\Users\johnnyappleseed/.ssh/id_rsa.pub. The key fingerprint is: SHA256:VW+V0gj2XvPCwuUbT402SZZja0I8d2+AbA1+5JDcO4k johnnyappleseed@DESKTOP-16DRCKD The key's randomart image is: +---[RSA 2048]----+ | 7 | | S 4 | | X | | l | | B | | | | 1 | | 3 | | 8 | +----[SHA256]-----+ PS C:\Users\johnnyappleseed\.ssh> explorer .
A file explorer window opened up from the PowerShell prompt. You should see two file:
The first file (id_rsa) is your private key. The second file (id_rsa.pub) is your public key. You will need to open the public key with Notepad or some other text editor. Now copy all of the public key and go to the POD Portal and sign in. Then navigate to and select “SSH Keys”. Under the section “Add an SSH Key” you will give the “Name” of the public key and paste the public key you copied into the “Key” section. Make sure to select which cluster you want the newly generated SSH key to be added too. Then select the “Upload New Key”. You will see the new key with a banner saying something like “Success Key successfully uploaded.” Now you are ready to use SSH!
Please Note: If you use a passphrase and forget it, then you will have to create a new key pair. There is no way to recover a lost passphrase.
How to Use Windows 10 SSH Client¶
You can now use the SSH client by running the
ssh command. This works in either a PowerShell window or a Command Prompt window, so use whichever you prefer.
To open a PowerShell window, right-click the Start button or press Windows+X and choose “Windows PowerShell” from the menu. To view the syntax of the
ssh command run as seen below:
PS C:\Users\johnnyappleseed> ssh usage: ssh [-46AaCfGgKkMNnqsTtVvXxYy] [-B bind_interface] [-b bind_address] [-c cipher_spec] [-D [bind_address:]port] [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11] [-i identity_file] [-J [user@]host[:port]] [-L address] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-Q query_option] [-R address] [-S ctl_path] [-W host:port] [-w local_tun[:remote_tun]] destination [command]
This command works the same as connecting to an SSH server by the way of the
ssh command on other operating systems like macOS or Linux. The syntax or command line options, are the same.
Now before we connect to one of the POD nodes, we want to make sure your SSH key is added. Run the following command
ssh-add $Your_Priavte_SSH_Key in the directory where your private SSH key is located:
PS C:\Users\johnnyappleseed\.ssh> ssh-add .\johnnyappleseed Identity added: .\johnnyappleseed (.\johnnyappleseed) PS C:\Users\johnnyappleseed\.ssh> ls Directory: C:\Users\johnnyappleseed\.ssh Mode LastWriteTime Length Name ---- ------------- ------ ---- -a---- 3/9/2019 9:18 PM 408 authorized_keys -a---- 7/29/2019 11:09 AM 1679 johnnyappleseed -a---- 7/29/2019 11:22 AM 414 johnnyappleseed.pub -a---- 7/29/2019 11:23 AM 384 known_hosts
You can see that your Private key Identity has now been added. We can also check that your identity has been added by running:
PS C:\Users\johnnyappleseed\.ssh> ssh-add -l 2048 SHA256:uDQpHUr/zF0G7P38BgU2yWMSlviPVBtX4O+QEy5PKaw .\johnnyappleseed (RSA)
We are now ready to connect into a POD node.
Please Note: Make sure the username used when connecting to any POD node, is the one you created when first setting up your POD account. Be aware that the user name is case sensitive.
We are now ready to connect into a POD node using SSH with the username johnnyappleseed. To do so, run the following:
PS C:\Users\johnnyappleseed> ssh firstname.lastname@example.org
By default, the command attempts to connect to an SSH server running on port 22.
Please Note: Before you try to SSH into one of your login nodes, make sure you have setup your SSH public/private keys setup, otherwise you will not be able to connect into any of your POD nodes.
The first time you connect to any system with SSH, you will be prompted to accept the hosts key as seen below:
PS C:\Users\johnnyappleseed> ssh email@example.com The authenticity of host 'login-12-34.pod.penguincomputing.com (184.108.40.206)' can't be established. ECDSA key fingerprint is SHA256:+1uEXIaP3SxgRNsq5qGUxBEHomh18g3H6ZakmWozeBo. Are you sure you want to continue connecting (yes/no)?
You have established a connect with POD and it is asking you “Are you sure you want to continue connecting (yes/no)?”. Type “yes” to continue accessing your node as seen below:
Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'login-12-34.pod.penguincomputing.com,220.127.116.11' (ECDSA) to the list of known hosts. Welcome to Penguin Computing's POD Use the 'podstatus' command to show available cores per queue See our documentation for examples and templates https://pod.penguincomputing.com/documentation Queue Compute Nodes Cores/Node RAM/Node ------------------------------------------------------- B30 2.4GHz Intel Broadwell 28 256GB S30 2.4GHz Intel Skylake 40 384GB Contact Support for Help: firstname.lastname@example.org * Use 'podstatus -v' to show available cores per queue based on duration * Use 'podstatus --help' for options to filter by queue, nodes, cores, duration [johnnyappleseed@podmt2-e0e974e ~]$
You are now connected to the remote node using SSH!
Remote File Management¶
Typically users will need to upload input and application files to their POD login nodes and download results and output files after a job completes. There are many ways to manage your files on POD, however we recommend using a desktop application like WinSCP or FileZilla. These applications can be installed locally, connect to your POD login node. This section of the documentation will describe how to configure and run these application on a Windows desktop machine. Use the following links to download and install any required software for Windows.
Managing Files on Your Login Node With WinSCP¶
Please Note: Make sure Pageant is running!
- Open WinSCP and put your login node into the Host name section. For File protocol, use either SCP or SFTP. Type in your username from your POD System Account and leave password blank. You can also save this login:
- The first time you connect, a security alert will pop up, click yes to accept the host key:
- It should then log in and authenticate you via pageant:
- Your computer files are on the left and your POD files are on the right:
Managing Files on Your Login Node With FileZilla¶
Please Note: Make sure Pageant is running!
- Open FileZilla and put your login node into the Host and your username from your POD System Account. Leave password blank and make sure port is set to 22. You can use Quickconnect or optionally save this login:
- Your computer files are on the left and your POD files are on the right:
Transferring Files Between MT2 and MT1 Storage¶
There are three tools for file transfer between MT2 and MT1 clusters. These tools are:
The node mt2-xfer1 is used as a gateway between the locations.
Using SCP to copy files and directories¶
This example copies a directory called test and it’s contents from my MT2 storage to my MT1 storage:
$ scp -r mt2-xfer1:test ./ nest7grid.parms 100% 270 0.3KB/s 00:00 namelist.input 100% 3821 3.7KB/s 00:00 projection.jpg 100% 42KB 41.8KB/s 00:00 namelist.wps 100% 1738 1.7KB/s 00:00
The -r switch recursively copies the entire directory. To copy a single file, just specify that file (or a use a wildcard [*] specification)
$ scp mt2-xfer1:*.sub ./ ls-dyna.sub 100% 1473 1.4KB/s 00:00 mpi-ring.sub 100% 153 0.2KB/s 00:00 n.sub 100% 68 0.1KB/s 00:00 sing.sub 100% 190 0.2KB/s 00:00 t.sub 100% 101 0.1KB/s 00:00
Note that SCP will overwrite existing files without warning. One option is to make a MT1 directory to store transferred files and then proceed with the SCP transfer.
$ mkdir mt2-files $ cd mt2-files
SFTP can be used to copy files, but not whole directories. It is useful for viewing the contents of MT1 or MT2 storage.
$ sftp mt2-xfer1 Connecting to mt2-xfer1... sftp> ls *.sub ls-dyna.sub mpi-ring.sub n.sub sing.sub t.sub sftp> quit
When connected to mt2-xfer1 with SFTP, use the question mark
? to get help with usage.
Please Note: See the section SSH Keys for Windows Users and use either WinSCP or FileZilla as the SFTP client on your local host to transfer files if needed (local <—> remote host).
Rsync is a versatile file-copying tool. It can be used much like SCP, but it first checks to see if the file is available locally and skips the copy of the file if it is already there. This is useful to keep directories in sync in different locations.
$ rsync -v mt2-xfer1:*.sub ./ ls-dyna.sub mpi-ring.sub n.sub sing.sub t.sub sent 106 bytes received 2307 bytes 1608.67 bytes/sec total size is 1985 speedup is 0.82 $ rsync -v mt2-xfer1:*.sub ./ ls-dyna.sub mpi-ring.sub n.sub sing.sub t.sub sent 148 bytes received 330 bytes 318.67 bytes/sec total size is 1985 speedup is 4.15
Please Note: The second time was much faster as the files already existed on MT1. For most users SCP and SFTP should be sufficient. Know that POD support will be happy to help with Rsync if needed as it has many options.
There are manual (man) pages available for all of these commands as seen below:
$ man scp $ man sftp $ man rsync
Please let us know if you need assistance with any transfers at POD Support: email@example.com