/category/howto
Loading Module in Jupyter
Users cannot load modules inside a JupyterLab session. If you need access to modules, please request a desktop session instead of JupyterLab. Fill out the form as you normally would for JupyterLab. After you get to a desktop, open a terminal (next to Firefox in the top bar) and type these commands:
module load jupyterlab module load … # your modules here jupyter-lab This should start up Firefox shortly. If you accidentally close the window, right-click on the link in the terminal and choose “open link” to restart.
An example of using LaTeX inside a JupyterLab session is shown in the screenshot below.
Clear OOD Files
To clear OOD Session files, the HPC system will need to be accessed via a terminal. See documentation for information on how to access via SSH.
You can find the session files and logs for all Open on Demand apps at:
~/ondemand/data/sys/dashboard/batch_connect/sys Under this directory you will see subdirectories for the Open on Demand applications that you have used before. Under each subdirectory you can find the files that are created when you launch a new session.
To quickly clear all session files for OnDemand from your /home directory run:
rm -rf ondemand Other directories related to Open on Demand such as .
Drive Mapping on Windows or Mac OSX
Research Standard and Research Project storage can be mapped to your Windows or Mac computer as a remote drive. If you are off Grounds you must be running a VPN, such as the UVA Anywhere or the More Secure VPN from ITS. We recommend the More Secure VPN if that is available to you.
Windows Open a File Explorer page. In the left column, right-click on This PC. In the drop-down box that appears, look for the Map Network Drive option. If you do not see this option, click on Show more Options, and then click on Map network drive….
How to add packages to a container?
Basic Steps Strictly speaking, you cannot add packages to an existing container since it is not editable. However, you can try to install missing packages locally. Using python-pip as an example:
module load apptainer apptainer exec <container.sif> python -m pip install –user <package> Replace <container.sif> with the actual filename of the container and <package> with the package name. The Python package will be installed in your home directory under .local/lib/pythonX.Y where X.Y is the Python version in the container.
If the installation results in a binary, it will often be placed in .local/bin. Remember to add this to your PATH:
The Make Tool
Overview Make is a program used primarily on Unix systems to manage compiling and linking (building) programs written in C, C++, Fortran, or other compiled languages. Make operates on targets using rules to create those targets. It has a set of built-in rules but users may write their own or override the default rules. Make scans the dependencies of each target looking for files newer than the target. If it finds them, it recreates the target. Targets may and usually do depend on other targets; make will work its way through the chain to rebuild the final target, which is typically an executable.
Converting a Jupyter Notebook to a Python Script
Sometimes it may be useful to convert a Jupyter notebook into a Python executable script. Once your notebook is opened in OOD you can select File > Export Notebook As … > Export Notebook to Executable Script:
This will download a Python executable with a ‘.py’ extension into your local computer’s Downloads folder. Your notebook may also show “Download as” instead of “Export Notebook As …”. Either of these selections will allow you to download a Python executable.
This script can be copied to the HPC system in the working directory where JupyterLab was accessing the notebook. Information on transferring files to and from Rivanna can be found here.
Converting a Jupyter Notebook to a PDF
Users cannot load modules inside the OpenOnDemand App for JupyterLab. Therefore it is not possible to convert a Jupyter Notebook to a PDF directly inside the JupyterLab Interactive App on OpenOnDemand.
There are 2 ways to convert a Jupyter Notebook to a PDF:
Directly from the command line. ssh from your terminal and type the following: module load anaconda/2020 texlive jupyter nbconvert –to pdf you_script.ipynb If you want to use GUI, please request a desktop session.
Fill out the form as you normally would for JupyterLab. After you get to a desktop, open a terminal (black box next to Firefox in the top bar) and type these commands: module load anaconda/2020 texlive jupyter notebook This will pull up JupyterLab.
Launching RStudio Server from an Apptainer Container
Rocker provides many software containers for R. Due to the default permission settings of our file system, launching an RStudio Server session is not straightforward. If you are interested in using their containers on the HPC system, please follow these steps.
Pull container Use Apptainer to pull the container. We will use geospatial in this example.
module load apptainer apptainer pull docker://rocker/geospatial You should see geospatial_latest.sif in your current directory.
One-time setup The commands in this section are to be executed as a one-time setup on the frontend. You may need to repeat the steps here when running a new rocker container.
Migrating Python packages
Scenario You have installed Python packages locally in one version and now wish to use them in a different version. For example, you have been using Python 3.6 but it is obsolete and will be removed soon, so you need to set up those packages for Python 3.8. There are several ways to accomplish this, depending on the package manager. In this how-to we will discuss pip and conda.
You will need to load the module for the newer Python version. For this example,
module load anaconda/2020.11-py3.8 $('#copybtn741632985').click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#741632985').text()).select(); document.
How-To Guides for UVA HPC Users
Guides Building compiled code Using make Building and running MPI Code Bioinformatics on UVA HPC Clear OOD Session Files Convert Jupyter Notebook to PDF Convert Jupyter Notebook to Python Script Custom Jupyter kernels Loading Modules in Jupyter Docker images on UVA HPC Adding packages to a container Migrate Python packages Launch RStudio Server from an Apptainer container More Documentation Connecting Using SSH Using a browser Using FastX Jobs / Slurm / Queues Slurm Overview Queues Storage and File Transfer Storage overview Data transfer methods Allocations Allocations Overview
Transfer Files Using Amazon S3
Setup You will need to install and configure the awscli package in order to access objects in S3.
Install the AWS CLI The AWS CLI is available through the pip/pip3 installer:
If you have administrator privileges type
pip install awscli $('#copybtn372519684').click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#372519684').text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); Otherwise use
pip install –user awscli $('#copybtn531674982').click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#531674982').text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); The project is open source, so you can also download the source at https://github.
Bioinformatics Resources and UVA HPC
The UVA research community has access to numerous bioinformatics software installed directly or available through the bioconda Python modules.
Click here for a comprehensive list of currently-installed bioinformatics software.
Popular Bioinformatics Software Below are some popular tools and useful links for their documentation and usage:
.tg {border-collapse:collapse;border-spacing:0;border-color:#ccc;} .tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#fff;} .tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#f0f0f0;} .tg .tg-hy9w{background-color:#eceeef;border-color:inherit;vertical-align:middle;} .tg .tg-dc35{background-color:#f9f9f9;border-color:inherit;vertical-align:middle;} .tg .tg-hy9w-nw{background-color:#eceeef;border-color:inherit;vertical-align:middle;white-space:nowrap;} .tg .tg-dc35-nw{background-color:#f9f9f9;border-color:inherit;vertical-align:middle;white-space:nowrap;} .tg .tg-0qmj{font-weight:bold;background-color:#eceeef;border-color:inherit;vertical-align:middle;} .scroll thead, .scroll tbody {display: block} .scroll tbody {overflow-y: auto; height: 500px;} .scroll thead tr:after {content: ‘';overflow-y: scroll; visibility: hidden; height: 0;} Tool Version Description Useful Links BEDTools 2.
How To Tips for Storage
Map your Research Standard or Research Project Storage to your Desktop Use Globus from the Command Line Work with files in Amazon S3
Building Your Code on the HPC System
Building your Application Creating an executable from source with a compiled language requires two steps, compiling and linking. The combination of these is generally called building. The output of the compiler is generally an object file, which on Unix will end in a .o suffix. Object files are machine code and are not human-readable, but they are not standalone and cannot be executed. The linker, which is usually invoked through the compiler, takes all object files, along with any external libraries, and creates the executable (also called a binary).
Compilers are invoked on source files with a line such as
Highly Sensitive Data Transfer with Globus
Login In to Globus and Choose Ivy Collection
Disconnect your High Security VPN – it will interfere with the high security network that Globus uses. Open https://www.globus.org/ and sign in through the University of Virginia using NetBadge. Once you are on the Transfer Files page, at the first “Collection” type ivy to search. Find the uva#ivy-DTN collection and select it. You will then be asked to authenticate. Once signed in, simply click through the names of your shares until you find the source or destination for your file transfers. Navigate to Your Personal Collection
Running a Bioinformatics Software Pipeline with Wdl/Cromwell
WDL (pronounced widdle) is a workflow description language to define tasks and workflows. WDL aims to describe tasks with abstract commands that have inputs, and once defined, allows you to wire them together to form complex workflows.
Learn More
CROMWELL is the execution engine (written in Java) that supports running WDL scripts on three types of platforms: local machine (e.g. your laptop), a local cluster/compute farm accessed via a job scheduler (e.g. Slurm, GridEngine) or a cloud platform (e.g. Google Cloud or Amazon AWS).
Learn More
Introduction Pre-requisites: This tutorial assumes that you have an understanding of the basic structure of a WDL script.
Globus Command-Line Interface
Install the Globus CLI The Globus CLI is available through the pip installer:
If you have administrator privileges type
pip install globus-cli Otherwise use
pip install –user globus-cli The project is open source, so you can also download the source at https://github.com/globus/globus-cli
If you would like to use the CLI from UVA HPC, please follow these directions below.
Authenticate using the Globus CLI Log in against your institutional authentication provider. In the case of UVA, we use NetBadge for signing in:
globus login This will open a page in your web browser where you select your institution and proceed to log in:
Docker - The Basics
Note that Docker requires sudo privilege and therefore it is not supported on the HPC system. To use a Docker image you will need to convert it into Apptainer. More information can be found here on our website.
What Is Docker? “Docker is a set of platform-as-a-service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. All containers are run by a single operating-system kernel and are thus more lightweight than virtual machines.
Custom Jupyter Kernels
You can create custom kernels from an Anaconda environment or an Apptainer container.
In both cases you’ll need to install the ipykernel package.
Jupyter kernel based on Anaconda environment To create a custom kernel of the Anaconda environment myenv that uses Python 3.7:
module load anaconda conda create -n myenv python=3.7 ipykernel <other_packages> source activate myenv python -m ipykernel install –user –name myenv –display-name "My Env" Note:
You can customize the display name for your kernel. It is shown when you hover over a tile in JupyterLab. If you do not specify a display name, the default Python [conda env:<ENV_NAME>] will be shown.
A Short MPI Tutorial
Tutorials and books on MPI A helpful online tutorial is available from the Lawrence Livermore National Laboratory. The following books can be found in UVA libraries:
Parallel Programming with MPI by Peter Pacheco. Using MPI : Portable Parallel Programming With the Message-Passing Interface by William Gropp, Ewing Lusk, and Anthony Skjellum. Using MPI-2: Advanced Features of the Message-Passing Interface by William Gropp, Ewing Lusk, and Rajeev Thakur. MPI: The Complete Reference : The MPI Core by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra. MPI: The Complete Reference : The MPI-2 Extensions by William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, and Marc Snir.
Building and Running MPI Code
Building an MPI Code All implementations provide wrappers around the underlying compilers that simplify compilation. As it is very important to use the headers that correspond to a given library, users are urged to make use of the wrappers whenever possible. For OpenMPI and MVAPICH2 these are:
mpicc (C) mpicxx (C++) mpif90 (Fortran free or fixed format) For Intel MPI these use gcc/g++/gfortran by default, which is generally not recommended; to use the Intel compilers the corresponding wrappers are:
mpiicc mpiicpc mpiifort Note: At this time, we recommend MPI users build with Intel 18.0 and IntelMPI 18.
Docker Images on the HPC System
Docker requires sudo privilege and therefore it is not supported on the HPC system. To use a Docker image you will need to convert it into Apptainer.
Convert a Docker image There are several ways to convert a Docker image:
Download a remote image from Docker Hub Build from a local image cached in Docker daemon Build from a definition file (advanced) Instructions are provided in each of the following sections.
Docker Hub Docker images hosted on Docker Hub can be downloaded and converted in one step via the apptainer pull command:
module load apptainer apptainer pull docker://account/image Use the exact same command as you would for docker pull.
General How To Guides
User Guides Docker - Basics Authentication with SSH Keys
How To Guides for Ivy Users
Transfer Files to or from Ivy Using Globus
How To
General General tips and tricks for computational research.
General HowTos › Rivanna and Afton High Performance Computing platforms
HPC HowTos › Ivy Secure Data Computing Platform
Ivy HowTos › Storage Research Data Storage & Transfer
Storage HowTos ›
Image Processing & Scientific Visualization on the HPC system
Available Software To get an up-to-date list of the installed image processing and visualization tools, log on to UVA HPC and run the following command in a terminal window:
module keyword vis To get more information about a specific module version, run the module spider command, for example:
module spider blender/2.78c List of Image Processing and Visualization Software Modules
function searchFunction() { var input, filter, table, tr, td, i, txtValue; input = document.getElementById(“searchInput”); filter = input.value.toUpperCase(); table = document.getElementById(“moduleTable”); tr = table.getElementsByTagName(“tr”); for (i = 0; i -1) { tr[i].style.display = “"; } else { tr[i].style.display = "
