Custom Environments and Package Installation (R and Python)

R

  • Create personal R library folder in home directory
    • This needs to be done only once per user
mkdir .Rpackages
  • Load R module (the version you need and is available)
    • Load any other library dependencies as needed
module load r-3.5.1-gcc-8.2.0-djzshna
  • Start R program
R
  • Install packages
install.packages("packageName")

Python

Multiple python environments can be created under a user profile

  • Start by first loading anaconda/miniconda 2 or 3
    • Load any other library dependencies as needed
module load miniconda3-4.5.11-gcc-8.2.0-oqs2mbg
  • You can create a new custom conda/python environment
conda create --name myenv

myenv: You can use the project name or specific application name you want associated with this environment.

In some situations, one may want to create an environment with a specific version of python or specific version of a package. This is important because as you add new packages, conda will update existing packages and dependencies.

For example, if you want to ensure that your python version doesn’t change from ver. 3.8

conda create -n myenv python=3.8
  • Once you have your environment created you can use/enter that environment with the following command:
conda activate myenv 

or

source activate myenv 

(where myenv is your custom environment name)

  • Here you can add additional packages with
conda install (package name)
  • To revert back to the default environment and out of the custom environment
conda deactivate

Additional commands:

  • To remove/delete a custom environment
conda env remove --name myenv
  • To list the various environments under your profile
conda env list
  • For the submission file
#SBATCH –qos ... 
...
module load miniconda3-4.5.11-gcc-8.2.0-oqs2mbg
...
conda activate myenv
...
python pythonScript
conda deactivate