Best practices with HPC

Last modified by vhautaka@helsinki_fi on 2024/02/08 06:49

Use tmux after login. 

Create symbolic link from home directory to working directory. Home directory disk quota is very limited. I personally link everything larger than 1MB.

ln -s /wrk-vakka/users/{YOUR_USERNAME}/.miniconda3 ~/.miniconda3

Connect to the same computing node by adding -w node_name, e.g

srun --interactive -wukko2-g01 -M ukko2 -n2 -pgpu --pty bash

Vscode remote development

Link some folders.


cd ~ln -s /wrk-vakka/users/lichengk/.config .localln -s /wrk-vakka/users/lichengk/.local .local

Remote development and debugging with GPU support is currently available with the help of code-server. See How to use Visual Studio Code in HPC environment and https://version.helsinki.fi/it-for-science/hpc/-/issues/6#note_35515 for setup.

Bash shell

My .bash_aliases for conda:


















function conda_init() {    __conda_setup="$('/home/{YOUR_USERNAME}/.miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"    if [ $? -eq 0 ]; then        eval "$__conda_setup"    else        if [ -f "/home/{YOUR_USERNAME}/.miniconda3/etc/profile.d/conda.sh" ]; then            . "/home/{YOUR_USERNAME}/.miniconda3/etc/profile.d/conda.sh"        else            export PATH="/home/{YOUR_USERNAME}/.miniconda3/bin:$PATH"        fi    fi    unset __conda_setup}function init_py39(){    conda_init    conda activate py39    module load CUDA/11.4.1    module load cuDNN/8.2.2.26-CUDA-11.4.1}

In .bashrc:



export PATH=~/.miniconda3/bin:$PATH# bash_aliasesif [ -f ~/.bash_aliases ]; then    . ~/.bash_aliasesfi

All terminal output to file

command |& tee -a output.txt

Others

For scipy 1.7.3:


module load fgci-common/1.0module load gcc/9.2.0