Resources for Research
- 1.0 Introduction
- 2.0 Cubbli Linux
- 3.0 HPC Environment
- 4.0 Storage and I/O
- 5.0 Licensed Software
- 6.0 Open Source Software & Libraries
- 7.0 VDI
- 8.0 GitLab Server
- 9.0 Infrastructure
- 10.0 Consulting Services
- 11.0 ePouta
1.0 Introduction
IT for Science group provides resources and consulting services for the research community. This document provides an compendium of the free resources available for the research community. In short, we provide following services:
- Cubbli Linux distribution
- GPU capacity, up to 20 GPU's available for interactive and batch use.
- High Performance Computational capacity in three facilities for interactive and batch use.
- Fast I/O
- Consulting services and best practices.
1.1 Can I buy my own servers?
You may be considering procuring computational resources, whether they are GPU machines for Deep Learning, regular servers for computing, servers for heavy duty database use, or storage capacity and have checked that your requirement is something we do not yet offer. We have a good news. We can house servers that are used for computational tasks and incorporate them gladly to the existing pool of resources. These can be for example:
- Dedicated to you alone, or...
- Your research group could ask for pre-emption capability, or...
- A standard addition to common pool without special privileges besides of becoming an Investor User.
You can always opt to increase the common pool of resources at any time. Even ones that have been dedicated before.
(Regardless of whether the server would be added into the cluster environment or remain a standalone machine, any purchases must be done through University's purchase contracts to ensure that legal aspects such as formal bidding contest are fulfilled properly.)
1.1.1 We offer benefits
- Standard, centrally managed Centos7 Linux environment.
- All the software modules listed later in this document.
- Very high throughput/low latency I/O ($WRKDIR).
- Automated data migration from $WRKDIR to CEPH data lake - Planned 2019Q2.
- Other resources in a same high-bandwidth interconnect fabric.
- Resource Usage statistics.
- You can utilize our procurement process and therefore get better deals.
1.2 If you still think you must choose yourself, we have something to say...
In exchange for above benefits, we have a few requirements for servers that are not bought by us:
- Rack mountable server, or chassis solution
- Infiniband nic (depending on the bandwidth requirement, this can be FDR (56Gb/s) and EDR (100Gb/s))
- Associated IB cable
1.3 Because
1.3.1 Why should I become Investor User?
- You will be granted higher percentage of the shares of the computational resources.
- Being investor user is especially beneficial when your jobs are resource intensive.
- Having more shares means that job priorities decay slower.
1.3.2 Why not to use dedicated servers?
- A history has shown that dedicated resources are often ill utilized. Utilization of less than 20% is very common.
- Larger pool of common resources enables greater availability for everyone.
- When resources are not dedicated, you can scale immediately upwards whenever you have temporary need.
- Pooled resources are immediately available for use.
- We are flexible, and if you know in advance, we can create resource reservations in advance.
If you would need to have dedicated pool of $WRKDIR I/O resources which is only accessible by named research group, we can do this as well. In this case you should contact us and we will discuss the details.
You can find additional details about access, restrictions etc and other topics in the User Guides.
1.4 Other topics
A brief information about general University wide terms and conditions can be found through the links listed below.
- Terms for purchasing and using the scientific software at the university.
- Availability and licensing of mathematical software.
For any concerns, questions or comments you can reach us in person at Kumpula campus, or through the university helpdesk at helpdesk (at) helsinki (dot) fi . Resource requests, and bug reports should be addressed through helpdesk (at) helsinki (dot) fi .
2.0 Cubbli Linux
Cubbli Linux distribution is fully supported by IT4SciFi. Please see the Cubbli Linux for additional information about availability and release notes. Cubbli Linux is available for servers, workstations and laptops.
Cubbli documentation also here (in Finnish).
3.0 HPC Environment
If, and when the demand of computational resources exceed the capabilities of ordinary workstation, we invite you to utilize our HPC resources. We offer significant GPU, CPU and I/O resources, and fairly comprehensive - and constantly updated library of scientific software, compilers and programming environments. Should you have specific resource need not listed here, you can contact us through helpdesk (at) helsinki (dot) fi. If you know that your workload scales beyond our resources, you can take a look at the national supercomputing resources available at CSC.
Below you can find a list of hardware resources available on the high performance computing platform. 'How to access example' refers to the use of batch scheduler flags. Notes include special details of the components included. 'HT' indicates that CPU Hyperthreading is used on the nodes to increase the core count.
Note that sometimes Vendors have grants for certain resources as indicated on the list. Do check if you are eligible.
Node Type | Node Qty | Sockets x Core (HT) per Node | Node Memory | Where | How to Use | Notes |
|---|---|---|---|---|---|---|
GPU, Tesla P100 | 2 | 2x 2.4GHz 14HT, 4xGPU | 512GB | Kale | -p --gres:gpu=1 | |
GPU Tesla k80 | 1 | 2x 2.4GHz 12, 4xGPU | Kale | -p --gres:gpu=1 | ||
Skylake | 23 | 2x 20HT cores | 384GB | Kale | --constraint=avx | |
Xeon E5 | 31 | 2x 2.4GHz 14HT cores | 256GB | Ukko2 | Default | |
Xeon E7 | 2 | 4x 2.2GHz 24HT cores | 3TB | Ukko2 | -p bigmem | |
Xeon | 19 | 2x 2.4GHz 12 cores | 129GB | Kale | --constraint=smem, Default | |
Xeon | 13 | 2x 2.4GHz 12 cores | 260GB | Kale | --constraint=mmem, Default | |
Xeon | 1 | 4x 2.4GHz 12 cores | 1.57TB | Kale | --constraint=bmem | |
GPU, V100 | 4 | 4x GPU | 384GB | Kale | 800GB NVMe SSD. 12GPU's with 32GB, 4GPU's with 16GB | |
Sandybridge | 180 | 2x 2.66GHz 8HT (32 HT cores) | 64GB | Vorna | About 6000 cores | |
Sandybridge | 12 | 2x 2.66GHz 8 cores | 64GB | Vorna | Service nodes | |
AMD Epyc | 10 | Carrington | ||||
AMD Epyc | 16 | ukko |
4.0 Storage and I/O
List of available I/O resources for the users. If a I/O resource is missing from the list, or there are bugs/problems or other concerns, please do contact us through university helpdesk at helpdesk (at) helsinki (dot) fi. We are investing on high performance I/O, and additional computational resources at the moment, and expect a fresh capability to be in production 1Q2019.
Actual Path | Nodes | Where | How to Access (examples) | Quota Limit | Notes |
|---|---|---|---|---|---|
/home/$USER | NFS | ALL | Home directories | Very low | |
Data Lake | ALL | N/A | |||
/wrk/users/$USER | (Vakka) | Turso, pangolin, melkki, melkinkari, desktops | #SBATCH --workdir=/wrk/users/<username> Samba desktops: ukko2-smb.cs.helsinki.fi pangolin, melkki and melkinkari: /home/ad/ukko2-wrk/$USER | User quota limit 50T | $WRKDIR Capacity: 1.2PB, redundant ZFS backend |
/wrk/users/$USER | (Kappa) | Temp directory | $WRKDIR Capacity: 750TB, non-redundant, ZFS backend | ||
/proj/$USER | NFS | Turso, pangolin, melkki, melkinkari, desktops | User application directory for applications, source codes. pangolin, melkki and melkinkari: /home/ad/turso-proj/$USER | 20GB | $USERAPPL |
/appl/ | NFS | Global software module repository (RHEL7, CentOS7) | N/A | Read Only | |
/proj/$USER | NFS | Turso, pangolin, melkki, melkinkari, desktops | Longer term storage for datasets and results. pangolin, melkki and melkinkari: /home/ad/turso-proj/$USER | 20GB | $PROJ |
5.0 Licensed Software
We provide licensed software, some which are available also for free for certain users, or user groups. It is advisable to check if you qualify for a free license, if the product information so implies. If the licensed program is not present and you would like it to be included, please do not hesitate to contact us through university helpdesk at helpdesk (at) helsinki (dot) fi.
6.0 Open Source Software & Libraries
This list includes most of the special scientific software packages, tools and specialized libraries made available by IT4SciFi. Items on this list have multiple versions available on the clusters and different versions cam be managed with Modules. If the utility, tool, library or program, or specific version is not present and you would like it to be included, please do not hesitate to contact us through university helpdesk at helpdesk (at) helsinki (dot) fi.
You can also create modules for of your own software and then share it conveniently within the research group if you like. We have brief instructions here.
All system and development libraries are present on the computational environment by default, and therefore not included in the list - except where tools are installed as modules in the HPC environment.
Aalto software modules
You are also eligible to use software module from Aalto univerisity. However these are not defaulted because the module list would be very long. To have Aalto modules included, please add modules:
module use /cvmfs/fgi.csc.fi/apps/el7/aalto/spack/lmod/linux-centos7-x86_64/allmodule use /cvmfs/fgi.csc.fi/apps/el7/aalto/spack/lmod/linux-centos7-westmere/all/
Local software modules
Name | Where Available | How to Access | Tutorials & Use Cases |
|---|---|---|---|
AOCC | As a module in clusters. | AMD Compiler Suite | |
Through Python Virtualenv | |||
Through Python Virtualenv | |||
As a module in clusters. Cubbli default | |||
As a module in clusters. | |||
As a module in clusters. | Molecular simulation toolkit | ||
As a module in clusters. | G romacs tutorials, GROMACS/2018.3-fosscuda-2018b with GPU support. | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. Cubbli default | GNU Compiler Collection | ||
Java | As a module in clusters. Cubbli default | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | Molecular Structure Analysis | ||
As a module in clusters. | Quantum Chemistry | ||
As a module in clusters. Cubbli default | |||
As a module in clusters. | |||
GMP | As a module in clusters. | ||
As a module in clusters. | |||
As a module in clusters. | |||
Mako | As a module in clusters. | ||
As a module in clusters. | |||
Cubbli default | |||
Cubbli default | |||
MRI Imaging | |||
Molecular Dynamics | |||
As a module in clusters. | |||
OpenPGM | As a module in clusters. | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
XZ | As a module in clusters. | ||
System Package | |||
Valgrind-openmpi | System Package | ||
System Package | v8 Development Library | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | Geospatial Data Abstraction Library | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | C subroutine library for discrete Fourier transform | ||
As a module in clusters. | |||
As a module in clusters. | |||
Cubbli default | |||
Cubbli default | |||
Oracle JDK | Cubbli default | ||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
As a module in clusters. | |||
SQLite | As a module in clusters. | ||
As a module in clusters. | |||
Commercial? | |||
Cubbli ? | |||
Cubbli ? | Commercial? | ||
Cubbli ? | Commercial? | ||
System package on a compute nodes | Container system |
7.0 VDI
Please see the Cubbli and VDI for additional information.
Note: that VDI is not an environment to be used for computationally intensive tasks, such as Tensorflow over GPU. You will get far better performance if you use these resources instead.
8.0 GitLab Server
IT4SciFi offers and manages GitLab server at version.helsinki.fi. Please see the User Guide for further details and usage policies.
9.0 Infrastructure
9.1 Geoinformatics
The university of Helsinki uses the CSC GIS services as a central source for GIS data.
9.2 VMWare Virtual Servers
What to say?
10.0 Consulting Services
IT4SciFi offers consulting services. Wether the question is about virtual machines, hardware, software or computational, or storage etc. resource requirements, project servers or any other topic, feel free to contact us through university helpdesk at helpdesk (at) helsinki (dot) fi.
11.0 ePouta
A cloud computing environment designed for processing and / or storing sensitive data. Service provided by csc but server administration handled by IT department. Request resources via helpdesk@helsinki.fi. See https://research.csc.fi/epouta for further details.