Accounts and shares¶
Do I have access?
- Employees: every employee with a NetID can log in to DelftBlue and can use the
innovationaccount straight away. - Researchers who need more resources: request access to a research account of your department.
- Bachelor and Master students working on a thesis: request an education account.
- Guest researchers: see Guest access.
The accounting of computing time on DelftBlue follows the fair share principle, as explained below. These faculties and institutions hold a share for research on the machine:
- CiTG/CEG 12%
- EWI/EEMCS 12%
- LR/AE 12%
- ME 12%
- TBM/TPM 12%
- TNW/AS 12%
- BK/ABE 6%
- IO/IDE 2%
- QuTech 2%
Resources dedicated to education and innovation:
- Education 13%
- Innovation 5%
%%{init: {'theme': 'base', 'themeVariables': {'pie1': '#e45756', 'pie2': '#4c78a8', 'pie3': '#f58518'}}}%%
pie title DelftBlue shares
"Research" : 82
"Innovation" : 5
"Education" : 13
Accounts, shares and partitions
- Account: the Slurm account your job is charged to, chosen with
--account(for exampleresearch-as-imphys,innovationoreducation-…). This is not your NetID: you may be able to use several accounts, and an account is used by many users. - Share: the part of DelftBlue an account is entitled to, relative to the other accounts. Every account belongs to a share; the research and education accounts are subdivided per faculty and per department or programme.
- Partition: a group of nodes with the same hardware, chosen with
--partition(for examplecompute-p1orgpu-a100).
The account determines your fairshare and your limits; the partition determines the hardware and how your usage is billed.
In order to see to which accounts you have access, you can use the command
If the users of a share are using more computing time than is allocated to them, the priority of their jobs goes down. Jobs may still run if other parties are under-using their computing time (so that some cluster resources are still idle and available for these jobs). How usage is counted and how it affects the priority of your jobs is explained in Understanding Job Priority and Fairshare.
Putting your CPU hours request into perspective
DelftBlue Phase 1 has 218 nodes with 48 CPUs each, and Phase 2 has 90 nodes with 64 CPUs each. That's a total of \((218*48 + 90*64)*24*365 =144\) M CPU hours a year for the entire university. If your faculty has 12% fair share, this comes down to ~17 M CPU hours a year for the entire faculty.
To make sure that everyone gets a fair chance of running their job, there are some limits associated with different account types. To see which limits are associated with your accounts, use the following command:
sacctmgr list user $USER withassoc format='user%-20,account%-45,maxjobs,maxsubmit,maxwall,maxtresperjob%-40'
How to request a correct account for your job¶
In order to request a correct account for your Slurm job, the following line needs to be added to the submission script:
The following accounts are available:
Innovation share¶
This is the default account for all TU Delft employees (i.e. it is assumed if the --account= line is absent from the Slurm submission script). This account gives you a relatively low priority, and only allows requesting limited amount of resources per job.
Note
The default "innovation" account has very restrictive limits implemented: no more than 4 jobs running simultaneously per user, no more than 64 CPUs, 2 GPUs and 250GB of memory (over 2 nodes maximum) can be requested per job, and a job can not run longer than 24 hours. No more than 10 jobs can be in the queue per user simultaneously. Furthermore, no more than 8 GPUs can be occupied in total by all users in the entire innovation account. The innovation share is meant for developing and testing your code; for production runs, request access to a research account.
For example, if you see the following reasons for your jobs pending: (AssocGrpCpuLimit) or (AssocMaxJobsLimit) - then you are hitting these limits. If this is a problem, then please request access to a research account via TopDesk!
Research share¶
Researchers (PhDs, postdocs, or "above") can request access to a research account, through the Self Service Portal. The research shares are organised hierarchically, per faculty and per department:
For example, for researchers from EEMCS faculty and DIAM department, the correct account name is:
Note
The "research" accounts are very generous, but not unlimited: no more than 2400 CPUs, 8 GPUs and 10TB of memory (over 50 nodes maximum) can be requested per job, and a job can not run longer than 120 hours (on CPU-nodes) or 48 hours (on GPU-nodes). Additionally, no more than 64 jobs of the same user can run simultaneously.
The full list of available account names is as follows:
ABE (BK)¶
#SBATCH --account=research-abe-aet
#SBATCH --account=research-abe-ar
#SBATCH --account=research-abe-mbe
#SBATCH --account=research-abe-ur
AE (LR)¶
#SBATCH --account=research-ae-asm
#SBATCH --account=research-ae-co
#SBATCH --account=research-ae-fpt
#SBATCH --account=research-ae-spe
AS (TNW)¶
#SBATCH --account=research-as-bn
#SBATCH --account=research-as-bt
#SBATCH --account=research-as-cheme
#SBATCH --account=research-as-imphys
#SBATCH --account=research-as-qn
#SBATCH --account=research-as-rid
#SBATCH --account=research-as-rst
CEG (CiTG)¶
#SBATCH --account=research-ceg-3md
#SBATCH --account=research-ceg-es
#SBATCH --account=research-ceg-grs
#SBATCH --account=research-ceg-gse
#SBATCH --account=research-ceg-he
#SBATCH --account=research-ceg-tp
#SBATCH --account=research-ceg-wm
EEMCS (EWI)¶
#SBATCH --account=research-eemcs-diam
#SBATCH --account=research-eemcs-ekl
#SBATCH --account=research-eemcs-ese
#SBATCH --account=research-eemcs-insy
#SBATCH --account=research-eemcs-me
#SBATCH --account=research-eemcs-qce
#SBATCH --account=research-eemcs-st
IDE (IO)¶
#SBATCH --account=research-ide-dos
#SBATCH --account=research-ide-hcd
#SBATCH --account=research-ide-sde
ME¶
#SBATCH --account=research-me-bme
#SBATCH --account=research-me-cor
#SBATCH --account=research-me-dcsc
#SBATCH --account=research-me-mse
#SBATCH --account=research-me-mtt
#SBATCH --account=research-me-pe
#SBATCH --account=research-me-pme
QuTech¶
#SBATCH --account=research-qutech-qcd
#SBATCH --account=research-qutech-qid
#SBATCH --account=research-qutech-qrd
TPM (TBM)¶
#SBATCH --account=research-tpm-ess
#SBATCH --account=research-tpm-mas
#SBATCH --account=research-tpm-vti
Education share¶
This should be your default account, if you are a Master student and have been granted access to DelftBlue for your master thesis research. The education shares are organised hierarchically, per faculty and per specialisation:
For example, for MSc students from EEMCS faculty and Applied Mathematics specialisation, the correct account name is:
Note
The "education" accounts have relatively restrictive limits implemented: no more than 8 jobs running simultaneously per user, no more than 64 CPUs, 2 GPUs and 250GB of memory (over 2 nodes maximum) can be requested per job, and a job can not run longer than 24 hours. No more than 100 jobs can be submitted to the queue simultaneously. Furthermore, no more than 2000 CPUs, 12 GPUs and 10000GB of memory can be occupied in total by all users in the entire set of education accounts.
For example, if you see the following reasons for your jobs pending: (AssocGrpCpuLimit) or (AssocMaxJobsLimit) - then you are hitting these limits. If you need more resources, and it is scientifically justified, you might request being added to the faculty research share of your supervisor. Your supervisor should contact us, if this is the case.
The full list of available account names is as follows:
MSc students¶
ABE (BK)¶
#SBATCH --account=education-abe-msc-a
#SBATCH --account=education-abe-msc-g
#SBATCH --account=education-abe-msc-gima
AE (LR)¶
AS (TNW)¶
#SBATCH --account=education-as-msc-ap
#SBATCH --account=education-as-msc-ce
#SBATCH --account=education-as-msc-lst
#SBATCH --account=education-as-msc-nb
#SBATCH --account=education-as-msc-qist
#SBATCH --account=education-as-msc-sec
#SBATCH --account=education-as-msc-set
CEG (CiTG)¶
#SBATCH --account=education-ceg-msc-aes
#SBATCH --account=education-ceg-msc-agp
#SBATCH --account=education-ceg-msc-ce
#SBATCH --account=education-ceg-msc-cme
#SBATCH --account=education-ceg-msc-envm
#SBATCH --account=education-ceg-msc-til
EEMCS (EWI)¶
#SBATCH --account=education-eemcs-msc-am
#SBATCH --account=education-eemcs-msc-cese
#SBATCH --account=education-eemcs-msc-cosse
#SBATCH --account=education-eemcs-msc-cs
#SBATCH --account=education-eemcs-msc-dsait
#SBATCH --account=education-eemcs-msc-ee
#SBATCH --account=education-eemcs-msc-set
IDE (IO)¶
#SBATCH --account=education-ide-msc-dfi
#SBATCH --account=education-ide-msc-ipd
#SBATCH --account=education-ide-msc-spd
ME¶
#SBATCH --account=education-me-msc-be
#SBATCH --account=education-me-msc-ewem
#SBATCH --account=education-me-msc-me
#SBATCH --account=education-me-msc-mse
#SBATCH --account=education-me-msc-mt
#SBATCH --account=education-me-msc-ode
#SBATCH --account=education-me-msc-ro
#SBATCH --account=education-me-msc-sc
#SBATCH --account=education-me-msc-tm
TPM (TBM)¶
#SBATCH --account=education-tpm-msc-cosem
#SBATCH --account=education-tpm-msc-epa
#SBATCH --account=education-tpm-msc-ie
#SBATCH --account=education-tpm-msc-mot
BSc students¶
ABE (BK)¶
AE (LR)¶
AS (TNW)¶
#SBATCH --account=education-as-bsc-lst
#SBATCH --account=education-as-bsc-mst
#SBATCH --account=education-as-bsc-nb
#SBATCH --account=education-as-bsc-tn
CEG (CiTG)¶
EEMCS (EWI)¶
#SBATCH --account=education-eemcs-bsc-ee
#SBATCH --account=education-eemcs-bsc-ti
#SBATCH --account=education-eemcs-bsc-tw
IDE (IO)¶
ME¶
#SBATCH --account=education-me-bsc-kt
#SBATCH --account=education-me-bsc-mt
#SBATCH --account=education-me-bsc-wb
TPM (TBM)¶
Teaching courses on DelftBlue¶
If you need access to DelftBlue as a part of the course, then you can get the account name from your supervisor, or from the person responsible for the course you are participating in.
Course organisers
Please contact us so that we can create a correct course sub-group for you!
Which accounts do I have access to?¶
Q: What is the command on DelftBlue to see which accounts I have access to?
A: sacctmgr is a useful command to retrieve such information.
To get information per user:
You will see something like this:
[NetID@login02 ~]$ sacctmgr list user $USER withassoc format='user%-20,account%-45'
User Account
-------------------- ---------------------------------------------
NetID research-eemcs-diam
NetID innovation
What resources limits are associated with my accounts?¶
Q: What are the maximum resources I can use with my account?
A: To get this information, use the following command:
sacctmgr list user $USER withassoc format='user%-20,account%-45,maxjobs,maxsubmit,maxwall,maxtresperjob%-40'
You should see output summarising the resources limits per account that you have access to:
User Account MaxJobs MaxSubmit MaxWall MaxTRES
-------------------- --------------------------------------------- ------- --------- ----------- ----------------------------------------
NetID education-eemcs-courses-delftblueworkshop 8 100 1-00:00:00 cpu=64,gres/gpu=2,mem=250G,node=2
NetID research-eemcs-diam 1000 5-00:00:00 cpu=2400,gres/gpu=8,mem=10000G,node=50
NetID innovation 4 10 1-00:00:00 cpu=64,gres/gpu=2,mem=250G,node=2
How do I get usage statistics for my username?¶
Q: What is the command on DelftBlue to see how much you as a user have used over time?
A: sreport is a useful command to retrieve such information.
To get information per user:
You should see output summarising your usage, per account:
--------------------------------------------------------------------------------
Cluster/User/Account Utilization 2022-06-01T00:00:00 - 2023-03-30T23:59:59 (26179200 secs)
Usage reported in CPU Hours
--------------------------------------------------------------------------------
Cluster Login Proper Name Account Used Energy
--------- --------- --------------- --------------- -------- --------
delftblue dpalagin Dennis Palagin research-eemcs+ 14728 0
delftblue dpalagin Dennis Palagin innovation 3812 0
You can also get information on the overall usage of your faculty:
sreport cluster AccountUtilizationByAccount -t Hours start=2023-01-01 Accounts=research-eemcs | head --lines=7
--------------------------------------------------------------------------------
Cluster/Account/User Utilization 2023-01-01T00:00:00 - 2023-03-30T23:59:59 (7686000 secs)
Usage reported in CPU Hours
--------------------------------------------------------------------------------
Cluster Account Login Proper Name Used Energy
--------- --------------- --------- --------------- ---------- --------
delftblue research-eemcs 1402616 0
How do I check my recent jobs and their efficiency?¶
To list the jobs you ran recently, see Which jobs did I run recently?. To check how much of the requested CPU time and memory a job actually used, see Checking Job Efficiency with seff.