...
What are the current Cluster queues?
Additional nodes were added to the cluster along with new queues. These changes streamline performance and application integration support. In addition contributed nodes by faculty are listed below. Use of contributed nodes is authorized by node owner. If you are not authorized to use the contributed node(s) you may use the short_allall6, normal_all all6 or long_all all6 queues at lower priority but subject to preemption and other constraints.
...
Node owner | New queue | Old queue | Comments |
|---|---|---|---|
Public | admin_public | admin | system usage only |
| express_publicpublic6 | express | short term high priority, special needs |
| int_publicpublic6 |
| interactive jobs, high priority, 4 hour limit |
| exbatch_publicpublic6 |
| batch jobs, high priority,low capacity, 2 week limit, exclusive |
| paralleltest_publicpublic6 | paralleltest | short term debugging |
| short_gpu |
| 2 hour limit |
| normal_gpu |
| 72 hour limit |
| long_gpu |
| 28 days limit |
| short_publicpublic6 | short | 2 hour limit |
| parallel_publicpublic6 | parallel | 2 week limit 2-64cpus |
| normal_publicpublic6 | normal | default queue - 3 days limit 1 cpu |
| long_publicpublic6 | long | 28 days 1 cpu |
| dregs_publicpublic6 | dregs | 364 days 1 cpu |
Public shared | short_all |
| shared across all nodes, lower priority |
| normal_allall6 |
| shared across all nodes, lower priority |
| *long_allall6 |
| shared across all nodes, lower priority |
ATLAS | atlas_prod |
| Physics Atlas support |
| atlas_analysis |
| Physics Atlas support |
Khardon | int_khardon |
| contributed nodes, interactive 4 hour limit |
| express_khardon |
| contributed nodes, 30 minute limit |
| short_khardon |
| contributed nodes, 2 hour limit |
| normal_khardon |
| contributed nodes, 3 day limit |
| long_khardon |
| contributed nodes, 14 day limit |
Miller | int_miller |
| contributed nodes, interactive 4 hour limit |
| express_miller |
| contributed nodes, 30 minute limit |
| short_miller |
| contributed nodes, 2 hour limit |
| normal_miller |
| contributed nodes, 3 day limit |
| long_miller |
| contributed nodes, 14 day limit |
Abriola | int_abriola |
| contributed nodes, interactive 4 hour limit |
| express_abriola |
| contributed nodes, 30 minute limit |
| short_abriola |
| contributed nodes, 2 hour limit |
| normal_abriola |
| contributed nodes, 3 day limit |
| long_abriola |
| contributed nodes, 1 year limit |
Napier | int_napier |
| contributed nodes, interactive 4 hour limit |
| express_napier |
| contributed nodes, 30 minute limit |
| short_napier |
| contributed nodes, 2 hour limit |
| normal_napier |
| contributed nodes, 3 day limit |
| long_napier |
| contributed nodes, 1 year limit |
...
And extra details by queue name:
-bash-3.2$ bqueues -l atlasnormal_prodpublic6
What is the default queue?
If you do not specify a queue by name in your bsub arguments, your job goes to the default queue, which is normal_public.
...
-bash-3.2$ bsub -R Mem24 -queue normal_public public6 ./myprogram
I see that there are some nodes with more than 32gig ram, such as 48 and 96 gig. How do I access them in exclusive mode since I need almost all the ram and my expected job duration is just minutes?
-bash-3.2$ bsub -q express_public public6 -x -R Mem48 ./myprogram
I have a program that is threaded. Is it possible to submit a job that will "take over" a node, or at least the 4 cores on a single chip?
...
How does one use a node exclusively?
Currently the only queues that allow exclusive use is the express_publicpublic6 and exbatch_publicpublic6 queues. However, not all jobs are suitable, so please inquire with an email to cluster support and describe what you intend to do.
How does one actually invoke a job exclusively?
LSF bsub command has the -x option. To send your job to a node that has extra memory and runs exclusively for hours.
-bash-3.2$ bsub -q exbatch_public public6 -x -R Mem16 ./myprogram
How does one make use of nodes with /scratch2 storage?
Note that is is disk storage and not ram memory.
Access to this storage is by request. Please make this request via cluster-support@tufts.edu.
If you submit a job with the following, LSF will place a job on nodes with /scratch2 partitions.
For example, to request at least 40gig of storage for a job to run in the long_public queue try:
-bash_3.2$ bsub -q long_public public6 -R "scratch2 > 40000" ./your_jobname
...
I have a job in one queue, but would rather have it in another. How do I migrate the job?
Use the lsf command, bmod. For example:
-bash-3.2$ bmod -q express_public public6 <job_number>
This will migrate your job with <job_number> to the express_public queue or some other queue.
...
If you have ssh on your workstation, try the following:
> ssh clustercluster6.uit.tufts.edu ". /etc/profile.d/lsf.sh && bsub -q queuename ./yourprogram"
where queuename is one of the above queues.
Suppose I want to copy data via scp from my bash script that is running on a compute node to the /scratch/utln storage area of the login node. How do I reference it?
scp filename tunictunic6.uit.tufts.edu:/scratch/utln
Note, your utln username is needed.
...
What is the path to reference from a job on a compute node to the storage on the login node?
/cluster/scratch/tunictunic6/utln/ ....
How do I convert mixed case file names in a directory to lower case?
...