2.

Bioinformatics services

Bioinformatics services

a. Timings for Genome Mapping

The newest update to SLURM has better handling of backfill, which means if you specify a expected time for your program to run it can be placed earlier as nodes open up. Using sbatch you can specify a limit on the total run-time with -t or --time d-h:m:s. Times can be specified as min, min:sec, hr:min:sec, day-hr, day-hr:min, and day-hr:min:sec. So -t 5 means five minutes -t 5:00:00 is five hours.

Here are some approximate run times (min sec) for BWA-mem, bowtie2, and samtools with 7, 15, 30, and 60 million fastq sequences. Samtools was used to convert SAM files and then sort the resulting BAM files.  These runs were done with 8 cores and 16 GB memory. The timings were obtained using the time command  ( e.g. time bwa mem -t 8  <hg38 ref> <sequences (human)> ) though some  programs report the runtimes in the output. To effectively use the back fill, take note of how long your programs run and add a bit more time to give your programs some extra run time using the -t or --time parameter.  Make sure to add run times for workflows, e.g. a bwa mem  run followed by  samtools reformatting as well as accounting for multiple runs, e.g. processing 6 fastq files. 

b. Genome Indexes on Cluster

Several mammalian and model system genomes, indexes, and annotations are located on the Tufts HPC cluster.  Currently the genomes are listed below in the indicated directory tree are UCSC genome builds, except for canFam3 which is a NCBI build.

/cluster/tufts/genomes
  /HomoSapiens
    /hg18
    /hg19
    /hg38 (New)
  /MusMusculus
    /mm9
    /mm10
  /RattusNorvegicus
    /rn4
    /rn5
    /rn6 (New)
  /CanisFamiliaris
    /canFam2
    /canFam3
  /BosTaurus
    /bosTau7
  /EquusCaballus
    /equCab2
  /GallusGallus
    /galGal4
  /SusScrofa
    /susScr3
  /DrosophilaMelanogaster
    /dm3
    /dm6 (New)
  /CaenorhabditisElegans
    /ce10 
  /DanioRerio
    /danRer7
    /danRer10 (New)
 /SaccharomycesCerevisiae  (no Transcriptome)
    /sacCer2   
    /sacCer3 

  /Annotation
  /Sequence

In the Annotation directory there are subdirectories for gene annotations ( Gene), and depending upon the degree of annotation, directories for smallRNA and Variation. 

Under the Sequence directory, there are subdirectories containing indexes for popular short read sequence mapping programs.

  /AbundantSequences -- data files with over-represented sequences 
  /BlastDB  -- blast formatted genomic indexes: use genome.fa as reference name
  /Bowtie2Index  -- Bowtie2 formatted indexes: use genome as reference name
  /BowtieIndex  -- Bowtie formatted indexes: use genome as reference name
  /BWAIndex  -- BWA formatted indexes: use genome.fa as reference name
  /Chromosomes  -- individual chromosomes as fasta files
  /Transcriptome  -- Bowtie2 formatted index of transcriptome sites: use transcript as ref name
  /WholeGenomeFasta -- Genome as one file with accessory files

Please read the documentation for a mapping program to understand the way in which the reference indexes are referred.

Examples

The examples listed below should be included in a script and then submitted

Example: BWA

It helps to set up environmental variables to avoid having to type long paths. Here a set of short reads ( myreads.fq) are mapped to the mouse genome (mm10) with a SAM formatted file as output. Note that bwa uses genome.fa as a reference index name and the bwa mem analysis is used. See the BWA documentation for other ways to invoke bwa.

  module load bwa/0.7.9a
  
  export MM10=/cluster/tufts/genomes/MusMusculus/mm10/Sequence/BWAIndex
  export MYDATADIR=/cluster/shared/myutln/mmdata

  bwa mem $MM10/genome.fa $MYDATADIR/myreads.fq >$MYDATADIR/myreads.sam

Example: Bowtie2

Similarly, environmental variables can be set up, and in the case of bowtie2  a BOWTIE2_INDEXES variable must be set also. Here we have an example of a paired end analysis, with minimal options. See the bowtie2 documentation for a complete set of command options. Note Bowtie2 uses genome as reference index name   (-x genome ).

  module load bowtie2

  export BOWTIE2_INDEXES=/cluster/tufts/genomes/MusMusculus/mm10/Sequence/Bowtie2Index
  export MYDATADIR=/cluster/shared/myutln/mmdata
  
  bowtie2 -q -x genome -1 $MYDATADIR/myreads_1.fq -2 $MYDATADIR/myreads_2.fq -S $MYDATADIR/myreads.sam

c. HPC Modules for Bioinformatics

  Note: some bioinformatic software modules, such as R modules like bioconductor or python modules, are not listed here because they are part of a larger module, for example R/3.1.0 or python/2.7.6. Load those modules to get to bioconductor or python modules such as numpy or matplotlib.

  To list the entirety of the module collection use this command

       module avail

To load a module use this command

       module load modulename/version

 as listed below. Default settings are annotated by '*'

       module list

shows currently loaded modules.

To unload a module use this command

       module unload modulename/version

Performance Considerations  using threads

In general, there are useful performance gains using threads, but it can also be abused by using too many.  Applications supporting thread parallelism may have varying degree of internal support.

Application performance is not always well documented and it may be beneficial to you to do some benchmarking.  By doing so you will be in a position to better utilize the cluster resources. For example here is a benchmark examination of blastp and other tools.  

d. Tufts Center for Neuroscience Research Genomics Core

The Tufts CNR Genomics Core supplies links to bioinformatics resources related to their operation. See Tufts CNR Genomics Core Resources for more information.

Misc

A separate server is used to support these services in some cases. However some software may require installation on the linux research cluster. Check the Installed Software for Bioinformatic software available on the cluster. To make a special request for software installation, please follow the instructions as noted elsewhere on this page.