Accounts

Account related FAQs:

How do I get an account and whom do can I contact if I have account related problems?
Access to the Cluster or Bioinformatic server requires a Research Cluster account. Faculty, other staff, and students are required to fill out the UIT Research Computer Account Application form.

Please contact the TTS service desk at x7-3376 or by email at it@tufts.edu if further assistance is needed.

What is the name of the cluster?

Is the cluster a suitable environment for project or class usage?
This depends.... Tufts provides a service called Trunk that is suitable for general collaboration and class support. These services are not on the cluster. However, sharing research data on the cluster is possible through optional research storage mounted in support of some project. See Cluster Storage Options section on this wiki for details.

What is the Tufts responsible use policy?

Find it here.

Do I need permission to download and install software under my account?
NO.

How do I login to the cluster?

Use your Tufts UTLN and password associated with your Tufts email.

What is a Tufts UTLN?

This is your Tufts username issued for purposes of Tufts email.

I have forgotten my password, how do I change my password?

If you have forgotten your Tufts password and need assistance
resetting it, please contact  TTS support center x73376.

I am unable to log onto the cluster despite having a valid UTLN. The message I see is that I am being rejected. This worked before, what can I do?

It is likely that your IP address is blocked by the cluster. It may be that you have the wrong password. A threshold of 10 failed login attempts will result in a locked account. Contact us at tts-support@tufts.edu so that your account can be reset.  Otherwise you may have to visit the following site to find out what your remote IP address is so that we can unblock it. There are many reasons why this might be the case. Copy the IP addresses displayed there and send it back to us via the cluster support email venue, tts-support@tufts.edu with an explanation.

I am going to graduate in May, how long do I have access to the cluster?
As a May graduate your eligibility for Tufts accounts (email and cluster included) will continue through the end of August. At the end of August when we process the term change for the Fall term your account will begin its 4 week expiration. This means that you will actually have your accounts through the 3rd or 4th week of September.

When I login I see the following -bash-3.2$ , what is it?

This is the default prompt for your bash shell. This is where you issue linux commands.

How to transfer files?

Any file transfer agent program supporting either scp or sftp protocol will work. There are many freeware choices.WinScp for Windows PCs is very good.

Use the file transfer node, xfer.cluster.tufts.edu  and not the cluster login node for all file transfers.  This node mounts all the directories that are available to you on the node:  login.cluster.tufts.edu  

Also, of interest, the graphical file transfer program filezilla is available to Windows, linux and unix users.

For Mac users consider fugu. Fugu is a graphical frontend to the commandline Secure File Transfer application (SFTP) and supports SFTP, SCP and SSH protocols. Another of the many free options of this type is cyberduck

What are some consequences of transferring files from my Windows PC?
If you create and edit ascii text files on your PC and transfer them to the cluster, you may experience odd error messages when you try to use them. Some of these messages may mention that the file is not found or is corrupt. Consider not doing this and instead edit and create the file(s) on the cluster using either a text editor or graphical text editor. There are many choices discussed on this wiki such as nano, nedit. There is a conversion tool called dos2unix that will make the conversion for you. A man page with documentation is available at the command line:
> man dos2unix

What is my home directory?

This is where you have access to storage that allows you to read, write and delete files. The path to your home directory is: /cluster/home/xx/yy/utln

where xx is the first letter of your first name

where yy is the first letter of your last name

where utln is your Tufts issued username for purposes of email.

Note, newer account creations(2016) will find a different directory structure: /cluster/home/your_utln

What is the disk quota on home directories?

All accounts are created with a 5 gigabyte quota.

May I request a disk quota increase to my home directory?

Often the best solution is to use cluster specific temporary storage options in conjunction with your desktop. Programs such as WinScp allow you to drag and drop files between the cluster and your desktop. You may use /scratch/utln for additional short term storage or if you require 100's of gigabytes, consider requesting access to /cluster/shared/ storage.

How do I keep track of all my commands that I type at the shell prompt? I hate having to type them over and over.

Your login shell is a linux bash shell. By default all your commands typed during a session are available to you by using the history command. To view them:
> history | more

You may replay any listed command by using the bang ! operator. For example to replay line 27 in the command history:
> !27

Also note that you can use the up and down arrow keys to view and edit on the command line.

When you logout your history is cleared. However, you may save it to a file by:
> history > my_history.txt

How can I increase the size of the history and have it persist across logins?

This is done by creating a text file called .bashrc and insert the line:
history -a -r 1000

This will keep a moving window of 1000 commands.
After you create this file, logout and then login to make it active.

What does Ctrl R do?
This is another way to search your history of commands. At the linux prompt, issue a Ctrl R sequence and then enter a search string. If found, you can cycle through all of them by doing Ctrl R repeatedly. Once the former command is found, issue Enter to send the command.

Storage related issues

How do I find out how much total diskspace I have used?

Login to your account and type the following:

-bash-3.2$ du . -sk

or

-bash-3.2$ quota -s

You may also use WinScp on your desktop or other such file transfer tools to view file summaries.

Other command line options to view what files and how big:

How do I find my largest ten files in my home directory and subdirectories?

How can I compress some old files so that I can save space?

Suppose you wish to compress lots of text files ten days or older with file extensions of the same type:

The resulting file will have a .gz added to the end of the files compressed.

Why should I pay attention to my home dir. disk usage?
When you go over quota some programs may fail in doing some tasks. It will appear as a mystery to you until you clean up and get back under quota.

How do I obtain access to additional temporary storage on file system /cluster/shared/ ?
You already have access to this filesystem. Look for a sub-directory named with your UTLN/username.

What is the local storage automated cleaning policy?

The head node of the cluster and compute nodes provide three areas of temporary storage separate from your home directory. File systems /scratch/utln/, /cluster/shared/utln/ and /tmp are available to support different storage needs. Each is subject to automated cleaning rules. All files older than 28 days are deleted and these file systems are not backed up! See the section on storage options for additional information.

Backups

What about backups to my home directory?
Your data residing in your home directory is automatically backed up by TTS. A moving window of one year is used for backup purposes. Your home directory is backed up in two ways; Tape and Snapshots.

It is likely that newer storage technology will over time change and the use of tape will be reduced.

Snapshots

Snapshots are provided for short term file access and recovery by the user. You may copy one or more of your snapshot files from the following location: /cluster/home/.snapshot/
Note this in not your home directory but instead an image of your files. In the directory you will see several directories with different look back windows such as:

-bash-3.2$ ls /cluster/home/.snapshot/
hourly.0 hourly.1 hourly.2 hourly.3 hourly.4 hourly.5 nightly.0 nightly.1 weekly.0 weekly.1

Within the directory of interest, you should navigate to your utln account in order to copy the file(s).
For example to see a 4 hour window for user John Doe, choose John Doe's account, /cluster/home/.snapshot/hourly.4/j/d/jdoe02/

Assuming you are in your home directory, to retrieve file abcdef.cc , do the following:
-bash-3.2$ cp /cluster/home/.snapshot/hourly.4/j/d/jdoe02/abcdef.cc old_abcdef.cc

For Research Storage NFS mounted data, use the appropriate path to your storage at the top level:

/cluster/tufts/your_research_share_name/.snapshot/

To retrieve files older than a week one must request a backup from Tape. To request a restore of data, contact the TTS support center x73376. You should have available basic info such as the file name(s) and approximately when it existed and what directory.

Sometimes my allocated space on a research storage share seems to indicate less space than allocated. What is going on?

Snapshots are doing their magic by adjusting to any data deletions and additions. Deletions can seem to temporarily make your quota less than it really is, but it will adjust back. Snapshots will adjust themselves and not eat into your quota. Also snapshot data will expire and you might see your available space increase slightly, but will adjust down in time. This is not something to worry about.