This post describes some tips for the workstation — specifically, how to configure huggingface libraries so that model files are stored in the appropriate location.

Workstation Hard Drives

The workstation contains several hard drives.

Your home directory is on the default hard drive. This drive is not very big; storing lots of data here can easily fill it up.

There is another storage drive at /mnt/bigstorage, which is much larger.

Storage Guidelines

If you have more than 20GB to store, consider moving that data to /mnt/bigstorage.

If you have more than 50GB to store, strongly consider moving that data to /mnt/bigstorage.

If you have more than 100GB to store, you should definitely store that data in /mnt/bigstorage.

Moving Your Stuff to Bigstorage

If you have access to the workstation, you should have permissions to create a directory in bigstorage:

Create a directory for your username (recommended).

mkdir -p /mnt/bigstorage/<your-username>

Create a subfolder for some specific stuff (optional).

mkdir -p /mnt/bigstorage/<your-username>/<subfolder-for-something-specific>

Huggingface

This link goes to the Huggingface docs on cache management. The Huggingface cache is where libraries like transformers store downloaded model weights. The default location is usually somewhere in your home dir. As mentioned previously, too much stuff in the home dir will fill up the default drive. To remedy this, you can tell Huggingface to put the cache dir somewhere else, like /mnt/bigstorage/<your-username>/hf_cache.

Useful commands

Read up on the du and df terminal commands to see how much disk space is available/used.

man df man du

Read up on rsync, a tool to move files between directories.

man rsync