Python Training Setup Guide for the BGU HPC Cluster

Introduction

This guide explains the one-time setup required to configure a Python training workload on the BGU HPC cluster. Once configured, you can refer to refer to the streamlined usage guide.

Prerequisites

  • Completed the SSH setup guide.

  • Have a Conda environment configured in your home directory. Learn more about managing environments here.

  • Have access to the BGU HPC cluster with VPN/internal network.

Running Python Scripts from the Command Line

To run a Python training job on our cluster, your script must be executable from the command line. The cluster initiates tasks using command-line instructions, much like you would run python main.py --epochs 50 --threshold 0.7 on your local machine. Designing your script to be command-line compatible ensures seamless integration with the cluster’s environment.

A common practice is to use a main.py file with an if __name__ == '__main__' block to control the training process. Using command-line arguments is an effective way to experiment with different parameters and hyperparameters. Python offers several libraries for parsing command-line arguments, including the built-in sys.argv for simple cases and argparse for more complex scenarios.

You are responsible for implementing the command-line interface for your script.

Next Steps

Your Python setup is done! You can now refer to refer to the streamlined usage guide to start using Python for your work.

Good luck!