Distributed Computing

Matthieu Pizenberg, Thomas Forgione
Your 10h+ computing sessions in the blink of an eye!
2016

Python code Bash helpers

If you happen to have ssh access to a network of 200 computers unused at night, you can have a lot of computing fun. This mini-project is composed of two parts,

  1. a python library with two main scripts start_computing_server.py and start_computing_client.py that connect to each other and distribute work,
  2. a set of utilitary bash scripts to help prepare the remote machines before work distribution, such as sending files, synchronizing some directories, etc.

The general workflow of setup, work distribution and collection is as follows.

Distributed workflow