Get Services >  High Performance Computing >  How to Run Batch Job >  Advanced Topics

Job Migration

A program can be migrated from one system to another using features built into LSF. However, the following conditions apply:

  • The program must be binary compatible across all the systems that it can potentially be migrated to.
  • The basic requirements for running an LSF job must still be met.

After the above conditions have been satisfied, you need to consider 2 scenarios:

  • If your program is checkpointable
  • If your program is not checkpointable

Checkpointable Jobs

If your program is checkpointable, and you have specified the checkpoint directory, LSF will perform a checkpoint of your job before migration.

Non-checkpointable Jobs

If your program is not checkpointable, or if you did not submit with the -k option, then LSF will terminate your program, re-queue and re-launch the job on the new server. The program will have to start from scratch.


Commands

To migrate a job, use command bmig:

   > bmig [JobId] [-m hostname]

If the jobid is not specified, LSF will migrate the most recently submitted job. If you also did not specify the host, LSF will choose the most suitable host that can run, based on the queue and host loads and threshold settings. (In general, LSF does not check to see if the program is binary compatible. If the job is sent to a wrong host, the program will fail and a message will be sent to the output/error file).


File transfers between submission host and execution host

It is possible to transfer files to the submission host before running a job and transfer the results file back after a program has finished, all within the same bsub command. This is provided that you know the exact filenames of the input and output files you want.


Commands

The bsub with the -f command can be used to transfer files between hosts:

> bsub -f "localfile op remotefile" ... ./a.out

where op can be "<", ">" or a combination of both. Note that "<>" means the same thing as "><" in this case. For each file, you will need a separate -f option. The bsub command can accept multiple -f options.


Automatic Restarting of Jobs

In the event that a job is terminated prematurely, LSF can be set to automatically re-start the job by adding the '-r' option to the bsub command:

> bsub -r -q [queuename] ./a.out

Jobs that have been checkpointed will be restarted from the latest checkpointed state. Non-checkpointable jobs will be restarted from the scratch.


Scheduling Jobs to run at a specified time

You can use the '-b' option in the bsub command to specify a specific time to launch a job, for example:

> bsub -b 11:06:23:59 -q [queuename] ./a.out
Job <212333> is submitted to queue <queuename>.
Job will be scheduled after Tue Jun 16 23:59:00 2009

LSF will only consider this job for submission after the specified date and time.

top