Compiling for GPUs
CUDA is installed and available for users. Simply find the version you would like to use and add it to your path by,
module avail cuda module load cuda/4.0RC2 (to load version 4.0RC2)
This module has libraries for CUDA and OpenCL programming. One will need the -lcuda or
-lopencl flags for CUDA or OpenCL, respectively.
Disclaimer: If you would like to use more than 4 GPUs per job, please contact help@xsede.org
Scheduling of GPUs
For both batch and interactive jobs that utilize GPUs one can request this resource
by -l gpus=N. Currently, there is a limitation in that Moab interprets this value
as GPUs-per-core, which may take some time to resolve. In the meanwhile, the
following combinations are known to work:
-l ncpus=[1-2],gpus=[1-2]
To determine which GPUs one has allocated, the environment variable $PBS_GPUS
can be used. For example, here is the result for a job that requests four GPUs and
four CPUs (-l ncpus=2,gpus=2):
> cat $PBS_GPUS /dev/nvidia1 /dev/nvidia0 /dev/nvidia3 /dev/nvidia2
