Hybrid job example
From MediaWiki
In this exercise user should obtain prepared hybrid job, extract archive, list content of files, submit job on PARADOX cluster, monitor his progress with information from queue and when job is done list resaults file.
1. Login on ui.ipb.ac.rs:
$ ssh ngrkic@ui.ipb.ac.rs
2. Navigate to your folder in nfs filesystem.
$ cd /nfs/ngrkic
3. Download tgz archive with example files.
wget http://wiki.ipb.ac.rs/images/b/bb/Hybrid.tgz
4. Extract archive :
$ tar xvzf Hybrid.tgz
5. Enter Mpich folder
$ cd Hybrid
6. List content of folder:
$ ll
7. List content of job.pbs and hybrid_example.c files:
$ cat job.pbs
#!/bin/bash #PBS -q hpsee #PBS -l nodes=4:ppn=8 #PBS -l walltime=10:00:00 #PBS -e ${PBS_JOBID}.err #PBS -o ${PBS_JOBID}.out export OMP_NUM_THREADS=8 cd $PBS_O_WORKDIR chmod +x job ${MPI_OPENMPI_MPIEXEC} -np 4 -npernode 1 ./job
$ cat hybrid_example.c
#include <mpi.h> /* MPI Library */
#include <omp.h> /* OpenMP Library */ #include <stdio.h> /* printf() */ #include <stdlib.h> /* EXIT_SUCCESS */
int main (int argc, char *argv[]) { /* Parameters of MPI.
- / int M_N; /* number of MPI ranks
- / int M_ID; /* MPI rank ID
- / int rtn_val; /* return value
- / char name[128]; /*
MPI_MAX_PROCESSOR_NAME == 128 */ int namelen;
/* Parameters of OpenMP. */ int O_P; /* number of OpenMP processors */ int O_T; /* number of OpenMP threads */ int O_ID; /* OpenMP thread ID */
/* Initialize MPI. */ /* Construct the default communicator MPI_COMM_WORLD. */ rtn_val = MPI_Init(&argc,&argv);
/* Get a few MPI parameters. */ rtn_val = MPI_Comm_size(MPI_COMM_WORLD,&M_N); /* get number of MPI ranks */ rtn_val = MPI_Comm_rank(MPI_COMM_WORLD,&M_ID); /* get MPI rank ID */ MPI_Get_processor_name(name,&namelen); printf("name:%s M_ID:%d M_N:%d\n", name,M_ID,M_N);
/* Get a few OpenMP parameters. */ O_P = omp_get_num_procs(); /* get number of OpenMP processors */ O_T = omp_get_num_threads(); /* get number of OpenMP threads */ O_ID = omp_get_thread_num(); /* get OpenMP thread ID */
printf("name:%s M_ID:%d O_ID:%d O_P:%d O_T:%d\n", name,M_ID,O_ID,O_P,O_T);
/* PARALLEL REGION */ /* Thread IDs range from 0 through omp_get_num_threads()-1. */
/* We execute identical code in all threads (data parallelization). */ #pragma omp parallel private(O_ID) { O_ID = omp_get_thread_num(); /* get OpenMP thread ID */ MPI_Get_processor_name(name,&namelen); printf("parallel region: name:%s M_ID=%d O_ID=%d\n", name,M_ID,O_ID); } /* Terminate MPI. */ rtn_val = MPI_Finalize(); /* Exit master thread. */ printf("name:%s M_ID:%d O_ID:%d Exits\n", name,M_ID,O_ID);
return EXIT_SUCCESS;
}
8. Submit job :
qsub job.pbs
qsub will print output :
<jobID>.ce64.ipb.ac.rs
9. Monitor your job :
qstat <jobID>.ce64.ipb.ac.rs
10.When job is done list content of <jobID>.ce64.ipb.ac.rs.out file :
cat <jobID>.ce64.ipb.ac.rs.out
n06.ipb.ac.rs n06.ipb.ac.rs n08.ipb.ac.rs n08.ipb.ac.rs n13.ipb.ac.rs n13.ipb.ac.rs Node: 2 Node integral: 0.523599 Node: 1 Node integral: 0.523599 Node: 3 Node integral: 0.523599 Node: 4 Node integral: 0.523599 Node: 5 Node integral: 0.523599 Compiled on Oct 12 2011 at 11:34:11 Number of nodes: 6 Number of intervals: 300000000 Node: 0 Node integral: 0.523599 PI exact : 3.1415926535897931 PI estimate: 3.1415926535899650 PI diff : 0.0000000000001719 Wall clock: 1.894531