Simple job with data manipulation

From MediaWiki

(Difference between revisions)
Jump to: navigation, search
 
(16 intermediate revisions not shown)
Line 1: Line 1:
-
This is simple single job submitting example with data management. In this case, WN will download source code and executing script from SE and after successful ending of job, result will be uploaded on SE.  
+
This is advanced job submitting example with data management. In this case, WN will download source code and executing script from SE and after successful ending of job, result will be uploaded on SE.  
-
[http://wiki.ipb.ac.rs/images/8/86/Hello.SE.tgz hello.SE.tgz]
+
[http://wiki.ipb.ac.rs/images/7/79/Src.tgz src.tgz]
-
Content of uploaded files:
+
(You can download src.tgz directly in your working folder with "wget <nowiki>http://wiki.ipb.ac.rs/images/7/79/Src.tgz</nowiki>")
 +
 
 +
 
 +
Content of uploaded file:
'''hello.c'''
'''hello.c'''
Line 15: Line 18:
  }
  }
-
'''hello.sh'''
+
1. Make a folder "adv"
-
  #Compile C file
+
  [ngrkic@ui jobovi]$ mkdir adv
-
+
-
gcc hello.c -o hello
+
-
+
-
#Execute hello program
+
-
+
-
./hello
+
 +
2. Enter adv folder
 +
[ngrkic@ui jobovi]$ cd adv
-
1. Extract files
+
3. Extract arhive
-
  [ngrkic@ui jobovi]$ tar xvzf hello.SE.tgz
+
  [ngrkic@ui adv]$ tar xvzf src.tgz
-
2. Enter directory hello.SE
 
-
[ngrkic@ui jobovi]$ cd hello.SE
+
4. Make a adv.jdl file with next content
-
3. List directory
 
-
 
-
[ngrkic@ui hello.SE]$ ll hello.SE
 
-
 
-
4. See content of files
 
-
 
-
[ngrkic@ui hello.SE]$ cat hello.jdl
 
  Executable = "run.sh";
  Executable = "run.sh";
  Arguments = "";
  Arguments = "";
Line 50: Line 41:
  Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
  Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
-
[ngrkic@ui hello.SE]$ cat run.sh  
+
5. Make a run.sh file with following content
 +
 
  #Downloading source code from SE
  #Downloading source code from SE
-
  lcg-cp lfn:/grid/aegis/nikola/hello.SE/hello.c 'pwd'
+
  lcg-cp lfn:/grid/aegis/power/src.tgz 'src.tgz'
 +
 +
#Extract code
 +
tar xvzf src.tgz
 +
 +
#Compile C file
 +
 +
gcc hello.c -o hello
 +
 +
#Execute hello program
   
   
-
  #Downloading script that compiles source code and execute program
+
  ./hello >> output
-
lcg-cp lfn:/grid/aegis/nikola/hello.SE/hello.sh 'pwd'
+
   
   
-
  #runing compile & execute script
+
  #Make tgz arhive of output
-
  ./hello.sh
+
  tar cvzf output.tgz output
   
   
  #Uploads output to SE
  #Uploads output to SE
-
  lcg-cr -l lfn:/grid/aegis/nikola/hello.SE/std.out file:`pwd`/std.out
+
lcg-cr -l lfn:/grid/aegis/nikola/hello.SE/output.tgz file:`pwd`/output.tgz
 +
  lcg-cr -l lfn:/grid/aegis/nikola/hello.SE/std.err file:`pwd`/std.err
-
5. Creating VOMS proxy:  
+
 
 +
6. Creating VOMS proxy:  
  [ngrkic@ui hello.SE]$ voms-proxy-init -voms aegis
  [ngrkic@ui hello.SE]$ voms-proxy-init -voms aegis
-
6. Submiting hello job:  
+
7. Submiting adv job:  
-
  [ngrkic@ui hello.SE]$ glite-wms-job-submit -a hello.jdl
+
  [ngrkic@ui hello.SE]$ glite-wms-job-submit -a adv.jdl
-
7. Requesting Job status:  
+
8. Requesting Job status:  
  [ngrkic@ui hello.SE]$ glite-wms-job-status https://wms.ipb.ac.rs:9000/PhTXU-tuefu4iuvIotYG9g
  [ngrkic@ui hello.SE]$ glite-wms-job-status https://wms.ipb.ac.rs:9000/PhTXU-tuefu4iuvIotYG9g
-
 
   
   
  ======================= glite-wms-job-status Success =====================
  ======================= glite-wms-job-status Success =====================
  BOOKKEEPING INFORMATION:
  BOOKKEEPING INFORMATION:
   
   
-
  Status info for the Job : https://wms.ipb.ac.rs:9000/PhTXU-tuefu4iuvIotYG9g
+
  Status info for the Job : https://wms.ipb.ac.rs:9000/g2ntHSCHYN7oeH2l9eS_dg
-
  Current Status:    Done (Exit Code !=0)
+
  Current Status:    Done (Success)
-
  Exit code:          1
+
Logged Reason(s):
-
  Status Reason:      Warning: job exit code != 0
+
    -
 +
    - Job terminated successfully
 +
  Exit code:          0
 +
  Status Reason:      Job terminated successfully
  Destination:        ce64.ipb.ac.rs:2119/jobmanager-pbs-aegis
  Destination:        ce64.ipb.ac.rs:2119/jobmanager-pbs-aegis
-
  Submitted:          Fri Dec 9 15:09:30 2011 CET
+
  Submitted:          Sat Dec 10 22:54:28 2011 CET
  ==========================================================================
  ==========================================================================
 +
 +
9. Download output file from SE
 +
 +
[ngrkic@ui hello.SE]$ lcg-cp lfn:/grid/aegis/nikola/hello.SE/output.tgz file:/home/ngrkic/jobovi/hello.SE/output.tgz
 +
 +
10. Extract arhive
 +
 +
tar xvzf output.tgz
 +
 +
10. Cat output
 +
 +
hello world!!

Latest revision as of 18:55, 27 May 2012

This is advanced job submitting example with data management. In this case, WN will download source code and executing script from SE and after successful ending of job, result will be uploaded on SE.

src.tgz

(You can download src.tgz directly in your working folder with "wget http://wiki.ipb.ac.rs/images/7/79/Src.tgz")


Content of uploaded file:

hello.c

#include <stdio.h>

int main()
{
 printf ("hello world!!\n");
 return 0;
}

1. Make a folder "adv"

[ngrkic@ui jobovi]$ mkdir adv

2. Enter adv folder

[ngrkic@ui jobovi]$ cd adv

3. Extract arhive

[ngrkic@ui adv]$ tar xvzf src.tgz


4. Make a adv.jdl file with next content

Executable = "run.sh";
Arguments = "";
StdOutput = "std.out";
StdError = "std.err";
InputSandbox = {"run.sh"};
OutputSandbox = {"std.out", "std.err"};
Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);

5. Make a run.sh file with following content

#Downloading source code from SE
lcg-cp lfn:/grid/aegis/power/src.tgz 'src.tgz'

#Extract code 
tar xvzf src.tgz

#Compile C file

gcc hello.c -o hello

#Execute hello program

./hello >> output

#Make tgz arhive of output 
tar cvzf output.tgz output

#Uploads output to SE
lcg-cr -l lfn:/grid/aegis/nikola/hello.SE/output.tgz file:`pwd`/output.tgz
lcg-cr -l lfn:/grid/aegis/nikola/hello.SE/std.err file:`pwd`/std.err


6. Creating VOMS proxy:

[ngrkic@ui hello.SE]$ voms-proxy-init -voms aegis

7. Submiting adv job:

[ngrkic@ui hello.SE]$ glite-wms-job-submit -a adv.jdl

8. Requesting Job status:

[ngrkic@ui hello.SE]$ glite-wms-job-status https://wms.ipb.ac.rs:9000/PhTXU-tuefu4iuvIotYG9g

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://wms.ipb.ac.rs:9000/g2ntHSCHYN7oeH2l9eS_dg
Current Status:     Done (Success)
Logged Reason(s):
    - 
    - Job terminated successfully
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        ce64.ipb.ac.rs:2119/jobmanager-pbs-aegis
Submitted:          Sat Dec 10 22:54:28 2011 CET
==========================================================================

9. Download output file from SE

[ngrkic@ui hello.SE]$ lcg-cp lfn:/grid/aegis/nikola/hello.SE/output.tgz file:/home/ngrkic/jobovi/hello.SE/output.tgz

10. Extract arhive

tar xvzf output.tgz

10. Cat output

hello world!!
Personal tools