Job collection

From MediaWiki

(Difference between revisions)
Jump to: navigation, search
 
(10 intermediate revisions not shown)
Line 2: Line 2:
-
This is job collection  submiting example.
+
This is job collection  submiting example. Collection of two simple jobs. First will print hostname, and second will print date.  
-
It does not requre some high skills and knowing JDL language.
+
For demonstration use attached file.
For demonstration use attached file.
-
[http://wiki.ipb.ac.rs/images/a/a2/Jdlcollection.zip jdlcollection.zip ]
+
[http://wiki.ipb.ac.rs/images/7/70/Jdlcollection.tgz jdlcollection.tgz ]
-
+
 
 +
[http://wiki.ipb.ac.rs/images/6/6a/Jdlcollection2.tgz jdlcollection2.tgz ]
 +
 
1.Extract file with:
1.Extract file with:
-
   [ngrkic@ui ~]$ unzip jdlcollection.zip
+
   [ngrkic@ui ~]$ tar xvzf jdlcollection.tgz
2.Enter directory jdlcollection with:
2.Enter directory jdlcollection with:
Line 21: Line 22:
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:07 job1.jdl
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:07 job1.jdl
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:17 job2.jdl
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:17 job2.jdl
 +
 +
Content of file job1.jdl
  [ngrkic@ui jdlcollection]$ cat job1.jdl  
  [ngrkic@ui jdlcollection]$ cat job1.jdl  
Line 27: Line 30:
  StdError = "std.err";
  StdError = "std.err";
  OutputSandbox = {"std.out","std.err"};
  OutputSandbox = {"std.out","std.err"};
 +
 +
Content of file job2.jdl
  [ngrkic@ui jdlcollection]$ cat job2.jdl  
  [ngrkic@ui jdlcollection]$ cat job2.jdl  
Line 34: Line 39:
  OutputSandbox = {"std2.out","std2.err"};
  OutputSandbox = {"std2.out","std2.err"};
-
This is job collection of two jobs.First will print hostname , and second will print date.
 
-
 
+
Then return one folder up with :
-
then return one folder up with :
+
  [ngrkic@jdlcollection ~]$ cd ..
  [ngrkic@jdlcollection ~]$ cd ..
Line 53: Line 56:
  Your proxy is valid until Fri Oct  2 01:11:00 2009
  Your proxy is valid until Fri Oct  2 01:11:00 2009
-
 
-
Now we are ready to submit job collection.
 
Line 61: Line 62:
  [ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection
  [ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection
-
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
+
  Connecting to the service <nowiki>https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server</nowiki>
   
   
   
   
Line 69: Line 70:
  Your job identifier is:  
  Your job identifier is:  
   
   
-
https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
<nowiki> https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki>
   
   
  ==========================================================================
  ==========================================================================
Line 79: Line 80:
6.Requesting Job status:
6.Requesting Job status:
-
  [ngrkic@ui ~]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  [ngrkic@ui ~]$ glite-wms-job-status <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
-
 
+
</nowiki>
  *************************************************************
  *************************************************************
  BOOKKEEPING INFORMATION:  
  BOOKKEEPING INFORMATION:  
   
   
-
  Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki>
  Current Status:    Done (Success)
  Current Status:    Done (Success)
  Exit code:          0
  Exit code:          0
Line 91: Line 92:
   
   
  - Nodes information for:  
  - Nodes information for:  
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA
+
     Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA</nowiki>
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
Line 102: Line 103:
  *************************************************************
  *************************************************************
      
      
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ
+
     Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ</nowiki>
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
Line 117: Line 118:
7.Requesting Job output:
7.Requesting Job output:
-
  [ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  [ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
-
 
+
</nowiki>
-
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
+
  Connecting to the service <nowiki>https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server</nowiki>
   
   
   
   
Line 127: Line 128:
   
   
  Output sandbox files for the DAG/Collection :
  Output sandbox files for the DAG/Collection :
-
  https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki>
  have been successfully retrieved and stored in the directory:
  have been successfully retrieved and stored in the directory:
  /home/ngrkic/test
  /home/ngrkic/test
Line 145: Line 146:
   
   
-
For each job there is folder with output and error file.You can see content with cat command.  
+
For each job there is folder with output and error files.You can see content with cat command.  
  [ngrkic@ui test]$cd Node_job1_jdl
  [ngrkic@ui test]$cd Node_job1_jdl
  [ngrkic@ui Node_job1_jdl]$ cat std.out  
  [ngrkic@ui Node_job1_jdl]$ cat std.out  
  rti30.etf.bg.ac.rs
  rti30.etf.bg.ac.rs

Latest revision as of 18:46, 26 May 2012

Submiting job collection How-to


This is job collection submiting example. Collection of two simple jobs. First will print hostname, and second will print date. For demonstration use attached file.

jdlcollection.tgz

jdlcollection2.tgz

1.Extract file with:

 [ngrkic@ui ~]$ tar xvzf jdlcollection.tgz

2.Enter directory jdlcollection with:

 [ngrkic@ui ~]$ cd jdlcollection

3.List directory:

[ngrkic@jdlcollection ~]$ ll
-rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:07 job1.jdl
-rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:17 job2.jdl

Content of file job1.jdl

[ngrkic@ui jdlcollection]$ cat job1.jdl 
Executable = "/bin/hostname";
StdOutput = "std.out";
StdError = "std.err";
OutputSandbox = {"std.out","std.err"};

Content of file job2.jdl

[ngrkic@ui jdlcollection]$ cat job2.jdl 
Executable = "/bin/date";
StdOutput = "std2.out";
StdError = "std2.err";
OutputSandbox = {"std2.out","std2.err"};


Then return one folder up with :

[ngrkic@jdlcollection ~]$ cd ..

4.Creating VOMS proxy:

[ngrkic@ui ~]$ voms-proxy-init -voms aegis
Cannot find file or dir: /home/ngrkic/.glite/vomses
Enter GRID pass phrase:
Your identity: /C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=Nikola Grkic
Creating temporary proxy ......................... Done
Contacting  voms.ipb.ac.rs:15001 [/C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=host/voms.ipb.ac.rs] "aegis" Done
Creating proxy ..................................................... Done
Your proxy is valid until Fri Oct  2 01:11:00 2009


5.Submiting job collection:

[ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection
Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is: 

https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g

==========================================================================


Copy the job ID.Job is running now, and it should finish in few moments...


6.Requesting Job status:

[ngrkic@ui ~]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g

*************************************************************
BOOKKEEPING INFORMATION: 

Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
Current Status:     Done (Success)
Exit code:          0
Submitted:          Thu Oct  1 14:50:50 2009 CEST
*************************************************************

- Nodes information for: 
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA
    Current Status:     Done (Success)
    Logged Reason(s):
       - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        cluster1.csk.kg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 14:50:50 2009 CEST
*************************************************************
    
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ
    Current Status:     Done (Success)
    Logged Reason(s):
        - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 14:50:50 2009 CEST
*************************************************************


7.Requesting Job output:

[ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g

Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server


================================================================================

			JOB GET OUTPUT OUTCOME

Output sandbox files for the DAG/Collection :
https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
have been successfully retrieved and stored in the directory:
/home/ngrkic/test

================================================================================ 
 

8.Go to test directory and see output with commands:

[ngrkic@ui ~]$ cd 
[ngrkic@ui ~]$ cd test
[ngrkic@ui test]$ ll
-rw-rw-r--  1 ngrkic ngrkic  359 Oct  1 15:14 ids_nodes.map
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 14:41 Node_job1_jdl
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 14:41 Node_job2_jdl


For each job there is folder with output and error files.You can see content with cat command.

[ngrkic@ui test]$cd Node_job1_jdl
[ngrkic@ui Node_job1_jdl]$ cat std.out 
rti30.etf.bg.ac.rs
Personal tools