JobSubmiting

From MediaWiki

(Difference between revisions)
Jump to: navigation, search
Line 1: Line 1:
-
'''Submiting job collection How to'''
+
'''Submiting DAG job how to'''
-
This is job collection  submiting example.
+
This is DAG job submiting example.
-
It does not requre some high skills and knowing JDL language.
+
For demonstration use attached file.
For demonstration use attached file.
1.Extract file with:
1.Extract file with:
-
  [ngrkic@ui ~]$ tar -xzf jdlcollection.tar.gz
+
[ngrkic@ui ~]$ tar -xzf DAG.tar.gz
-
2.Enter directory jdlcollection with:
+
2.Enter directory single with:
-
  [ngrkic@ui ~]$ cd jdlcollection
+
[ngrkic@ui ~]$ cd DAG
3.List directory:
3.List directory:
-
  [ngrkic@jdlcollection ~]$ ll
+
[ngrkic@ui DAG]$ ll
-
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:07 job1.jdl
+
-
  -rw-rw-r--  1 ngrkic ngrkic 115 Oct  1 14:17 job2.jdl
+
-rw-r--r--  1 ngrkic ngrkic 1469 Oct  1 15:54 jobDAG.jdl
-
 
+
-rwxr-xr-x 1 ngrkic ngrkic   92 Oct  1 15:54 job.sh
-
then return one folder up with :
+
-
 
+
-
[ngrkic@jdlcollection ~]$ cd ..
+
4.Creating VOMS proxy:
4.Creating VOMS proxy:
-
  [ngrkic@ui ~]$ voms-proxy-init -voms aegis
+
  [ngrkic@ui single]$ voms-proxy-init -voms aegis
  Cannot find file or dir: /home/ngrkic/.glite/vomses
  Cannot find file or dir: /home/ngrkic/.glite/vomses
Line 31: Line 27:
  Your proxy is valid until Fri Oct  2 01:11:00 2009
  Your proxy is valid until Fri Oct  2 01:11:00 2009
-
Now we are ready to submit job collection.
 
-
5.Submiting job collection:
+
Now we are ready to submit job.
-
  [ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection
+
 
 +
5.Submiting DAG job:
 +
 
 +
  [ngrkic@ui DAG]$ glite-wms-job-submit -a jobDAG.jdl
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
Line 43: Line 41:
   
   
  The job has been successfully submitted to the WMProxy
  The job has been successfully submitted to the WMProxy
-
  Your job identifier is:  
+
  Your job identifier is:
   
   
-
  https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
   
   
  ==========================================================================
  ==========================================================================
Line 53: Line 51:
-
6.Requesting Job status:
+
6.Requesting Job status:  
-
 
+
-
[ngrkic@ui ~]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
 +
[ngrkic@ui DAG]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
 +
 +
  *************************************************************
  *************************************************************
-
  BOOKKEEPING INFORMATION:  
+
  BOOKKEEPING INFORMATION:
   
   
-
  Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
  Current Status:    Done (Success)
  Current Status:    Done (Success)
  Exit code:          0
  Exit code:          0
-
  Submitted:          Thu Oct  1 14:50:50 2009 CEST
+
Status Reason:      Job terminated successfully
 +
Destination:        dagman
 +
  Submitted:          Thu Oct  1 16:44:32 2009 CEST
  *************************************************************
  *************************************************************
   
   
  - Nodes information for:  
  - Nodes information for:  
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA
+
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/4xL7Bu5cI6eaixg9y0oezg
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
-
        -  
+
        -  
         - Job terminated successfully
         - Job terminated successfully
     Exit code:          0
     Exit code:          0
     Status Reason:      Job terminated successfully
     Status Reason:      Job terminated successfully
-
     Destination:        cluster1.csk.kg.ac.rs:2119/jobmanager-pbs-aegis
+
     Destination:        grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis
-
     Submitted:          Thu Oct  1 14:50:50 2009 CEST
+
     Submitted:          Thu Oct  1 16:44:32 2009 CEST
  *************************************************************
  *************************************************************
-
   
+
   
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ
+
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/LCE9ULBND3jYd2JUJPJ9wQ
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
Line 85: Line 86:
     Exit code:          0
     Exit code:          0
     Status Reason:      Job terminated successfully
     Status Reason:      Job terminated successfully
-
     Destination:        grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis
+
     Destination:        grid01.rcub.bg.ac.rs:2119/jobmanager-pbs-aegis
-
     Submitted:          Thu Oct  1 14:50:50 2009 CEST
+
     Submitted:          Thu Oct  1 16:44:32 2009 CEST
 +
*************************************************************
 +
   
 +
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/quIESEluISRrAXdcC7ELKQ
 +
    Current Status:    Done (Success)
 +
    Exit code:          0
 +
    Status Reason:      Job terminated successfully
 +
    Destination:        grid01.elfak.ni.ac.rs:2119/jobmanager-pbs-aegis
 +
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
 +
*************************************************************
 +
   
 +
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/yNu3yqhSwf4sZ25bWuzKDg
 +
    Current Status:    Done (Success)
 +
    Exit code:          0
 +
    Status Reason:      Job terminated successfully
 +
    Destination:        grid01.elfak.ni.ac.rs:2119/jobmanager-pbs-aegis
 +
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
  *************************************************************
  *************************************************************
-
 
-
 
7.Requesting Job output:
7.Requesting Job output:
-
+
 
-
  [ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  [ngrkic@ui DAG]$ glite-wms-job-output --dir /home/ngrkic/test https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
-
 
   
   
  ================================================================================
  ================================================================================
Line 103: Line 117:
   
   
  Output sandbox files for the DAG/Collection :
  Output sandbox files for the DAG/Collection :
-
  https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
+
  https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
  have been successfully retrieved and stored in the directory:
  have been successfully retrieved and stored in the directory:
  /home/ngrkic/test
  /home/ngrkic/test
   
   
-
  ================================================================================  
+
  ================================================================================
-
 
+
 
-
+
 
-
+
8.Go to test directory and see output with commands:
8.Go to test directory and see output with commands:
  [ngrkic@ui ~]$ cd  
  [ngrkic@ui ~]$ cd  
  [ngrkic@ui ~]$ cd test
  [ngrkic@ui ~]$ cd test
-
  [ngrkic@ui test]$ ll
+
  [ngrkic@ui ~]$ ll
-
 
+
-
-rw-rw-r--  1 ngrkic ngrkic  359 Oct  1 15:14 ids_nodes.map
+
-
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 14:41 Node_job1_jdl
+
-
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 14:41 Node_job2_jdl
+
-
 
+
   
   
-
For each job there is folder with output and error file.You can see content with cat command.
+
  drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeA
-
 
+
  drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeB
-
  [ngrkic@ui test]$cd Node_job1_jdl
+
  drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeC
-
  [ngrkic@ui Node_job1_jdl]$ cat std.out
+
  drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeD
-
  rti30.etf.bg.ac.rs
+

Revision as of 10:42, 2 October 2009

Submiting DAG job how to


This is DAG job submiting example. For demonstration use attached file.

1.Extract file with: 

[ngrkic@ui ~]$ tar -xzf DAG.tar.gz

2.Enter directory single with: 

[ngrkic@ui ~]$ cd DAG

3.List directory: 

[ngrkic@ui DAG]$ ll

-rw-r--r--  1 ngrkic ngrkic 1469 Oct  1 15:54 jobDAG.jdl
-rwxr-xr-x  1 ngrkic ngrkic   92 Oct  1 15:54 job.sh

4.Creating VOMS proxy: 

[ngrkic@ui single]$ voms-proxy-init -voms aegis

Cannot find file or dir: /home/ngrkic/.glite/vomses
Enter GRID pass phrase:
Your identity: /C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=Nikola Grkic
Creating temporary proxy ......................... Done
Contacting  voms.ipb.ac.rs:15001 [/C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=host/voms.ipb.ac.rs] "aegis" Done
Creating proxy ..................................................... Done
Your proxy is valid until Fri Oct  2 01:11:00 2009


Now we are ready to submit job.


5.Submiting DAG job: 

[ngrkic@ui DAG]$ glite-wms-job-submit -a jobDAG.jdl 

Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ

==========================================================================


Copy the job ID.Job is running now, and it should finish in few moments...


6.Requesting Job status: 

[ngrkic@ui DAG]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ


*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        dagman
Submitted:          Thu Oct  1 16:44:32 2009 CEST
*************************************************************

- Nodes information for: 
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/4xL7Bu5cI6eaixg9y0oezg
    Current Status:     Done (Success)
    Logged Reason(s):
        - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
*************************************************************
   
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/LCE9ULBND3jYd2JUJPJ9wQ
    Current Status:     Done (Success)
    Logged Reason(s):
        - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid01.rcub.bg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
*************************************************************
    
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/quIESEluISRrAXdcC7ELKQ
    Current Status:     Done (Success)
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid01.elfak.ni.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
*************************************************************
    
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/yNu3yqhSwf4sZ25bWuzKDg
    Current Status:     Done (Success)
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid01.elfak.ni.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Thu Oct  1 16:44:32 2009 CEST
*************************************************************

7.Requesting Job output: 

[ngrkic@ui DAG]$ glite-wms-job-output --dir /home/ngrkic/test https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ

Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server

================================================================================

			JOB GET OUTPUT OUTCOME

Output sandbox files for the DAG/Collection :
https://wms-aegis.ipb.ac.rs:9000/O3Bxi2I9DoE2Lltmcoe0eQ
have been successfully retrieved and stored in the directory:
/home/ngrkic/test

================================================================================


8.Go to test directory and see output with commands: 

[ngrkic@ui ~]$ cd 
[ngrkic@ui ~]$ cd test
[ngrkic@ui ~]$ ll

drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeA
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeB
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeC
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  1 17:13 nodeD
Retrieved from "http://wiki.ipb.ac.rs/index.php/JobSubmiting"
Personal tools