Parametric Job

From MediaWiki

(Difference between revisions)
Jump to: navigation, search
 
(14 intermediate revisions not shown)
Line 1: Line 1:
-
'''Submiting parametric job how to'''
 
Line 5: Line 4:
For demonstration use attached file.
For demonstration use attached file.
-
[http://wiki.ipb.ac.rs/index.php/Image:Parametric.zip parametric]
+
Script message_PARAM_.sh will just print "message number _PARAM_ " where _PARAM_ is value of parameter. Second file is parametric.jdl.In this case number of parameter is 6, but the ParameterStep is 2, which means that every second parameter will be used in this job.
 +
 
 +
[http://wiki.ipb.ac.rs/images/2/25/Parametric.tgz Parametric.tgz]
 +
 
 +
[http://wiki.ipb.ac.rs/images/7/7d/Parametric_example_2.tgz Parametric_example_2.tgz]
 +
 
 +
[http://wiki.ipb.ac.rs/images/1/10/Parametric_example_3.tgz Parametric_example_3.tgz]
1.Extract file with:
1.Extract file with:
-
  [ngrkic@ui ~]$ unzip parametric.zip
+
  [ngrkic@ui ~]$ tar zxvf parametric.zip
2.Enter directory single with:
2.Enter directory single with:
  [ngrkic@ui ~]$ cd parametric
  [ngrkic@ui ~]$ cd parametric
Line 18: Line 23:
  -rw-rw-r--  1 ngrkic ngrkic  22 Oct  2 11:29 message4.sh  
  -rw-rw-r--  1 ngrkic ngrkic  22 Oct  2 11:29 message4.sh  
  -rw-rw-r--  1 ngrkic ngrkic 383 Oct  2 11:29 parametric.jdl
  -rw-rw-r--  1 ngrkic ngrkic 383 Oct  2 11:29 parametric.jdl
 +
 +
This is content of file message0.sh
 +
 +
echo message number 0
 +
 +
This is content of file parametric.jdl
 +
 +
[  JobType = "Parametric";
 +
    Executable = "/bin/sh";
 +
    Arguments = "message_PARAM_.sh";
 +
    InputSandbox = "message_PARAM_.sh";
 +
    Parameters= 6;
 +
    ParameterStep =2;
 +
    ParameterStart = 0;
 +
    StdOutput = "myoutput_PARAM_.txt";
 +
    StdError = "myerror_PARAM_.txt";
 +
    OutputSandbox  = {"myoutput_PARAM_.txt", "myerror_PARAM_.txt"};
 +
    Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
 +
]
 +
 +
Script message_PARAM_.sh will just print "message number _PARAM_ " where _PARAM_ is value of parameter.Second file is parametric.jdl.In this case
 +
number of parameter is 6, but the ParameterStep is 2, which means that every second parameter will be used in this job.
4.Creating VOMS proxy:
4.Creating VOMS proxy:
Line 30: Line 57:
  Creating proxy ..................................................... Done
  Creating proxy ..................................................... Done
  Your proxy is valid until Fri Oct  2 01:11:00 2009
  Your proxy is valid until Fri Oct  2 01:11:00 2009
-
 
-
 
-
Now we are ready to submit job.
 
Line 39: Line 63:
  [ngrkic@ui parametric]$ glite-wms-job-submit -a parametric.jdl  
  [ngrkic@ui parametric]$ glite-wms-job-submit -a parametric.jdl  
-
  Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server
+
  Connecting to the service <nowiki>https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server</nowiki>
   
   
   
   
Line 47: Line 71:
  Your job identifier is:
  Your job identifier is:
   
   
-
  https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
+
  <nowiki>https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw</nowiki>
   
   
  ==========================================================================
  ==========================================================================
Line 57: Line 81:
-
  [ngrkic@ui parametric]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
+
  [ngrkic@ui parametric]$ glite-wms-job-status <nowiki>https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw</nowiki>
   
   
   
   
Line 63: Line 87:
  BOOKKEEPING INFORMATION:
  BOOKKEEPING INFORMATION:
   
   
-
  Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
+
  Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw</nowiki>
  Current Status:    Done (Success)
  Current Status:    Done (Success)
  Exit code:          0
  Exit code:          0
Line 70: Line 94:
   
   
  - Nodes information for:  
  - Nodes information for:  
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/8ZKpCDU01saOfQj_S4JFqg
+
     Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/8ZKpCDU01saOfQj_S4JFqg</nowiki>
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
Line 81: Line 105:
  *************************************************************
  *************************************************************
      
      
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/IvbGjKoP1vzF1mCG8jgyZg
+
     Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/IvbGjKoP1vzF1mCG8jgyZg</nowiki>
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Exit code:          0
     Exit code:          0
Line 89: Line 113:
  *************************************************************
  *************************************************************
      
      
-
     Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/msglck3KPVDh3CytDDu_ow
+
     Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/msglck3KPVDh3CytDDu_ow</nowiki>
     Current Status:    Done (Success)
     Current Status:    Done (Success)
     Logged Reason(s):
     Logged Reason(s):
Line 104: Line 128:
7.Requesting Job output:
7.Requesting Job output:
-
  [ngrkic@ui parametric]$ glite-wms-job-output --dir /home/ngrkic/test https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
+
  [ngrkic@ui parametric]$ glite-wms-job-output --dir /home/ngrkic/test <nowiki>https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw</nowiki>
  ================================================================================
  ================================================================================
Line 111: Line 135:
   
   
  Output sandbox files for the DAG/Collection :
  Output sandbox files for the DAG/Collection :
-
  https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
+
  <nowiki>https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw</nowiki>
  have been successfully retrieved and stored in the directory:
  have been successfully retrieved and stored in the directory:
  /home/ngrkic/test
  /home/ngrkic/test

Latest revision as of 09:00, 8 December 2011


This is simple parametric job submiting example. For demonstration use attached file.

Script message_PARAM_.sh will just print "message number _PARAM_ " where _PARAM_ is value of parameter. Second file is parametric.jdl.In this case number of parameter is 6, but the ParameterStep is 2, which means that every second parameter will be used in this job.

Parametric.tgz

Parametric_example_2.tgz

Parametric_example_3.tgz

1.Extract file with:

[ngrkic@ui ~]$ tar zxvf parametric.zip

2.Enter directory single with:

[ngrkic@ui ~]$ cd parametric

3.List directory:

[ngrkic@ui parametric]$ ll
-rw-rw-r--  1 ngrkic ngrkic  22 Oct  2 11:29 message0.sh
-rw-rw-r--  1 ngrkic ngrkic  22 Oct  2 11:29 message2.sh
-rw-rw-r--  1 ngrkic ngrkic  22 Oct  2 11:29 message4.sh 
-rw-rw-r--  1 ngrkic ngrkic 383 Oct  2 11:29 parametric.jdl

This is content of file message0.sh

echo message number 0

This is content of file parametric.jdl

[   JobType = "Parametric";
    Executable = "/bin/sh";
    Arguments = "message_PARAM_.sh";
    InputSandbox = "message_PARAM_.sh";
    Parameters= 6;
    ParameterStep =2;
    ParameterStart = 0;
    StdOutput = "myoutput_PARAM_.txt";
    StdError = "myerror_PARAM_.txt";
    OutputSandbox  = {"myoutput_PARAM_.txt", "myerror_PARAM_.txt"};
    Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
] 

Script message_PARAM_.sh will just print "message number _PARAM_ " where _PARAM_ is value of parameter.Second file is parametric.jdl.In this case number of parameter is 6, but the ParameterStep is 2, which means that every second parameter will be used in this job.

4.Creating VOMS proxy:

[ngrkic@ui parametric]$ voms-proxy-init -voms aegis
Cannot find file or dir: /home/ngrkic/.glite/vomses
Enter GRID pass phrase:
Your identity: /C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=Nikola Grkic
Creating temporary proxy ......................... Done
Contacting  voms.ipb.ac.rs:15001 [/C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=host/voms.ipb.ac.rs] "aegis" Done
Creating proxy ..................................................... Done
Your proxy is valid until Fri Oct  2 01:11:00 2009


5.Submiting single job:

[ngrkic@ui parametric]$ glite-wms-job-submit -a parametric.jdl 
Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw

==========================================================================

Copy the job ID.Job is running now, and it should finish in few moments...


6.Requesting Job status:


[ngrkic@ui parametric]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw


*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
Current Status:     Done (Success)
Exit code:          0
Submitted:          Fri Oct  2 11:40:25 2009 CEST
*************************************************************

- Nodes information for: 
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/8ZKpCDU01saOfQj_S4JFqg
    Current Status:     Done (Success)
    Logged Reason(s):
        - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Fri Oct  2 11:40:25 2009 CEST
*************************************************************
    
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/IvbGjKoP1vzF1mCG8jgyZg
    Current Status:     Done (Success)
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid01.elfak.ni.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Fri Oct  2 11:40:25 2009 CEST
*************************************************************
    
    Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/msglck3KPVDh3CytDDu_ow
    Current Status:     Done (Success)
    Logged Reason(s):
        - 
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        grid01.rcub.bg.ac.rs:2119/jobmanager-pbs-aegis
    Submitted:          Fri Oct  2 11:40:25 2009 CEST
*************************************************************


7.Requesting Job output:

[ngrkic@ui parametric]$ glite-wms-job-output --dir /home/ngrkic/test https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
================================================================================

			JOB GET OUTPUT OUTCOME

Output sandbox files for the DAG/Collection :
https://wms-aegis.ipb.ac.rs:9000/Vh8g7EU-gCH9LRDnhbx5iw
have been successfully retrieved and stored in the directory:
/home/ngrkic/test

================================================================================

8.Go to test directory and see output with commands:

[ngrkic@ui parametric]$ cd 
[ngrkic@ui single]$ cd test 
[ngrkic@ui test]$ ll
total 16
-rw-rw-r--  1 ngrkic ngrkic  463 Oct  2 12:21 ids_nodes.map
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  2 11:26 Node_0
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  2 11:26 Node_2
drwxr-xr-x  2 ngrkic ngrkic 4096 Oct  2 11:26 Node_4

For each node there is folder with myerror.txt and myoutput.txt files.

[ngrkic@ui test]$ cd Node_2
[ngrkic@ui Node_2]$ ll
-rw-rw-r--  1 ngrkic ngrkic  0 Oct  2 12:21 myerror2.txt
-rw-rw-r--  1 ngrkic ngrkic 17 Oct  2 12:21 myoutput2.txt
[ngrkic@ui Node_2]$cat myoutput2.txt
message number 2
Personal tools