Job collection
From MediaWiki
(Difference between revisions)
(7 intermediate revisions not shown) | |||
Line 2: | Line 2: | ||
- | This is job collection submiting example. | + | This is job collection submiting example. Collection of two simple jobs. First will print hostname, and second will print date. |
- | + | ||
For demonstration use attached file. | For demonstration use attached file. | ||
- | [http://wiki.ipb.ac.rs/images/ | + | [http://wiki.ipb.ac.rs/images/7/70/Jdlcollection.tgz jdlcollection.tgz ] |
- | + | ||
+ | [http://wiki.ipb.ac.rs/images/6/6a/Jdlcollection2.tgz jdlcollection2.tgz ] | ||
+ | |||
1.Extract file with: | 1.Extract file with: | ||
- | [ngrkic@ui ~]$ | + | [ngrkic@ui ~]$ tar xvzf jdlcollection.tgz |
2.Enter directory jdlcollection with: | 2.Enter directory jdlcollection with: | ||
Line 37: | Line 38: | ||
StdError = "std2.err"; | StdError = "std2.err"; | ||
OutputSandbox = {"std2.out","std2.err"}; | OutputSandbox = {"std2.out","std2.err"}; | ||
- | |||
- | |||
Line 57: | Line 56: | ||
Your proxy is valid until Fri Oct 2 01:11:00 2009 | Your proxy is valid until Fri Oct 2 01:11:00 2009 | ||
- | |||
- | |||
Line 65: | Line 62: | ||
[ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection | [ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection | ||
- | Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server | + | Connecting to the service <nowiki>https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server</nowiki> |
Line 73: | Line 70: | ||
Your job identifier is: | Your job identifier is: | ||
- | + | <nowiki> https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki> | |
========================================================================== | ========================================================================== | ||
Line 83: | Line 80: | ||
6.Requesting Job status: | 6.Requesting Job status: | ||
- | [ngrkic@ui ~]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g | + | [ngrkic@ui ~]$ glite-wms-job-status <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g |
- | + | </nowiki> | |
************************************************************* | ************************************************************* | ||
BOOKKEEPING INFORMATION: | BOOKKEEPING INFORMATION: | ||
- | Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g | + | Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki> |
Current Status: Done (Success) | Current Status: Done (Success) | ||
Exit code: 0 | Exit code: 0 | ||
Line 95: | Line 92: | ||
- Nodes information for: | - Nodes information for: | ||
- | Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA | + | Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA</nowiki> |
Current Status: Done (Success) | Current Status: Done (Success) | ||
Logged Reason(s): | Logged Reason(s): | ||
Line 106: | Line 103: | ||
************************************************************* | ************************************************************* | ||
- | Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ | + | Status info for the Job : <nowiki>https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ</nowiki> |
Current Status: Done (Success) | Current Status: Done (Success) | ||
Logged Reason(s): | Logged Reason(s): | ||
Line 121: | Line 118: | ||
7.Requesting Job output: | 7.Requesting Job output: | ||
- | [ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g | + | [ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g |
- | + | </nowiki> | |
- | Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server | + | Connecting to the service <nowiki>https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server</nowiki> |
Line 131: | Line 128: | ||
Output sandbox files for the DAG/Collection : | Output sandbox files for the DAG/Collection : | ||
- | https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g | + | <nowiki>https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g</nowiki> |
have been successfully retrieved and stored in the directory: | have been successfully retrieved and stored in the directory: | ||
/home/ngrkic/test | /home/ngrkic/test | ||
Line 149: | Line 146: | ||
- | For each job there is | + | For each job there is folder with output and error files.You can see content with cat command. |
[ngrkic@ui test]$cd Node_job1_jdl | [ngrkic@ui test]$cd Node_job1_jdl | ||
[ngrkic@ui Node_job1_jdl]$ cat std.out | [ngrkic@ui Node_job1_jdl]$ cat std.out | ||
rti30.etf.bg.ac.rs | rti30.etf.bg.ac.rs |
Latest revision as of 18:46, 26 May 2012
Submiting job collection How-to
This is job collection submiting example. Collection of two simple jobs. First will print hostname, and second will print date.
For demonstration use attached file.
1.Extract file with:
[ngrkic@ui ~]$ tar xvzf jdlcollection.tgz
2.Enter directory jdlcollection with:
[ngrkic@ui ~]$ cd jdlcollection
3.List directory:
[ngrkic@jdlcollection ~]$ ll -rw-rw-r-- 1 ngrkic ngrkic 115 Oct 1 14:07 job1.jdl -rw-rw-r-- 1 ngrkic ngrkic 115 Oct 1 14:17 job2.jdl
Content of file job1.jdl
[ngrkic@ui jdlcollection]$ cat job1.jdl Executable = "/bin/hostname"; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out","std.err"};
Content of file job2.jdl
[ngrkic@ui jdlcollection]$ cat job2.jdl Executable = "/bin/date"; StdOutput = "std2.out"; StdError = "std2.err"; OutputSandbox = {"std2.out","std2.err"};
Then return one folder up with :
[ngrkic@jdlcollection ~]$ cd ..
4.Creating VOMS proxy:
[ngrkic@ui ~]$ voms-proxy-init -voms aegis
Cannot find file or dir: /home/ngrkic/.glite/vomses Enter GRID pass phrase: Your identity: /C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=Nikola Grkic Creating temporary proxy ......................... Done Contacting voms.ipb.ac.rs:15001 [/C=RS/O=AEGIS/OU=Institute of Physics Belgrade/CN=host/voms.ipb.ac.rs] "aegis" Done Creating proxy ..................................................... Done Your proxy is valid until Fri Oct 2 01:11:00 2009
5.Submiting job collection:
[ngrkic@ui ~]$ glite-wms-job-submit -a --collection jdlcollection
Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is:
https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g
==========================================================================
Copy the job ID.Job is running now, and it should finish in few moments...
6.Requesting Job status:
[ngrkic@ui ~]$ glite-wms-job-status https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g Current Status: Done (Success) Exit code: 0 Submitted: Thu Oct 1 14:50:50 2009 CEST ************************************************************* - Nodes information for: Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/7Vhpq3YM3OJIPGfB_yEQzA Current Status: Done (Success) Logged Reason(s): - - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: cluster1.csk.kg.ac.rs:2119/jobmanager-pbs-aegis Submitted: Thu Oct 1 14:50:50 2009 CEST ************************************************************* Status info for the Job : https://wms-aegis.ipb.ac.rs:9000/heZyUPQvY4_kFD3V-vzIbQ Current Status: Done (Success) Logged Reason(s): - - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid-ce.etf.bg.ac.rs:2119/jobmanager-pbs-aegis Submitted: Thu Oct 1 14:50:50 2009 CEST *************************************************************
7.Requesting Job output:
[ngrkic@ui ~]$ glite-wms-job-output --dir /home/ngrkic/test/ https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g Connecting to the service https://wms-aegis.ipb.ac.rs:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the DAG/Collection : https://wms-aegis.ipb.ac.rs:9000/e00XBJLV2BRYWfDxOfHL6g have been successfully retrieved and stored in the directory: /home/ngrkic/test ================================================================================
8.Go to test directory and see output with commands:
[ngrkic@ui ~]$ cd [ngrkic@ui ~]$ cd test [ngrkic@ui test]$ ll
-rw-rw-r-- 1 ngrkic ngrkic 359 Oct 1 15:14 ids_nodes.map drwxr-xr-x 2 ngrkic ngrkic 4096 Oct 1 14:41 Node_job1_jdl drwxr-xr-x 2 ngrkic ngrkic 4096 Oct 1 14:41 Node_job2_jdl
For each job there is folder with output and error files.You can see content with cat command.
[ngrkic@ui test]$cd Node_job1_jdl [ngrkic@ui Node_job1_jdl]$ cat std.out rti30.etf.bg.ac.rs