ESE2LOM Prototype - 1st Implementation

From MediaWiki

(Difference between revisions)
Jump to: navigation, search
(Created page with 'Here we describe the steps performed during the initial gridification of ESE2LOM component: * The source code is compiled to the executable JAR file ESE2LOM.jar, and can be exe…')
 
(2 intermediate revisions not shown)
Line 1: Line 1:
Here we describe the steps performed during the initial gridification of ESE2LOM component:
Here we describe the steps performed during the initial gridification of ESE2LOM component:
-
* The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:
+
* The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:
   java -jar ESE2LOM.jar PARAM1 PARAM2
   java -jar ESE2LOM.jar PARAM1 PARAM2
Line 7: Line 7:
where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.
where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.
-
* ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:
+
* ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:
   #!/bin/bash
   #!/bin/bash
Line 30: Line 30:
   done
   done
-
* In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:
+
* In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:
   [
   [
Line 43: Line 43:
   ]
   ]
-
All files necessary for the submission and execution on the Grid can be downloaded from here.
+
All files necessary for the submission and execution on the Grid can be downloaded from [http://wiki.ipb.ac.rs/images/c/c4/ESE2LOM.tgz here].
Further developments will include:
Further developments will include:
-
* Job submission to the other Grid sites in the agINFRA infrastructure.
+
* Job submission to the other Grid sites in the agINFRA infrastructure.
-
* Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.
+
* Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.
-
* Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).
+
* Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).
-
* Web monitoring.
+
* Web monitoring.
-
* Web service APIs.
+
* Web service APIs.

Latest revision as of 09:21, 30 May 2012

Here we describe the steps performed during the initial gridification of ESE2LOM component:

  • The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:
  java -jar ESE2LOM.jar PARAM1 PARAM2

where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.

  • ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:
  #!/bin/bash
  
  TARGET_URL="http://jme.collections.natural-europe.eu/oai/"
  LFC_HOST=lfc.ipb.ac.rs
  LFC_DEST="lfn:/grid/vo.aginfra.eu/outs/ESE2LOM/jme.collections.natural-europe.eu"
  
  ESE2LOM_JAR="ESE2LOM.jar"
  OUTPUT_DIR="output"
  
  CURRENT_DIR=$(pwd)
  JAVA=$(which java)
  
  mkdir ${OUTPUT_DIR}
  ${JAVA} -jar ${ESE2LOM_JAR} ${TARGET_URL} ${CURRENT_DIR}/${OUTPUT_DIR}
  
  export ${LFC_HOST}
  
  for file in $(ls ${OUTPUT_DIR}); do
     lcg-cr -l ${LFC_DEST}/${file} ${OUTPUT_DIR}/${file}
  done
  • In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:
  [
     Type = "Job";
     Executable = "ESE2LOM.sh";
     Arguments = "";
     StdOutput = "ESE2LOM.out";
     StdError = "ESE2LOM.err";
     InputSandbox = {"ESE2LOM.sh","ESE2LOM.jar"};
     OutputSandbox = {"ESE2LOM.out","ESE2LOM.err"};
     Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
  ]

All files necessary for the submission and execution on the Grid can be downloaded from here.

Further developments will include:

  • Job submission to the other Grid sites in the agINFRA infrastructure.
  • Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.
  • Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).
  • Web monitoring.
  • Web service APIs.
Personal tools