ESE2LOM Prototype - 1st Implementation

From MediaWiki

Here we describe the steps performed during the initial gridification of ESE2LOM component:

• The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:

  java -jar ESE2LOM.jar PARAM1 PARAM2

where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.

* ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:

  #!/bin/bash
  
  TARGET_URL="http://jme.collections.natural-europe.eu/oai/"
  LFC_HOST=lfc.ipb.ac.rs
  LFC_DEST="lfn:/grid/vo.aginfra.eu/outs/ESE2LOM/jme.collections.natural-europe.eu"
  
  ESE2LOM_JAR="ESE2LOM.jar"
  OUTPUT_DIR="output"
  
  CURRENT_DIR=$(pwd)
  JAVA=$(which java)
  
  mkdir ${OUTPUT_DIR}
  ${JAVA} -jar ${ESE2LOM_JAR} ${TARGET_URL} ${CURRENT_DIR}/${OUTPUT_DIR}
  
  export ${LFC_HOST}
  
  for file in $(ls ${OUTPUT_DIR}); do
     lcg-cr -l ${LFC_DEST}/${file} ${OUTPUT_DIR}/${file}
  done

* In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:

  [
     Type = "Job";
     Executable = "ESE2LOM.sh";
     Arguments = "";
     StdOutput = "ESE2LOM.out";
     StdError = "ESE2LOM.err";
     InputSandbox = {"ESE2LOM.sh","ESE2LOM.jar"};
     OutputSandbox = {"ESE2LOM.out","ESE2LOM.err"};
     Requirements =  RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID);
  ]

All files necessary for the submission and execution on the Grid can be downloaded from here.

Further developments will include:

* Job submission to the other Grid sites in the agINFRA infrastructure.

* Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.

* Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).

* Web monitoring.

* Web service APIs.