ESE2LOM Prototype - 1st Implementation

From MediaWiki

Jump to: navigation, search

Here we describe the steps performed during the initial gridification of ESE2LOM component:

  • The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:
  java -jar ESE2LOM.jar PARAM1 PARAM2

where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.

  • ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:
  JAVA=$(which java)
  mkdir ${OUTPUT_DIR}
  export ${LFC_HOST}
  for file in $(ls ${OUTPUT_DIR}); do
     lcg-cr -l ${LFC_DEST}/${file} ${OUTPUT_DIR}/${file}
  • In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:
     Type = "Job";
     Executable = "";
     Arguments = "";
     StdOutput = "ESE2LOM.out";
     StdError = "ESE2LOM.err";
     InputSandbox = {"","ESE2LOM.jar"};
     OutputSandbox = {"ESE2LOM.out","ESE2LOM.err"};
     Requirements =  RegExp("*",other.GlueCEUniqueID);

All files necessary for the submission and execution on the Grid can be downloaded from here.

Further developments will include:

  • Job submission to the other Grid sites in the agINFRA infrastructure.
  • Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.
  • Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).
  • Web monitoring.
  • Web service APIs.
Personal tools