ESE2LOM Prototype - 1st Implementation
From MediaWiki
(Difference between revisions)
Line 43: | Line 43: | ||
] | ] | ||
- | All files necessary for the submission and execution on the Grid can be downloaded from here. | + | All files necessary for the submission and execution on the Grid can be downloaded from [http://wiki.ipb.ac.rs/images/c/c4/ESE2LOM.tgz here]. |
Further developments will include: | Further developments will include: |
Latest revision as of 09:21, 30 May 2012
Here we describe the steps performed during the initial gridification of ESE2LOM component:
- The source code is compiled to the executable JAR file ESE2LOM.jar, and can be executed with the following command:
java -jar ESE2LOM.jar PARAM1 PARAM2
where PARAM1 is OAI-PMH target for harvesting, and PARAM2 the path where the harvested and transformed records would be saved.
- ESE2LOM execution and upload of harvested/transformed records to the Grid storage is wrapped by the following BASH script:
#!/bin/bash TARGET_URL="http://jme.collections.natural-europe.eu/oai/" LFC_HOST=lfc.ipb.ac.rs LFC_DEST="lfn:/grid/vo.aginfra.eu/outs/ESE2LOM/jme.collections.natural-europe.eu" ESE2LOM_JAR="ESE2LOM.jar" OUTPUT_DIR="output" CURRENT_DIR=$(pwd) JAVA=$(which java) mkdir ${OUTPUT_DIR} ${JAVA} -jar ${ESE2LOM_JAR} ${TARGET_URL} ${CURRENT_DIR}/${OUTPUT_DIR} export ${LFC_HOST} for file in $(ls ${OUTPUT_DIR}); do lcg-cr -l ${LFC_DEST}/${file} ${OUTPUT_DIR}/${file} done
- In order to submit ESE2LOM as a Grid job, the following JDL file is prepared:
[ Type = "Job"; Executable = "ESE2LOM.sh"; Arguments = ""; StdOutput = "ESE2LOM.out"; StdError = "ESE2LOM.err"; InputSandbox = {"ESE2LOM.sh","ESE2LOM.jar"}; OutputSandbox = {"ESE2LOM.out","ESE2LOM.err"}; Requirements = RegExp("ce64.ipb.ac.rs*",other.GlueCEUniqueID); ]
All files necessary for the submission and execution on the Grid can be downloaded from here.
Further developments will include:
- Job submission to the other Grid sites in the agINFRA infrastructure.
- Module for installation of the unique versions of Java at all sites in the infrastructure in the VO space.
- Module for installation of ESE2LOM component at remote sites in the VO space (currently the executable has to be transferred with the job).
- Web monitoring.
- Web service APIs.