Personal tools
Straightforward port
A bare bones porting of the application, still with the features required for a basic usage of the grid
Basic port of the application
This is the bare minimum required to run CSCDriver on the grid: a simple script paired with a jdl (which should be changed to run on different data sets) are used to submit a plain Normal Job that brings the data back with the output sandbox.
Major drawbacks of such approach are that the job handling (re-submitting of failed/aborted jobs and so on) are usually a major annoyance.
This set can be of practical use at least in one occasion: to test the validity of the package with the CSCDisplay code without concerning oneself with the complexity of the further ports.
Here is the jdl used for the submission:
# CSCDriver_basic.jdl Executable = "CSCDriver_basic.py"; Arguments = "srm://t2-dpm-01.na.infn.it/dpm/na.infn.it/home/(...)/Dataset.aan.root"; StdOutput = "output.txt"; StdError = "output.txt"; InputSandbox = {"CSCDriver_basic.py","atlas_voms_cert"}; OutputSandbox = {"code/results.txt","code/summary/summary_gridjob.root","output.txt"};
As can be seen, the jdl requires the jdl for the VO holding the data to be present in the submission directory, with the name atlas_voms_cert, since we are using data from the atlas VO.
An euindia certificate is also required, but in it's standard location, for the submission of the jobs. This is the list of commands to execute to produce the atlas certificate and then create the euindia one:
$ voms-proxy-init -valid 100:0 -voms atlas -out atlas_voms_cert $ voms-proxy-init -valid 100:0 -voms euindia
during the execution of those lines, the password protecting the certificate will be required twice.
Validity of the atlas cerfificate (as well as it's expiration) can be checked with the voms-proxy-info -file atlas_voms_cert -all command.
Here is the script CSCDriver_basic.py, which will be submitted with the application.
#!/usr/bin/env python ### ### Created by Riccardo Di Meo (International Centre for Theoretical ### Physic - EUIndiaGRID Project) and Stefano Cozzini (Democritos - SISSA) ### ### This work is licensed under the Creative Commons "2.5 Italy ### Attribution-Noncommercial" License. ### ### To view a copy of this license, visit ### "http://creativecommons.org/licenses/by-nc/2.5/it/legalcode" ### import os, sys, time # Error codes DONE = 0 ARG_ERR = 1 VOMS_ERROR = 2 CODE_ERROR1 = 3 CODE_ERROR1 = 4 LCG_GT_ERR1 = 5 LCG_GT_ERR2 = 6 GSIFTP_ERR = 7 CSCD_ERROR = 8 UNEXP_EXCP = 9 # Second parameter for CSCDriver BLOCKS = 100000 # Location where the tar package with the CSCDriver and library is # stored on the euindia grid CODE_LOCATION = "lfn:/grid/euindia/user/CSCDriver_code.tar.bz2" # BDII for atlas, should be correctly set up before the lcg-gt step LCG_GFAL_INFOSYS = "lcg-bdii.cern.ch:2170" # Name of the atlas certificate: must match the jdl input sandbox DATA_VOMS = "atlas_voms_cert" def log(msg, error = None): print msg if error != None: sys.exit(error) def run_CSCDisplay(): # First and only argument: the SURL to retrieve and work with try: surl = sys.argv[1] except IndexError, err: log("surl not passed to the script!", ARG_ERR) # Change the permissions to the certificate try: os.chmod(DATA_VOMS, 0600) except OSError,err: log("Cannot chmod the data certificate: %s" % err, VOMS_ERROR) # Retrieve the program start = time.time() res = os.system("lcg-cp %s file:`pwd`/code.tar.bz2" % CODE_LOCATION) if res != 0: log("lcg-cp of the program returned %d" % res, CODE_ERROR1) # Unpack it res = os.system("tar xjf code.tar.bz2") if res != 0: log("untar of the program returned %d" % res, CODE_ERROR2) stop = time.time() log("Program set up in %d\"" % (stop - start)) # That was the last operation with the euindia certificates: # change the environment os.putenv("LCG_GFAL_INFOSYS", LCG_GFAL_INFOSYS) os.putenv("X509_USER_CERT", os.path.join(os.getcwd(), DATA_VOMS)) os.putenv("X509_USER_KEY", os.path.join(os.getcwd(), DATA_VOMS)) # Enter the data directory os.chdir("code/Data") # Get the turl for the surl start = time.time() res = os.popen("lcg-gt %s gsiftp" % surl, "r") tmp = res.readlines() if res.close() != None: log("Error executing lcg-gt (%d)!" % res, LCG_GT_ERR1) if tmp[0].find("gsiftp") != 0: log("Unrecognized string returned by lcg-gt (%s)!" % tmp[0][:-1], LCG_GT_ERR2) turl = tmp[0][:-1] stop = time.time() log("The turl was obtained in %d\"" % (stop - start)) # Download the turl start = time.time() res = os.system("globus-url-copy %s file:`pwd`/data.root" % turl) if res != 0: log("Failed to download the data from the SE (%d)" % res, GSIFTP_ERR) os.chdir("..") stop = time.time() log("Data downloaded in %d\"" % (stop - start)) # Create the .def file in sample f = file(os.path.join("samples", "gridjob.def"), "w") f.write("TITLE: gridjob\nData/data.root\n") f.close() # Export the LD_LIBRARY_PATH to include the root libs as well ldvar = os.getenv("LD_LIBRARY_PATH", None) rootpath = os.path.join(os.getcwd(), "lib") if ldvar == None: os.putenv("LD_LIBRARY_PATH", rootpath) else: os.putenv("LD_LIBRARY_PATH", ldvar + ":" + rootpath) # Run the code start = time.time() res = os.system("./CSCDriver gridjob %d" % BLOCKS) if res != 0: log("CSCDriver didn't returned 0 (%d)!" % res, CSCD_ERROR) stop = time.time() log("Program executed in %d\"" % (stop - start)) # Run the code only if called as a script if __name__ == "__main__": try: run_CSCDisplay() except SystemExit, code: sys.exit(code) except Exception, err: log("Uncaught exception (%s)!", UNEXP_EXCP)
As you can see, since it is assumed to be constant for each set of data, the second parameter to the CSCDriver program is encoded in the variable BLOCK, with a list of other which can be changed (cum grano salis).