Dirac FAQs

DIRAC FAQs

How to set "ApplicationStatus" from my running Job?

You have to use an instance of a JobReport object initialized with the JobID of the running job. To do so you can use a code like the following:

from DIRAC.Core.Base import Script
Script.parseCommandLine()

import DIRAC
from DIRAC.WorkloadManagementSystem.Client.JobReport        import JobReport
import os

# Get the JOBID from the environment (set by the JobWrapper)
jobID = os.environ.get('JOBID',False )
if not jobID:
  DIRAC.gLogger.error( 'JOBID not properly defined' )
else:
  # Create an instance of the JobWrapper
  jobReport = JobReport( jobID, 'MyApp' )
  # Update the Application Status
  jobReport.setApplicationStatus( '%s Running' % jobID )

If you want to report different States of your running application, you may call the setApplicationStatus as many times as you need.

How to set use the JobID in the name of my std.out/err files?

You can make the std.out/err of your jobs to be named using the JobID as part of their name. This will allow you to retrieve them (or to save them as OutputData) into a single directory without them being overwritten. By default the names "std.out" and "std.err" are used and, thus they have to be kept in separated directories.

If you are using a JDL to submit your job, you need to add the following lines:

StdOutput = "my%j.out";
StdError = "%j.err";
    OutputSandbox =
    {
        "%j.err",
        "%j.out"
    };

If you are using Job objects, follow this example:

from DIRAC.Interfaces.API.Job import Job
myJob = Job( stdout = "%j.out", stderr="%j.err" )

If you want to keep then permanently in a StorageElement, you can use the following lines in your JDL:

OutputData = "%j.out";
OutputPath = "DIRAC/OutputPath";

And if your are using the Job interface:

myJob.setOutputData( ["%j.out","%j.err" ], outputPath = 'DIRAC/OutputPath')

In all cases, after the job is received by the JobManager and receives a JobID, it will parse the JDL and replace every appearance of "%j" by the assigned JobID, thus obtaining the expected result.

How to avoid SE overloads from running jobs?

If the capacity of the SE does not match that of the CPU resources producing or consuming data one can easily overload a StorageElement with the upload or download activity from your jobs. The most typical case is when many jobs are starting (or ending) in a short period of time producing a kind of DOS attack to your SE.

To avoid this situation DIRAC provides a handle to limit the rate at which jobs are matched. This limit will slowdown the load of your CPU resources, but will reduce the peak load on your SE. In order to apply this limit Jobs have to be submitted with a given "JobType":

myjob.setType( "MCJobs" )

Then for each Site participating in your production define a proper delay in the Operations section of the configuration:

/Operations/Defaults/JobScheduling/MatchingDelay/[SiteName]/JobType/MCJobs = 60

Jobs will start at an interval of at least 60 seconds, thus alleviating the load on your SE.