Pilot APIs
The Pilot is normally used as a command-line tool and launched by a wrapper script sent to the batch systems by a Pilot Factory. In case this is not wanted but some Pilot functionality is still needed, an external user may call relevant functions via simplified Pilot APIs that provide convenient access to internal Pilot modules and functions that otherwise may be difficult to use. Other APIs include the Services API (benchmarking, memory monitoring and analysis package) and Communicator API (server interactions).
The APIs are further described below.
The Data API was the first API to be delivered for use with Harvester on HPCs.
Example: stage-in without InfoSys initialisation
from pilot.api import data
from pilot.info import FileSpec
client = data.StageInClient()
files = [{'scope': 'mc16_13TeV', 'lfn': 'EVNT.11320990._003958.pool.root.1', 'workdir': '.',
'ddmendpoint': 'RRC-KI-T1_DATADISK'}]
xfiles = [FileSpec(type='input', **f) for f in files]
r = client.transfer(xfiles)
Example: stage-in with InfoSys initialisation
from pilot.api import data
from pilot.info import InfoService, FileSpec, infosys
infoservice = InfoService()
infoservice.init('ANALY_CERN', infosys.confinfo, infosys.extinfo)
client = data.StageInClient(infoservice)
files = [{'scope': 'mc16_13TeV', 'lfn': 'EVNT.11320990._003958.pool.root.1', 'workdir': '.'}]
xfiles = [FileSpec(type='input', **f) for f in files]
r = client.transfer(xfiles)
The Services API is further divided into a Benchmark API, Memory Monitoring API and Analysis API.
The Analysis API is currently limited to a linear fit module that is used to fit memory monitoring data and to report memory leaks.
Example: linear fit
from pilot.api import analytics
x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
y = [0.5, 1.1, 2.2, 3.6, 4.0, 5.0, 6.1, 7.7, 8.9, 9.1]
client = analytics.Analytics()
fit = client.fit(x, y)
slope = fit.slope()
intersect = fit.intersect()
(In development)
(In development)