-
Notifications
You must be signed in to change notification settings - Fork 176
Dirac v6r11
The main point of this version is to remove the uselessly complex ReplicaManager class. It does not contain any change from the logic perspective, but introduces slightly different interface. It is a first step in the implementation of RFC #17. The replacement of PFN with (LFN,SE name) is to happen in v6r12.
The calls to the ReplicaManager class have to be replaced by calls :
- to the FileCatalog, if the method acts only on the FileCatalog. It is the case for all methods of the ReplicaManager which contains "Catalog" (eg. getCatalogReplicaStatus, removeCatalogReplica, etc).
- to the StorageElement if the method acts only on the StorageElement. It is the case for all methods of the ReplicaManager which contains "Storage" (eg. getStorageFileMetadata, prestageStorageFile, etc).
- to the new DataManager class if it implies both the StorageElement and the FileCatalog.
The SingleFile and SingleDir parameters are deprecated as well. The enforced return type convention is the successful/failed dictionaries. However, a utility function is provided to mock the deprecated arguments :
res = ReplicaManager().doSomething(arg1, arg2, singleFile = True )becomes
from DIRAC.Core.Utilities.ReturnValues import returnSingleResult
res = returnSingleResult(DataManager().doSomething(arg1, arg2 ))The catalogs parameter of the ReplicaManager is also removed. It allowed you to specify on which catalogs your method should act. This information should be now given in the constructor of the DataManager or the FileCatalog itself, and all the methods called with these instances will act only on the given catalogs.
rm = ReplicaManager()
res = rm.registerFile( fileTuple, catalog = 'LFC' )becomes
dm = DataManager(catalogs = ['LFC'])
res = dm.registerFile( fileTuple )The StorageElement class was slightly refactored in order to prepare the replacement of PFN with (LFN,SE name). The interface remains the same except for getPfnForProtocol and getPfnForLfn which now follow the successful/failed return convention, and a new getLfnForPfn method, that replaces the eponymous methods in the FileCatalog and the ReplicaManager.
Also, it is now encouraged to call the StorageElement methods with an LFN rather than a PFN. If you give it a PFN, a warning will be issued.
PR https://github.com/DIRACGrid/DIRAC/pull/1815 fixes the missing FK of ReqDB. To apply by hand the changes:
- PK of Request table is RequestID now (was RequestID + RequestName)
- RequestName is UNIQUE (unique index)
- added FK to Operation table to reference the request (index on Operation.RequestID automatically created)
- added FK to File table to reference the operation (index on File.OperationID automatically created)
Fields defined as VARCHAR(256) are now VARCHAR(255). This eventually fixes issues when utf8 encoding is used in the database and an index is created on these fields: it prevents mysql error 1071. Current databases in deployment to be changed by hand are:
- AccountingDB (several tables: keyFields, valueFields, bucketsLength)
- TransferDB (Channel: SourceSE, SourceSURL, TargetSE, TargetSURL; FTSReq: SourceSE, TargetSE)
- NotificationDB (ntf_AlarmLog: Modifications)
- JobDB (Jobs: ApplicationStatus)
- JobLoggingDB (LoggingInfo: ApplicationStatus)
To answer the ticket https://github.com/DIRACGrid/DIRAC/issues/1331, the reported value when removing a file is now the data size deleted. There is also a report done when using getFile.
- Job.setSystemConfig() method is dropped. Use setPlatform() which has the same meaning but is consistent with the rest of the WMS nomenclature
Pilot script installs DIRAC client with python 2.7 by default
New PoolComputingElement class to be used to explore algorithms of using multi-core computing slots
Possibility to specify arbitrary Tags in the computing resources description that can be matched to the job requirements