v0.1.19

Breaking changes

• Rename abs_dirname -> absdirname in file targets. (5593f18)

Features & improvements

• Add 'law luigid' sub cli. (bd0cd88)
• Adapt live task resources in remote workflow. (88c4215)
• Unified permission support for target formatter dump. (976d53b)
• Automatic pattern detection in multi_match. (673c2ac)
• Add mirrored targets. (f4a0203)
• Human readable temporary job submission directories (4a05454, 93574bd)
• Improve node naming in ForestMerge. (7ff7513)
• Remove potentially expensive check in ForestMerge. (f295d05)
• Add keep_missing flag to mask_struct util. (e22d2bd)
• Additional early stopping in remote workflow. (245e0a7)
• Improve remote fs defaults. (893eb5f)
• Unify sandbox variable parsing, allow pre and post setup commands. (44b34c5)
• Improve workflow and sandbox proxy handling. (1ac4f1a)
• Add option to eval workflow condition as workflow only. (bb7b363)

Fixes

• Handle file collisions for grouped job submissions, fix #183. (d60ea71)
• Fix mirrored target checks in collections. (87b4017)
• Fix root path check in mirrored targets. (ff594e7)
• Fix TargetCollection directory check. (291723b)
• Fix empty workflows. (f57a677)
• Fix target makedirs. (2f4e664)
• Fix indentation in print-status output. (ea3585c)
• Fix workflow resources for branches. (a179eae)
• Fix workflow attribute fallbacks. (83d9023)
• Fix deferred config resolution. (4389a86)
• Fix TypeError causing function call (#175, 1848c57)
• Remove redundant global args from remote commands. (ab5bb48)
• Drop effective_workflow in remote command. (62d4576)
• Fix local target copy with non-root fs. (e02edde)

contrib packages

• [htcondor] Grouped submission using job clusters (682c983, 34ba133, c475ebb)
• [htcondor] Allow -spool jobs. (e9408bf)
• [htcondor] Fix htcondor log files. (08804d1)
• [pandas] Add pandas contrib with file formatter (8ac97e0, ca4e237, @Nollde)
• [singularity] Fix law sw forwarding into singularity sandboxes (#190, @EthanMarx, e97ab26)
• [cms] Fix crab wrapper file incrementation. (af59c5e)
• [cms] Add option to pass args to custom cmssw setup. (64dbe8d)
• [matplotlib] Set default file permissions in matplotlib formatter (caacf90)
• [slurm] Change tmp dir usage (#184, 6c714a8)

v0.1.18

Breaking changes

None

Features & improvements

• Refactor dynamic workflow condition. (95bedec, 306046d)
• Place remote workflow run() into optional context. (2146c7a)
• Add local_workflow_pre_run hook to local workflow proxy. (bda49ee)
• Add mypy to future-proof images. (05292ba, fbaaa5b)
• Change docs requirements. (84c0a0c)
• Add range expansion to range parameters. (5d0f2ed)

Fixes

• Fix FileSystemFileTarget.move_from. (cde51a9)
• Minor fix of local fs base handling. (d687241)

contrib packages

None

v0.1.17

Breaking changes

None

Features & improvements

• Fallback for python executable in scripts. (1b6388e)
• Update cern htcondor settings in examples. (457c653)

Fixes

• Fix stray line in slurm job definition. (9ae93e6)

contrib packages

None

v0.1.16

Breaking changes

None

Features & improvements

• Silence redundant interface logs when sandboxed. (5561678)
• Improve CI actions (324a43a, a320169, 449add7, 84b2f2e)
• Start util tests. (e1e742c)

Fixes

• Fix interactive parameter evaluation for sandbox tasks. (a9dc218)
• Fix setup. (e6fe30e)

contrib packages

None

v0.1.15

Breaking changes

None

Features & improvements

• Split image workflows. (65e6b32)
• Streamline docker image builds, add alma9 images. (#169) (e4afe3a)
• Add custom parameter base class. (e16012b)
• Preserve job data of skipped jobs. (57b5328)

Fixes

• Fix parameter encoding for law_run(). (4a0372b)
• Fix localize decorator. (9c2df74)
• Update CmdlineParser patch. (6d39db0)

contrib packages

• [htcondor] Improve error extraction from query response. (4d843dd)

v0.1.14

Breaking changes

None

Features & improvements

• Add dynamic_workflow_condition. (5173f91)
• Handle potential pathlib.Path's throughout law, close #167. (dca607b)
• Add 'law quickstart' cli, close #164. (ce82c5f)
• Improve config handling in remote file interface. (6fb41c1)
• Improve config variable expansion in sandboxing. (e2fc8fc)
• Simplify log file handling and extra data for remote jobs. (e1ccb60)
• Improve sandbox key parsing. (e300819)
• Print contrib package location via 'law location'. (bae1576)

Fixes

• Explicit sandbox config section names, close #165. (52fe3f9)
• Fix sandbox stage-in vs. localization interplaye, fix #168. (d301b94)

contrib packages

• Add mechanism to have contrib packages declare configs. (4af6f5f)
• [cms] Add option to include additional files in bundling. (d44a138)
• [cms] Minor fixes in crab config building. (2ca2340)
• [cms] Extract input files in job wrapper. (6b15fac)

v0.1.13

Breaking changes

• Combine {cached_,}workflow_property. (d4b12ff)
• Rename cfg.get_expanded_boolean -> cfg.get_expanded_bool. (4c3a503)

Features and improvements

Larger enhancements

• Crab job submission and workflows (c7f39b3) 🎉
• Workflow parameters (#159) (cec32d9)
• Pass through requested workflow (#155) (f54534a)
• Add support for group jobs. (9cd02d9, 8f2d086, f8ce46c) 🎉
• Deferred config values. (18e32ed)

Workflow and job handling

• Add mechanism to register contrib install scripts. (c718d67)
• Improve separating of workflow and branch attributes. (7d4f679)
• Improve parameter parsing. (d62f1f9)
• Add --assistant to global cli params. (3e5c233)
• Add options to read stdin in interruptable_popen. (8e281ff)
• Allow poll_callback return value to stop polling loop. (574ed58)
• Add additional hook for adding workflow control output postfixes. (aaaa6f6)
• Allow delay before job completeness checks in remote workflow. (b18d4ce)
• Add switch to check task completion after remote jobs finsihed. (e75728b)

General improvements

• Improve generic job script. (8bd879c)
• Mark job section in crab job wrapper. (a2d1d05)
• Add hook to setup job managers. (55515ba)
• Update job error summary grouping. (db1e74c)
• Improve error logs during job status checks in remote workflow. (4caf842)
• Make sandbox executable configurable. (a0c559c)
• Add make_set to utils. (c414b45)
• Add convert_types to util.mask_struct. (71f3796)
• Add classmethod checking to util. (e56e6cf)
• Support brace expansion in module list for 'law index'. (43fee29)
• Improve interactive task status formatting. (91de7b6)
• Better "optional" handling in collections (#149) (bdb908d, 752b1a5)
• Streamline sandbox env caching and activity status. (cca3e22)
• Improve control over repr parameters when empty. (991208f)
• Refactor sandbox target stage-in/-out. (a87377d)
• Allow single value parsing in CSVParameter and MultiCSVParameter. (cc2aa0a)

Improvements in contrib packages

• Consistent naming of utils in arc contrib. (d56380d)
• Refactor voms proxy renewal and myproxy delegation. (f9b1d7d)
• Add myproxy delegation tools to wlcg and cms contribs. (d696c93)
• Add rlocks to guard checksum creation in contrib bundle tasks. (04fedbf)
• Catch xrootd listdir error in gfal interface. (f823440)

Fixes

• Backwards compatible lookup of log file in job data. (d067e02)
• Fix username in myproxy_info util in wlcg contrib. (2ae185f)
• Fix is_classmethod util, close #162. (02a71d4)
• Fix disabling of instance caching for proxy tasks. (2751204)
• Fix gfal_transfer_overwrite (#160) (6199d42)
• Consistent usage of destination_info. (02d504c)
• Patch luigi run result in sandboxes. (6c02198)
• Fix inheritance issue in contrib ForestMerge task. (a892b9f)
• Fix entry point in example image, close #154. (e470fd3)
• Fix deps in docker and singularity sandboxes. (3b18aff)
• Fix readable_popen util. (e9100d4)
• Fix interactive messages. (cf1dfa3)
• Fix slurm job status queries. (22ced6c)
• Fix bookkeeping of finished jobs in remote workflow. (eee5d26, 6d88cf4)
• Fix job file merging setting in htcondor workflow, fix job id assignment. (fb58ec4)
• Fix cached workflow forwarding (#147) (2f99321, f334bd3)
• Fix duration parsing. (09caa5d)
• Fix branch selection workflows. (098ec08)

Misc

• Update project list. (d277954)
• Update install instructions. (ba8c4f2)
• Restructure readme. (830a5e3, eceea23)
• Modernize docs. (4a9c2f3, 9d2d55c, a0cd688)
• Use LFS. (6450ef5)

v0.1.12

New features

• Add load and dump support for directory targets. (fa18d32)
• Add localization support for directory targets. (2108295)
• Add TFSavedModelFormatter to tensorflow contrib. (78e5ef7)
• Add DaskAwkwardFormatter to awkward contrib. (a36219a)
• Add notebook example. (e92fb58)

Improvements

• Forward some requests from remote to local fs. (585e3de)
• Refactor parquet file merging in pyarrow contrib. (f13d6a5)
• Improve %ilaw ipython magic. (e4a213a)
• Parse left-over string parameters. (b1abe37)

Breaking changes

None.

Fixes

• Fix legacy getargspec usage. (097245c)
• Fix stat messages after file merigng in root contrib. (d9bc203)
• Fix logging on colab. (b8d00e3)
• Reset parser objects after law_run util. (8105dee)

v0.1.11

Features

• Add parameter copy helper. (118f977)
• Make c8-py310 image default one. (eef8289, a84fea3)

Fixes & improvements

• Fix interactive output of tasks' fetch_output. (02dc374)
• Improve branch boundary updates in workflows. (5ba9d74)
• Disable instance caching for proxy tasks. (❗️a2a8e49) (Note: this fix solves issues with sandboxes complaining about LAW_SANDBOX_WORKER_ID not being set)

v0.1.10

Features

• Add OptionalBoolParameter for transparently handling None's. (9c2e0d6)
• Update examples. (b9adec9, d7d8e9a)
• Allow and invoke auto load hooks in contrib packages. (2f6246e)
• Pass-through multi-core setting to remote jobs. (🎉, 70a223f)
• Allow non-parameter values in task repr. (f98d92d)

Fixes & improvements

• Fix remote target cache invalidation. (❗️, 239454d)
• Enforce abspaths in contrib packages. (dbcc1b5)
• Fix prog in %ilaw ipython magic, use auto_load. (fa84765)
• Fix interactive task output. (c845aeb)