v0.17.0
Features
- jdbc-*: drop worker_instance table #3650 (Loïc Mathieu)
- core: add generateRealtimeExecution on TriggerService (Ludovic DEHON)
- allows to use svg for tenant logo (#3646) #3646 (YannC)
- script: switch to Apache HTTP based Docker HTTP client #3656 (Loïc Mathieu)
- core: add inputFiles and outputFiles to WorkingDirectory (#3554) #3660 (Florian Hussonnois)
- core: add new WorkerJobLifecycle interface (#1498) #3647 (Florian Hussonnois)
- script: add support for killing docker container (#973) #3647 (Florian Hussonnois)
- ui: when readonly and JSON, format code (YannC)
- core: add a dump expression (Ludovic DEHON)
- ui: restore tab on monaco editor type input (Ludovic DEHON)
- ui: metric graph text are visible (Ludovic DEHON)
- ui: replace prism with shiki for code highlight (Ludovic DEHON)
- core: rename Pebble context() function to printContext() #3675 (Loïc Mathieu)
- add taskrunner usage #3676 (Loïc Mathieu)
- task to resume an execution #3681 (Loïc Mathieu)
- ui: replacing vscode editor with custom monaco implementation (#3568) #3568 (Miloš Paunović)
- ui: remove VSCode Namespace Files editor (brian.mulier)
- ui: structured settings page by sections (#3703) #3703 (Miloš Paunović)
- core: keep empty object on json serialization #2358 (Loïc Mathieu)
- core: use JsonInclude.NON_DEFAULT on plugin classes #2358 (Loïc Mathieu)
- core: if iteration is available, add it in runCOntext (YannC)
- ui: better concurrency UI (YannC)
- core: trigger task icon (YannC)
- core: remove our custom ObjectMapperFactory #3723 (Loïc Mathieu)
- core/scripts: properly handle kill of TaskRunner (#3700) #3719 (Florian Hussonnois)
- core: add a few methods to NamespaceFilesService and use existing methods for recursive listing (brian.mulier)
- subflow: change the defaults to wait until subflow completion (Anna Geller)
- core: new WaitFor task (#3652) #3652 (YannC)
- ui: removed draft (YannC)
- core: add alias to the old plugin name #3743 (Loïc Mathieu)
- *****: move plugin classes to a dedicated package #3743 (Loïc Mathieu)
- core: rename task Counts to Count #3743 (Loïc Mathieu)
- core: remove Flow and Worker task and use aliases #3743 (Loïc Mathieu)
- core: add HTTP tasks to the core plugin #3754 (Loïc Mathieu)
- support aliased ForEachItem and Schedule on the UI #3758 (Loïc Mathieu)
- core: allows multiple task for WaitFor (#3751) #3751 (YannC)
- webserver: autoload tutorial flows from blueprints (#3607) #3763 (Florian Hussonnois)
- core: enhance FlowService and add findByNamespacePrefix (brian.mulier)
- rename task in tests, comments, docs and UI (#3753) #3753 (Loïc Mathieu)
- core: include all variables in the run context #3774 (Loïc Mathieu)
- improve task default (#3775) #3775 (YannC)
- core: add missing test for null inputs #3779 (Loïc Mathieu)
- core: load task defaults from aliases #3781 (Loïc Mathieu)
- *****: rename task defaults as plugin defaults #3782 (Loïc Mathieu)
- core: rename VariableCondition to ExpressionCondition #3791 (Loïc Mathieu)
- core: Download - get the filename form the content disposition #3793 (Loïc Mathieu)
- allow file creation with specific name from runContext (#3796) #3796 (YannC)
- ui: implemented fresh and improved onboarding flow (#3804) #3804 (Miloš Paunović)
- cli: allow to start only some executors #3807 (Loïc Mathieu)
- core: add warning when a Flow trigger has no condition (brian.mulier)
- examples: update an example to use 0.17.0 plugin names (#3820) #3820 (Shruti Mantri)
- ui: improved file explorer empty state (#3838) #3838 (Miloš Paunović)
- check allowed namespaces on the Purge task #3836 (Loïc Mathieu)
- ui: bring relative date filter everywhere instead of date range only (brian.mulier)
- ui: autocompletion also works for nested tasks (brian.mulier)
- core: namespace files tasks (#3866) #3866 (YannC)
- core: use ant style matcher for namespaces tasks (YannC)
- ui: add autocompletion for root attributes (brian.mulier)
- ui: send template on guided tour events (Ludovic DEHON)
- core: create util for Path (YannC)
- ui: better error toast (Ludovic DEHON)
- ui: display server error when possible (Ludovic DEHON)
- ui: add auditlogs tabs on execution & flow (Ludovic DEHON)
- new CI and release process #3894 (Loïc Mathieu)
- ui: add link for ee on the left menu (Ludovic DEHON)
Bug Fixes
- core: prevent PluginRegistry from being init alongside ObjectMapper #3651 (Florian Hussonnois)
- core: return all tasks in execution dto task class (YannC)
- ui: prevent lazy-loaded fonts leading to cursor shift bug on Windows (brian.mulier)
- core: Also set parent taskrun to RETRIED (YannC)
- core: new execution retry fix (YannC)
- build: no more limit on the jar file amount (brian.mulier)
- force Commons Compress version (Loïc Mathieu)
- ui: refresh input when open trigger flow (YannC)
- ui: emit default value from inputsForm (YannC)
- core: preserve case on task default values (Ludovic DEHON)
- ui: prevent breaking execution overview during loading of flow definition (Ludovic DEHON)
- ui: add a filter on execution output (Ludovic DEHON)
- cli: ensure PluginRegistry init is lazy #3662 (Florian Hussonnois)
- ui: layout of plugins pages (Ludovic DEHON)
- ui: links on the Welcome page (Anna Geller)
- ui: better layout of markdown (Ludovic DEHON)
- ui: top level await are not supported (Ludovic DEHON)
- ui: task logo are missing on logs page (Ludovic DEHON)
- core: rename Pebble dump() function to context() function (Loïc Mathieu)
- repository-memory: possible NPE in MemoryLogRepository.findByExecutionIdAndTaskId #3676 (Loïc Mathieu)
- ui: fix flow inputs (#3668) #3679 (Florian Hussonnois)
- ui: handle inputs for type array (#3644) (Florian Hussonnois)
- core: redact the KESTRA_PLUGIN_PATH from the env vars available in a flow #1595 (Loïc Mathieu)
- core: move trigger tests on their own namespace #3682 (Loïc Mathieu)
- ui: restore autocompletion for flows (brian.mulier)
- core: fix task timeout (Florian Hussonnois)
- ui: ameded editor breaking on problematic yaml code pasting (#3704) #3704 (Miloš Paunović)
- ui: autocompletion for flowId was not working (brian.mulier)
- docs: correct the url to huggingface #3688 (Shruti Mantri)
- re-add the company name (Anna Geller)
- core: don't serialize the state endDate #2358 (Loïc Mathieu)
- core: remove the custom BeanSerializerFactory #2358 (Loïc Mathieu)
- ci: prevent running sonar if skip test (brian-mulier-p)
- core: UriProvider should output null if empty string (Ludovic DEHON)
- ui: metadata remove the outputs (Ludovic DEHON)
- ui: monaco editor is mixing models (Ludovic DEHON)
- ui: amended tabs closing issue on re-rendering (#3718) #3718 (Miloš Paunović)
- retry (Anna Geller)
- ui: added flow retry to metadata editor (#3721) #3721 (Miloš Paunović)
- ui: ability to scroll through multiple triggers to limit the column width (#3722) #3722 (Miloš Paunović)
- ui: logsWrapper use correctly namespace (YannC)
- ui: properly show editor on execution payload input (#3725) #3725 (Miloš Paunović)
- scripts: fix wrong warning on volume-enabled (#3608) #3726 (Florian Hussonnois)
- core: add outputs to WorkingDirectory (#3693) #3711 (Florian Hussonnois)
- ui: load more than 10 triggers in flow trigger page (YannC)
- remove handlebar expressions (Anna Geller)
- ui: remove vscode-web from gitignore (brian.mulier)
- typo (Anna Geller)
- ui: adjusting editor layout issues (#3746) #3746 (Miloš Paunović)
- docs: update link to templates migration page (#3739) #3739 (Pranav Nagarajan)
- ui: editor layout fix on templates page (#3747) #3747 (Miloš Paunović)
- ui: combined editor topology view height amended (#3748) #3748 (Miloš Paunović)
- ui: made copy flow functionality work again (#3750) #3750 (Miloš Paunović)
- ui: improved flow validation (#3755) #3755 (Miloš Paunović)
- ui: adding safeguard against current tab (#3756) #3756 (Miloš Paunović)
- core: avoid casting TaskForExecution to executableTask in graphs (YannC)
- ui: don't use lodash full (Ludovic DEHON)
- core: prevent a crash if a plugin is invalid (Ludovic DEHON)
- ui: improved flow and editor behaviour (#3761) #3761 (Miloš Paunović)
- core: check trigger before casting (YannC)
- core: Pebble JsonFunction and JsonFiter should use the stanard ObjectMapper #3765 (Loïc Mathieu)
- remove useless test #3764 (YannC)
- webserver: prevent FlowAutoLoaderService to crash when api is unavailable (Florian Hussonnois)
- core: add AbstractFlowRepositoryTest.findByNamespacePrefix test (brian.mulier)
- core: remove PluginCategory.DATA_QUALITY as it's not used (#3766) #3766 (Loïc Mathieu)
- ui: amended execute dialog form width issue (#3777) #3777 (Miloš Paunović)
- ui: dashboard date range shows full field (#3778) #3778 (Miloš Paunović)
- ui: prevent sending empty strings as payload for flow exectutions (#3780) #3780 (Miloš Paunović)
- tasks: add missing http icons (Ludovic DEHON)
- core: add a better error message for invalid expression (Ludovic DEHON)
- cli: load flows for webservers and standalone (#3783) #3784 (Florian Hussonnois)
- core: restore secrets encryption in logs #3787 (brian.mulier)
- webserver: return a Publisher and carefully handle all exceptional path #3797 (Loïc Mathieu)
- script: render the container image before passing it to the task runner #3799 (Loïc Mathieu)
- ui: dont add file type to formdata if value is null (YannC)
- core: add dry run importFlow in FlowService #3795 (brian.mulier)
- tests: add retrying to some flaky tests #3795 (brian.mulier)
- tests: increase readTimeout for RequestTest.selfSigned #3795 (brian.mulier)
- tests: increase HTTP readTimeout for webserver tests #3795 (brian.mulier)
- WaitFor docs (Anna Geller)
- core: Return metric correctly use rendered format (YannC)
- core: add missing icon for th runner subgroup (Loïc Mathieu)
- core: correctly use updated taskRun in waitFor nextDateEvaluation (YannC)
- core: aliases are now lowercased in Map for easier class retrieval #3803 (brian.mulier)
- ui: made import paths relative for ee build to work (#3806) #3806 (Miloš Paunović)
- outdated wdir examples (Anna Geller)
- core: fix kill method call for triggers #3808 (Florian Hussonnois)
- ui: null-safe Usages.vue #3809 (brian.mulier)
- ui: plugin card clicks now also open the first element of the plugin even if there is no tasks (triggers, conditions or taskRunners) (brian.mulier)
- add deprecation marker for Template (Anna Geller)
- webserver: prevent zombie folders when deleting namespace files and there is no more file in it (brian.mulier)
- storage-local: regression on LocalStorage allByPrefix (brian.mulier)
- storage-local: add log in case of failing file visit (brian.mulier)
- ui: improved onboarding styles on light theme (#3815) #3815 (Miloš Paunović)
- ui: more explicit "save task" button to avoid confusion with saving flow in task editor (brian.mulier)
- improve the error message (Anna Geller)
- ui: amended editor sidebar to properly refresh list of files (#3833) #3833 (Miloš Paunović)
- core: proper error message for invalid input #3813 (Ludovic DEHON)
- core: create new runcontextlogger when cleaning up in runContext (YannC)
- ui: adapted layout on welcome page for different screen sizes (#3839) #3839 (Miloš Paunović)
- docs: remove vscode editor reference (#3741) #3741 (Will Russell)
- ui: prevented tabs to overflow into a global scroll (#3846) #3846 (Miloš Paunović)
- core: dynamic tasks #3850 (Loïc Mathieu)
- tests: cross tenant test in StorageTestSuite was wrong (brian.mulier)
- core: render input files #3854 (Loïc Mathieu)
- core: missing @PluginProperty on Pause.onResume (Loïc Mathieu)
- core: WaitFor task #3860 (Loïc Mathieu)
- ui: amended first click on resume execution butotn (#3865) #3865 (Miloš Paunović)
- core: multipleConditionTriggerFailed() is flaky #3862 (Loïc Mathieu)
- docker task runner type (Anna Geller)
- ui: editor cursor position on windows is now recalculated properly (#3872) #3872 (Miloš Paunović)
- ui: scope buttons css to prevent autocompletion from being invisible (brian.mulier)
- core/jdbc: fix JdbcServiceInstanceRepository predicate for finding active instances (Florian Hussonnois)
- ui: included python into list of languages for markdown highlighter (#3890) #3890 (Miloš Paunović)
- QA: resolve QA issues (YannC)
- improve Docker process stability on Windows OS (#3834) #3834 (YannC)
- core: prevent updating disable state of a real time trigger from the trigger api (YannC)
- core: waitfor maxIteration do not stop too early (YannC)
- core: use attribute instead of method for line separator in TargetOS (YannC)
- script: do not use
set -e
option when on windows (#3892) #3892 (YannC) - core: dont keep null value when transforming input files (YannC)
- script: set task-runner type legacy DockerOptions (#3896) #3898 (Florian Hussonnois)
- core: missing variables in rendering (#3910) #3910 (YannC)
- ui: flow buttons were no longer showing (brian.mulier)
- ui: entreprise tooltip term can be null (Ludovic DEHON)
- core: avoid using removeIf in PluginUtilsService (YannC)
- ui: import javascript langage for shiki (YannC)
- support alias for plugin-configuration #3912 (Florian Hussonnois)
- webserver: validate start/end date timeline and throw if not correct (#3911) #3911 (YannC)
- core: force shutdown after
terminationGracePeriod
#3909 (brian.mulier) - tests: null-safe findByExecutionId in MemoryLogRepository (brian.mulier)
- tests: run receive consumers cancellations to prevent assignment stealings between tests (brian.mulier)
- namespace files tasks (Anna Geller)
- Maven description (Loïc Mathieu)
Documentation
- add Pebble expressions to the main UI documentation for a quick reference (Anna Geller)
- Resume task (Anna Geller)
Code Refactoring
- core: don't expose multiple entry on collector service (Ludovic DEHON)
- ui: optimized file icon package (#3844) #3844 (Miloš Paunović)
Builds
- deps: use jackson version 2.16.2 (Florian Hussonnois)
Chores
- script: deprecate outputDir in favor of outputFiles (#3492) #3657 (Florian Hussonnois)
- refactor runContext deprecated method #3655 (Florian Hussonnois)
- deps: update most java deps (Ludovic DEHON)
- deps: update javascript deps (Ludovic DEHON)
- use parallelStream for loading plugins (Florian Hussonnois)
- fix makefile to wait for pg before starting kestra (Florian Hussonnois)
- ui: amending the initial editor implementation (#3689) #3689 (Miloš Paunović)
- ui: improvements of the editor usability (#3696) #3696 (Miloš Paunović)
- upgrade kestra ui-lib version (YannC)
- core: remove unnecessary saveFlowableOutput method calls #3711 (Florian Hussonnois)
- fix more Jackson lib versions #3727 (Loïc Mathieu)
- core: deprecated LocalFiles task (YannC)
- core: clean PluginScanner log (Florian Hussonnois)
- ui: minor improvements on settings page and cleaning up (#3773) #3773 (Miloš Paunović)
- ui: changed the color of number input buttons on settings page (#3776) #3776 (Miloš Paunović)
- core: filter log message (Loïc Mathieu)
- core: use const instead of magic number (YannC)
- ui: task icon improvements on onboarding (#3816) #3816 (Miloš Paunović)
- upgrade to Micronaut 4.4.3 (Loïc Mathieu)
- ui: increased size of the select icons (#3841) #3841 (Miloš Paunović)
- *****: dependency upgrade #3842 (Loïc Mathieu)
- use the Micronaut Platform BOM (Loïc Mathieu)
- ui: include subtasks in flow icons for onboarding tour (#3845) #3845 (Miloš Paunović)
- ui: improved code example image on welcome page (#3848) #3848 (Miloš Paunović)
- ui: improved overall onboarding tour (#3868) #3868 (Miloš Paunović)
- core: enhance collectors service (Florian Hussonnois)
- ui: tweaked onboarding tour for both light and dark modes (#3878) #3878 (Miloš Paunović)
- ui: preventing execution modal closing during onboarding tour (#3879) #3879 (Miloš Paunović)
- ui: hiding tour skip button and repositioning card on last step (#3880) #3880 (Miloš Paunović)
- ui: improved overall onboarding experience (#3887) #3887 (Miloš Paunović)
- ui: more improvements of the overall onboarding experience (#3889) #3889 (Miloš Paunović)
- ui: added code image for welcome page for light mode (#3893) #3893 (Miloš Paunović)
- ui: properly colouring task icons on both modes (#3895) #3895 (Miloš Paunović)
- fix docker wokflow job as no dependency (Loïc Mathieu)
- ui: no onboarding menu item on ee instances (#3913) #3913 (Miloš Paunović)
- fix download artifact (Loïc Mathieu)
- move PluginUtilsService out of the tasks package so it could be removed soon #3914 (Loïc Mathieu)
- version: update to version 'v0.17.0'. (Loïc Mathieu)
Commits
- f5586d2: closes #3670 (Anna Geller)
- a72e391: feat(core, webserver): alias improvements (Loïc Mathieu) #3740
- 9e5a605: fix(core, ui): display alias doc in the editor (Loïc Mathieu) #3745
- 17ccfcf: feat(core, script): rename DockerTaskRunner to Docker and ProcessTaskRunner to Process (Loïc Mathieu) #3743
- 513e69d: fix(ui); remove lang settings from date/time section (YannC)
- 7f6881d: WIP feat(core): add new stop method WorkerJobLifecycle (Florian Hussonnois) #3767
- d93761e: fix(core, webserver): file upload (Loïc Mathieu) #3429
- f6f5011: Merge branch 'develop' of https://github.com/kestra-io/kestra into develop (Anna Geller)
Non-breaking changes related to renaming
Many plugins have been renamed for consistency and extensibility. We’ve implemented aliases for backward compatibility so that you don't need to update those names immediately. Everything will continue to work seamlessly, and you can take as much time as needed to rename those tasks and triggers.
Here are the key changes:
taskDefaults
are nowpluginDefaults
to reflect the fact that you can set default values for all plugins, not just tasks.- HTTP tasks are now part of the
core
plugin rather than the file-system (fs
) plugin. - All core plugins have been renamed, e.g.
io.kestra.core.models.triggers.types.Schedule
is nowio.kestra.plugin.core.trigger.Schedule
. Again, this is a non-breaking change as we use aliases — don't worry when you see warnings in the UI. Those warnings are here to spread the message about renamed tasks and triggers, but you can take as much time as you need to rename them. - Serialization tasks have been renamed from Readers and Writers to explicit conversion tasks e.g.
CsvReader
→CsvToIon
andCsvWriter
→IonToCsv
. - Redis plugins have been renamed to make them easier to extend and maintain. For example,
io.kestra.plugin.redis.TriggerList
is nowio.kestra.plugin.redis.list.Trigger
.
For more details about all these changes, check the Renamed Plugins Migration Guide.
Breaking Changes
Task Runner names have been shortened
The task runner classes are now shorter. For example, io.kestra.plugin.aws.runner.AwsBatchTaskRunner
is now io.kestra.plugin.aws.runner.Batch
. Check the task runner documentation for the full list of updated names.
Note that in contrast to the renaming mentioned in the previous section, we've renamed them directly (without using aliases) given it's a beta feature.
New build process for custom plugins
Kestra 0.17.0 uses a new mechanism to discover and load plugins. Plugins are now discovered and loaded using the standard Java Service Loader rather than Micronaut-specific Bean introspection. If you use custom plugins, follow the Migration Guide to make the necessary adjustments.
Subflow default values have changed
The wait
and transmitFailed
boolean properties of the Subflow
task are now set to true by default. This means that the default behavior of the Subflow
task is now waiting for the subflow execution to finish and transmitting the failed status to the parent flow's Subflow
task runs. If you want to keep the old default behavior, make sure to add the following to your pluginDefaults
:
pluginDefaults:
- type: io.kestra.core.tasks.flows.Subflow
values:
wait: false
transmitFailed: false
Change in the JSON objects serialization strategy
We previously serialized JSON objects with a NON_DEFAULT
strategy to save space (in the database) and optimize network bandwidth.
We changed that behavior so that all properties that are not null will now be included in the serialized JSON document. This fixes several limitations and lets us know when a default is purposely set. This also means that empty lists and maps are now serialized as empty objects (rather than not being serialized at all).