2.4.1

What's Changed

• Convert given CF names by @Zelldon in #408
• fix: err fallback on key formatting by @Zelldon in #409

Full Changelog: 2.4.0...2.4.1

2.4.0

What's Changed

Features

Extended Key formatters

Extend key formatters by @Zelldon in #405

Example

$ ./zdb state list -cf ELEMENT_INSTANCE_KEY -p raft-partition/partitions/1/snapshots/15-1-38-53/ | jq
{
  "data": [
    {
      "cf": "ELEMENT_INSTANCE_KEY",
      "key": "2251799813685251",
      "value": {
        "parentKey": -1,
        "childCount": 1,
        "childActivatedCount": 0,
        "childCompletedCount": 0,
        "childTerminatedCount": 0,
        "jobKey": 0,
        "multiInstanceLoopCounter": 0,
        "interruptingElementId": "",
        "calledChildInstanceKey": -1,
        "elementRecord": {
          "key": 2251799813685251,
          "state": "ELEMENT_ACTIVATED",
          "processInstanceRecord": {
            "bpmnElementType": "PROCESS",
            "elementId": "run-release",
            "bpmnProcessId": "run-release",
            "version": 1,
            "processDefinitionKey": 2251799813685249,
            "processInstanceKey": 2251799813685251,
            "flowScopeKey": -1,
            "bpmnEventType": "UNSPECIFIED",
            "parentProcessInstanceKey": -1,
            "parentElementInstanceKey": -1,
            "tenantId": "<default>"
          }
        },
        "activeSequenceFlows": 0,
        "userTaskKey": -1,
        "executionListenerIndex": 0
      }
    },

We support most of the currently used column families with built-in key formatters now.

Frontend for ZDB

Add Frontend (JavaFX based) to ZDB by @Zelldon in #407

New deliverable and script added to release zdb-frontend and zdb-frontend.jar .
Run ./zdb-frontent or java -jar zdb-frontend.jar and start the new ZDB frontend, allowing similar investigation to the CLI tool.

Investigating the log, with extra filters to reduce noise

Investigating the state, with possibility to check all column families, reusing key-formatters to show the right key-values.

Dependencies

• build(deps-dev): bump org.slf4j:slf4j-simple from 2.0.12 to 2.0.13 by @dependabot in #389
• build(deps): bump version.slf4j from 2.0.12 to 2.0.13 by @dependabot in #388
• build(deps): bump dependabot/fetch-metadata from 2.0.0 to 2.1.0 by @dependabot in #391
• build(deps): bump kotlin.version from 1.9.23 to 1.9.24 by @dependabot in #392
• build(deps): bump zeebe.version from 8.5.0 to 8.5.1 by @dependabot in #394
• build(deps): bump version.picocli from 4.7.5 to 4.7.6 by @dependabot in #395
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.7 to 1.19.8 by @dependabot in #396
• build(deps): bump com.mycila:license-maven-plugin from 4.3 to 4.5 by @dependabot in #397
• build(deps): bump kotlin.version from 1.9.24 to 2.0.0 by @dependabot in #398
• build(deps): bump org.assertj:assertj-core from 3.25.3 to 3.26.0 by @dependabot in #399
• build(deps): bump docker/login-action from 3.1.0 to 3.2.0 by @dependabot in #400
• build(deps): bump com.google.cloud.tools:jib-maven-plugin from 3.4.2 to 3.4.3 by @dependabot in #401
• build(deps): bump zeebe.version from 8.5.1 to 8.5.3 by @dependabot in #406
• build(deps): bump org.jetbrains.kotlinx:kotlinx-serialization-json-jvm from 1.6.3 to 1.7.0 by @dependabot in #404
• build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.1 to 3.4.0 by @dependabot in #403

Full Changelog: 2.3.0...2.4.0

2.3.0

What's Changed

• build(deps): bump dependabot/fetch-metadata from 1.6.0 to 2.0.0 by @dependabot in #385
• build(deps): bump com.google.cloud.tools:jib-maven-plugin from 3.4.1 to 3.4.2 by @dependabot in #386
• build(deps): bump zeebe.version from 8.4.5 to 8.5.0 by @dependabot in #387
• Add new version v85 test by @Zelldon in #390

Full Changelog: 2.2.0...2.3.0

2.2.0

What's Changed

• build(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.21.1 to 2.22 by @dependabot in #351
• build(deps-dev): bump io.zeebe:zeebe-test-container from 3.6.2 to 3.6.3 by @dependabot in #353
• build(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.22 to 2.22.1 by @dependabot in #355
• build(deps): bump org.assertj:assertj-core from 3.25.1 to 3.25.2 by @dependabot in #356
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.3 to 1.19.4 by @dependabot in #357
• build(deps): bump zeebe.version from 8.4.0 to 8.4.1 by @dependabot in #358
• build(deps): bump org.junit.jupiter:junit-jupiter-api from 5.10.1 to 5.10.2 by @dependabot in #359
• build(deps): bump org.assertj:assertj-core from 3.25.2 to 3.25.3 by @dependabot in #361
• build(deps): bump org.junit.jupiter:junit-jupiter-engine from 5.10.1 to 5.10.2 by @dependabot in #360
• build(deps-dev): bump org.slf4j:slf4j-simple from 2.0.11 to 2.0.12 by @dependabot in #363
• build(deps): bump version.slf4j from 2.0.11 to 2.0.12 by @dependabot in #362
• build(deps): bump zeebe.version from 8.4.1 to 8.4.2 by @dependabot in #364
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.4 to 1.19.5 by @dependabot in #365
• build(deps): bump zeebe.version from 8.4.2 to 8.4.3 by @dependabot in #366
• build(deps): bump org.jetbrains.kotlinx:kotlinx-serialization-json-jvm from 1.6.2 to 1.6.3 by @dependabot in #367
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.5 to 1.19.6 by @dependabot in #368
• build(deps): bump com.google.cloud.tools:jib-maven-plugin from 3.4.0 to 3.4.1 by @dependabot in #372
• build(deps): bump zeebe.version from 8.4.3 to 8.4.4 by @dependabot in #371
• build(deps): bump actions/setup-java from 4.0.0 to 4.1.0 by @dependabot in #373
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.6 to 1.19.7 by @dependabot in #374
• build(deps): bump kotlin.version from 1.9.22 to 1.9.23 by @dependabot in #375
• build(deps): bump zeebe.version from 8.4.4 to 8.4.5 by @dependabot in #376
• build(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.22.1 to 2.23 by @dependabot in #377
• build(deps): bump org.apache.maven.plugins:maven-assembly-plugin from 3.6.0 to 3.7.0 by @dependabot in #378
• build(deps): bump docker/login-action from 3.0.0 to 3.1.0 by @dependabot in #379
• build(deps): bump actions/setup-java from 4.1.0 to 4.2.1 by @dependabot in #381
• build(deps-dev): bump org.awaitility:awaitility from 4.2.0 to 4.2.1 by @dependabot in #382
• build(deps): bump org.apache.maven.plugins:maven-compiler-plugin from 3.12.1 to 3.13.0 by @dependabot in #384
• build(deps): bump org.apache.maven.plugins:maven-assembly-plugin from 3.7.0 to 3.7.1 by @dependabot in #383
• feat: decode keys in state list command by @oleschoenburg in #370

Full Changelog: 2.1.0...2.2.0

2.1.0

What's Changed

• Update Zeebe dependency to 8.4, additionally migrated to Java 21.

Deps

build(deps): bump org.junit.jupiter:junit-jupiter-engine from 5.10.0 to 5.10.1 by @dependabot in #323

• build(deps): bump org.junit.jupiter:junit-jupiter-api from 5.10.0 to 5.10.1 by @dependabot in #324
• build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.1 to 3.2.2 by @dependabot in #325
• build(deps): bump zeebe.version from 8.3.1 to 8.3.2 by @dependabot in #326
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.1 to 1.19.2 by @dependabot in #327
• build(deps): bump org.jetbrains.kotlinx:kotlinx-serialization-json-jvm from 1.6.0 to 1.6.1 by @dependabot in #328
• build(deps): bump zeebe.version from 8.3.2 to 8.3.3 by @dependabot in #329
• build(deps): bump org.testcontainers:junit-jupiter from 1.19.2 to 1.19.3 by @dependabot in #330
• build(deps): bump kotlin.version from 1.9.20 to 1.9.21 by @dependabot in #331
• build(deps): bump actions/setup-java from 3.13.0 to 4.0.0 by @dependabot in #333
• build(deps): bump org.jetbrains.kotlinx:kotlinx-serialization-json-jvm from 1.6.1 to 1.6.2 by @dependabot in #334
• build(deps): bump zeebe.version from 8.3.3 to 8.3.4 by @dependabot in #335
• build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.2 to 3.2.3 by @dependabot in #336
• build(deps): bump github/codeql-action from 2 to 3 by @dependabot in #337
• build(deps): bump org.apache.maven.plugins:maven-compiler-plugin from 3.11.0 to 3.12.0 by @dependabot in #338
• build(deps-dev): bump io.zeebe:zeebe-test-container from 3.6.0 to 3.6.1 by @dependabot in #339
• build(deps-dev): bump io.zeebe:zeebe-test-container from 3.6.1 to 3.6.2 by @dependabot in #341
• build(deps): bump kotlin.version from 1.9.21 to 1.9.22 by @dependabot in #340
• build(deps): bump org.apache.maven.plugins:maven-compiler-plugin from 3.12.0 to 3.12.1 by @dependabot in #342
• build(deps): bump version.slf4j from 2.0.9 to 2.0.10 by @dependabot in #344
• build(deps-dev): bump org.slf4j:slf4j-simple from 2.0.9 to 2.0.10 by @dependabot in #343
• build(deps): bump org.assertj:assertj-core from 3.24.2 to 3.25.0 by @dependabot in #345
• build(deps): bump org.assertj:assertj-core from 3.25.0 to 3.25.1 by @dependabot in #346
• build(deps-dev): bump org.slf4j:slf4j-simple from 2.0.10 to 2.0.11 by @dependabot in #349
• build(deps): bump version.slf4j from 2.0.10 to 2.0.11 by @dependabot in #348
• build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 by @dependabot in #350

Full Changelog: 2.0.2...2.1.0

2.0.2

What's Changed

• Fix reading of old records and rejectiontype by @Zelldon in #322

Full Changelog: 2.0.1...2.0.2

2.0.1

What's Changed

• build(deps): bump kotlin.version from 1.9.10 to 1.9.20 by @dependabot in #318
• Ignore empty async created segments by @Zelldon in #319

Full Changelog: 2.0.0...2.0.1

2.0.0

Refactoring of ZDB

Key facts

• Dependencies were reduced/removed
  • no longer depending on engine/state classes to read state, allows to handle newer versions better
  • reduces the size of the end artifact by ~30% (from 90 to 60 MB).
• Several bug fixes and better error handling
  • especially for unknown values (e.g. column families)
• Large improvement in test coverage (by 353% from 45 to 159 tests)
  • especially IT tests with different versions
  • allows support for different Zeebe versions, 8.1, 8.2, 8.3
• Experimental state feature moved to stable
• Remove unused features
  • banned instances sub-command, can be also investigated with state command
  • general status, was replaced with the state command

Details

State

The previously introduced experimental feature, where we simply open a read-only RocksDB instance and read the data without state classes, and the ZeebeDB module was upgraded to a stable state. It replaced all previous usage of state and engine classes.

This means now, instead of using engine-related classes to deserialize data we use simply msgpack to json conversation. This allows to remove the dependency to such modules and classes and to better handle newer versions, where classes or data formats are not known. The assumption is that all values are always msgpack encoded (which is right now the case in Zeebe).

Data is read via visitor patters, this means we no longer read all data into memory before printing to system out.

Log

The log classes have been updated to the recent changes, which means the versions checks are removed again. Mostly copied from Zeebe journal code.

Remove several Zeebe interface usages to avoid incompatibles and breaking changes on newer versions

Tests

Several tests have been adjusted and for different versions they are repeated now. We have now a test suite that runs tests against the following versions: 8.1, 8.2, 8.3, and SNAPSHOT.

SNAPSHOT should allow us to easily detect when something breaking is introduced. The other versions allow us to verify that ZDB is working against such versions, which wasn't the case before.

There were several new edge cases verified and tested, like what if a different format is used in the values (but still msgpack), what if the column family is not known (not corresponding to the current version) etc.

Make tests reproducible and predictable on all versions, via enabling WAL and enabling batch processing, etc.

Related issues

• #309
• #302
• #171
• #167

PR's merged

• build(deps): bump actions/setup-java from 3.12.0 to 3.13.0 by @dependabot in #299
• build(deps): bump com.google.cloud.tools:jib-maven-plugin from 3.3.2 to 3.4.0 by @dependabot in #300
• build(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.20 to 2.21.1 by @dependabot in #301
• build(deps): bump zeebe.version from 8.2.13 to 8.2.14 by @dependabot in #303
• build(deps): bump zeebe.version from 8.2.14 to 8.2.15 by @dependabot in #304
• build(deps-dev): bump org.testcontainers:junit-jupiter from 1.19.0 to 1.19.1 by @dependabot in #306
• build(deps): bump com.mycila:license-maven-plugin from 4.2 to 4.3 by @dependabot in #305
• build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by @dependabot in #312
• Refactoring of ZDB by @Zelldon in #311
• build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.0 to 3.3.1 by @dependabot in #313
• build(deps): bump zeebe.version from 8.3.0 to 8.3.1 by @dependabot in #314
• build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by @dependabot in #315

Full Changelog: 1.9.0...2.0.0

1.9.0

What's Changed

Features

Added new option to print in table/csv format see #298

Allows to print the Zeebe log in a more human-readable format

Index Term Position SourceRecordPosition Timestamp Key RecordType ValueType Intent ProcessInstanceKey BPMNElementType 
836304299 304 6888891251 6888891165 1692869669764 2251802375814764 EVENT JOB_BATCH ACTIVATED
836304300 304 6888891252 -1 1692869671138 -1 COMMAND PROCESS_INSTANCE_CREATION CREATE -1 
836304301 304 6888891253 6888891180 1692869671126 2251802375814766 EVENT VARIABLE CREATED 2251802375814765 
836304301 304 6888891254 6888891180 1692869671126 2251802375814767 EVENT VARIABLE CREATED 2251802375814765 
836304301 304 6888891255 6888891180 1692869671126 2251802375814768 EVENT VARIABLE CREATED 2251802375814765 
836304301 304 6888891256 6888891180 1692869671126 2251802375814769 EVENT VARIABLE CREATED 2251802375814765 
836304301 304 6888891257 6888891180 1692869671126 2251802375814765 COMMAND PROCESS_INSTANCE ACTIVATE_ELEMENT 2251802375814765 PROCESS 
836304301 304 6888891258 6888891180 1692869671126 2251802375814770 EVENT PROCESS_INSTANCE_CREATION CREATED 2251802375814765 
836304301 304 6888891259 6888891180 1692869671126 2251802375814765 EVENT PROCESS_INSTANCE ELEMENT_ACTIVATING 2251802375814765 PROCESS 
836304301 304 6888891260 6888891180 1692869671126 2251802375814765 EVENT PROCESS_INSTANCE ELEMENT_ACTIVATED 2251802375814765 PROCESS 
836304301 304 6888891261 6888891180 1692869671126 -1 COMMAND PROCESS_INSTANCE ACTIVATE_ELEMENT 2251802375814765 START_EVENT 

Deps

• build(deps-dev): bump org.testcontainers:junit-jupiter from 1.18.3 to 1.19.0 by @dependabot in #289
• build(deps): bump org.jetbrains.kotlinx:kotlinx-serialization-json-jvm from 1.5.1 to 1.6.0 by @dependabot in #290
• build(deps): bump kotlin.version from 1.9.0 to 1.9.10 by @dependabot in #291
• build(deps): bump version.picocli from 4.7.4 to 4.7.5 by @dependabot in #292
• build(deps-dev): bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 by @dependabot in #294
• build(deps): bump version.slf4j from 2.0.7 to 2.0.9 by @dependabot in #293
• build(deps): bump actions/checkout from 3 to 4 by @dependabot in #295
• build(deps): bump docker/login-action from 2.2.0 to 3.0.0 by @dependabot in #297
• build(deps): bump zeebe.version from 8.2.11 to 8.2.13 by @dependabot in #296

Full Changelog: 1.8.0...1.9.0

1.8.0

Release 1.8.0

The newest release comes with some improvements to the log print sub-command.

zdb now prints the log incrementally, which avoids OOM on larger logs. Furthermore, you can now limit the output of the printed log. You can specify from which position the log print should start and/or at which position it should stop. Additionally, you can define a new filter that limits the output to records that only correspond to a given process instance.

Read more about this in the corresponding readme section.

Details

Fixes

• Support incremental read of log by @Zelldon in #284
• Fix path position in state list subcommand by @Zelldon in #287

Features

• Filter log for position by @Zelldon in #285
• Filter log for process instance by @Zelldon in #286
• Move new log print logic to backend by @Zelldon in #288

What else has changed

• build(deps): bump zeebe.version from 8.2.8 to 8.2.9 by @dependabot in #280
• build(deps): bump zeebe.version from 8.2.9 to 8.2.10 by @dependabot in #281
• build(deps): bump zeebe.version from 8.2.10 to 8.2.11 by @dependabot in #282
• build(deps): bump actions/setup-java from 3.11.0 to 3.12.0 by @dependabot in #279
• build(deps-dev): bump org.junit.jupiter:junit-jupiter-engine from 5.9.3 to 5.10.0 by @dependabot in #278
• build(deps-dev): bump org.junit.jupiter:junit-jupiter-api from 5.9.3 to 5.10.0 by @dependabot in #277

Full Changelog: 1.7.0...1.8.0