Skip to content

2.0.0
Compare
Choose a tag to compare
@Zelldon Zelldon released this 25 Oct 07:40
· 222 commits to main since this release
2.0.0
3593397
This commit was signed with the committer’s verified signature.
Zelldon Christopher Kujawa (Zell)
GPG key ID: 347E28B45AAF9D2D
Learn about vigilant mode.

Refactoring of ZDB

Key facts

  • Dependencies were reduced/removed
    • no longer depending on engine/state classes to read state, allows to handle newer versions better
    • reduces the size of the end artifact by ~30% (from 90 to 60 MB).
  • Several bug fixes and better error handling
    • especially for unknown values (e.g. column families)
  • Large improvement in test coverage (by 353% from 45 to 159 tests)
    • especially IT tests with different versions
    • allows support for different Zeebe versions, 8.1, 8.2, 8.3
  • Experimental state feature moved to stable
  • Remove unused features
    • banned instances sub-command, can be also investigated with state command
    • general status, was replaced with the state command

Details

State

The previously introduced experimental feature, where we simply open a read-only RocksDB instance and read the data without state classes, and the ZeebeDB module was upgraded to a stable state. It replaced all previous usage of state and engine classes.

This means now, instead of using engine-related classes to deserialize data we use simply msgpack to json conversation. This allows to remove the dependency to such modules and classes and to better handle newer versions, where classes or data formats are not known. The assumption is that all values are always msgpack encoded (which is right now the case in Zeebe).

Data is read via visitor patters, this means we no longer read all data into memory before printing to system out.

Log

The log classes have been updated to the recent changes, which means the versions checks are removed again. Mostly copied from Zeebe journal code.

Remove several Zeebe interface usages to avoid incompatibles and breaking changes on newer versions

Tests

Several tests have been adjusted and for different versions they are repeated now. We have now a test suite that runs tests against the following versions: 8.1, 8.2, 8.3, and SNAPSHOT.

SNAPSHOT should allow us to easily detect when something breaking is introduced. The other versions allow us to verify that ZDB is working against such versions, which wasn't the case before.

There were several new edge cases verified and tested, like what if a different format is used in the values (but still msgpack), what if the column family is not known (not corresponding to the current version) etc.

Make tests reproducible and predictable on all versions, via enabling WAL and enabling batch processing, etc.

Related issues

PR's merged

  • build(deps): bump actions/setup-java from 3.12.0 to 3.13.0 by @dependabot in #299
  • build(deps): bump com.google.cloud.tools:jib-maven-plugin from 3.3.2 to 3.4.0 by @dependabot in #300
  • build(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.20 to 2.21.1 by @dependabot in #301
  • build(deps): bump zeebe.version from 8.2.13 to 8.2.14 by @dependabot in #303
  • build(deps): bump zeebe.version from 8.2.14 to 8.2.15 by @dependabot in #304
  • build(deps-dev): bump org.testcontainers:junit-jupiter from 1.19.0 to 1.19.1 by @dependabot in #306
  • build(deps): bump com.mycila:license-maven-plugin from 4.2 to 4.3 by @dependabot in #305
  • build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by @dependabot in #312
  • Refactoring of ZDB by @Zelldon in #311
  • build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.0 to 3.3.1 by @dependabot in #313
  • build(deps): bump zeebe.version from 8.3.0 to 8.3.1 by @dependabot in #314
  • build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by @dependabot in #315

Full Changelog: 1.9.0...2.0.0

Contributors

Zelldon and dependabot
Assets 4
Loading