Release 4.0.0 (#1492)

* Add UUIDModule, JavaDateTimeModule and RecordModule to support Java 17. Since UUID fields, Date/DateTime fields are final baker cannot instantiate them in the pojo way. * Remove java.util.Date * Disabled the scala 2.12 compilation. * Disabled the AkkaCassandraJmxMetricsSpec since it does not work with Java 17. * Enabled the getIngredient endpoint for Bakery * Made the java.util.Duration backwards compatible for Java versions. * Disabled incompatible modules/plugins for Scala 2.13. * Added support for the OffSetDateTime, ZonedDateTime & java.util.Date * Set target back to Java 11 to ensure it does not break for java 11. * Added missing function to Kotlin interface. * Improve Kotlin DSL * Added the getJavaBaker method to the kotlinDSL to ensure it can be used for frameworks that need a normal Baker. * Added an optional rememberProcessDuration that forgets a process after the configured duration. This keeps the ProcessIndex shards from growing indefinitely. * Fix to the ProcessDeleted not being handled during Bake. * Made the lastupdate be set correctly after the RecipeInstance was used. (#1536) * Reprovider functionality * Fixed the cleanup for non Cassandra datastores. We use the original cleanup of the data as before our upgrade to the Cassandra cleanup tool for other datastores. (#1551) * Wrote a type modules for the java.util.Currency. * Added a section on Recipe Retention period to the docs (#1575) * Remove generics for Place and Transition from PetriNet and IL (#1582) * Fixed output ingredient rename example/doc for Java (#1574) * Fixed output ingredient rename example/doc for Java * fixed forgotten import * Added some missing imports (#1583) * Fixed the Wait interaction to work if you also have a retry strategy that fires and event. (#1585) * Fixed a bug in the getOutputEventName where it would not work if the FailureStrategy would also throw an event. * Added RecipeId and RecipeName to log lines (#1589) * Set the default retention period to 14 days for deletion if the recipe is not found. * Changed the behaviour of the FireEvent retry strategy. It now blocks the interaction instead of resolving it. This allows the interaction to be retried with the retyInteraction method. * RecipeInstanceMetadata and RecipeInstanceEventList as ingredient input for interactions (#1643) * Added extra logging if a recipe is not found during deletion of a process. * Set the default retention period to 14 days for deletion if the recipe is not found. * Changed the behaviour of the FireEvent retry strategy. It now blocks the interaction instead of resolving it. This allows the interaction to be retried with the retyInteraction method. * Added the RecipeInstanceMetadata as input ingredient. * Added a new Baker Event to represent internal interaction output events, checkpoint events and delayed transition events. * Added the RecipeInstanceEventList as a input ingredient. * Made the RecipeInstanceMetaData an internal concept instead of putting it in the ingredients and needing to transform it every time. * Added validations that the Baker internal ingredients are not provided in the recipe. * Filter out the CheckPoint event interactions from the visualisation --------- Co-authored-by: yl02pf <[email protected]> Co-authored-by: Wim Leuning <[email protected]> Co-authored-by: Bas <[email protected]> Co-authored-by: Willem Veelenturf <[email protected]> Co-authored-by: Bas de Groot <[email protected]> Co-authored-by: Elmar Wachtmeester <[email protected]> Co-authored-by: Elmar Wachtmeester <[email protected]>
ing-bank · Mar 19, 2024 · 63f90bf · 63f90bf
1 parent ef2c0e6
commit 63f90bf
Show file tree

Hide file tree

Showing 309 changed files with 7,308 additions and 4,327 deletions.
diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml
@@ -13,11 +13,11 @@ jobs:
 
     steps:
     - uses: actions/checkout@v2
-    - name: Set up JDK 11
+    - name: Set up JDK 17
       uses: actions/setup-java@v2
       with:
         distribution: adopt
-        java-version: 11
+        java-version: 17
 
     - name: Dependency check
       run: sbt test:compile updateClassifiers

diff --git a/README.md b/README.md
@@ -25,8 +25,8 @@ Use the list below to learn more about Baker:
 
 - [Documentation](https://ing-bank.github.io/baker/)
     - [Concepts](https://ing-bank.github.io/baker/sections/concepts/): high level concepts and terminology
-    - [Recipe DSL](https://ing-bank.github.io/baker/sections/reference/dsls/): how to use the recipe DSL
-    - [Visualization](https://ing-bank.github.io/baker/sections/reference/visualization/): how to visualize a recipe
+    - [Recipe DSL](https://ing-bank.github.io/baker/sections/cookbook/recipe-dsl/): how to use the recipe DSL
+    - [Visualization](https://ing-bank.github.io/baker/sections/cookbook/visualizations/): how to visualize a recipe
 - [Baker Talk @ J-Fall 2021](https://www.youtube.com/watch?v=U4aCUT9zIFk): API orchestration taken to the next level
 
 ## A bird's-eye view of Baker

diff --git a/azure-build.yaml b/azure-build.yaml
@@ -67,7 +67,7 @@ steps:
 
   - task: JavaToolInstaller@0
     inputs:
-      versionSpec: '11'
+      versionSpec: '17'
       jdkArchitectureOption: 'x64'
       jdkSourceOption: 'PreInstalled'
       cleanDestinationDirectory: false

diff --git a/bakery/state/src/main/scala/com/ing/bakery/AkkaBakery.scala b/bakery/state/src/main/scala/com/ing/bakery/AkkaBakery.scala
@@ -9,6 +9,7 @@ import com.ing.baker.runtime.model.InteractionManager
 import com.ing.baker.runtime.recipe_manager.RecipeManager
 import com.ing.baker.runtime.scaladsl.Baker
 import com.ing.bakery.components.AkkaBakeryComponents
+import com.ing.bakery.metrics.MetricService
 import com.typesafe.config.Config
 import com.typesafe.scalalogging.LazyLogging
 import io.prometheus.client.CollectorRegistry
@@ -78,10 +79,10 @@ object Bakery {
                  externalContext: Option[Any] = None,
                  interactionManager: Option[InteractionManager[IO]] = None,
                  recipeManager: Option[RecipeManager] = None,
-                 registry: CollectorRegistry = CollectorRegistry.defaultRegistry): Resource[IO, AkkaBakery] = {
+                 metricService: MetricService = new MetricService(CollectorRegistry.defaultRegistry)): Resource[IO, AkkaBakery] = {
 
     val akkaBakeryComponents: AkkaBakeryComponents =
-      new AkkaBakeryComponents(optionalConfig, externalContext, registry) {
+      new AkkaBakeryComponents(optionalConfig, externalContext, metricService) {
 
         override def interactionManagerResource(config: Config,
                                                 actorSystem: ActorSystem,

diff --git a/bakery/state/src/main/scala/com/ing/bakery/components/AkkaBakeryComponents.scala b/bakery/state/src/main/scala/com/ing/bakery/components/AkkaBakeryComponents.scala
@@ -25,11 +25,9 @@ import scala.concurrent.ExecutionContext
   */
 class AkkaBakeryComponents(optionalConfig: Option[Config] = None,
                            externalContext: Option[Any] = None,
-                           registry: CollectorRegistry = CollectorRegistry.defaultRegistry
+                           metricService: MetricService = new MetricService(CollectorRegistry.defaultRegistry)
                           ) extends LazyLogging {
 
-  val metricService: MetricService = new MetricService(registry)
-
   def configResource: Resource[IO, Config] = Resource.eval(IO {
     val configPath = sys.env.getOrElse("CONFIG_DIRECTORY", "/opt/docker/conf")
     val config = optionalConfig.getOrElse(ConfigFactory.load(ConfigFactory.parseFile(new File(s"$configPath/application.conf"))))

diff --git a/bakery/state/src/test/scala/com/ing/bakery/components/AkkaCassandraJmxMetricsSpec.scala b/bakery/state/src/test/scala/com/ing/bakery/components/AkkaCassandraJmxMetricsSpec.scala
@@ -13,7 +13,7 @@ import com.typesafe.scalalogging.LazyLogging
 import io.prometheus.client.CollectorRegistry
 import io.prometheus.client.exporter.common.TextFormat
 import org.cassandraunit.utils.EmbeddedCassandraServerHelper._
-import org.scalatest.BeforeAndAfterAll
+import org.scalatest.{BeforeAndAfterAll, Ignore}
 import org.scalatest.concurrent.Eventually
 import org.scalatest.freespec.AnyFreeSpec
 import org.scalatest.time.{Millis, Seconds, Span}
@@ -28,7 +28,13 @@ import scala.collection.convert.ImplicitConversions._
 import scala.concurrent.duration._
 import scala.concurrent.{Await, ExecutionContext}
 
+
+/**
+  * Test is ignored since CassandraUnit is not supported in Java 17.
+  * We can enable this once this is supported again.
+  */
 @nowarn
+@Ignore
 class AkkaCassandraJmxMetricsSpec extends AnyFreeSpec with LazyLogging with Eventually with BeforeAndAfterAll {
 
   import TestActors._

diff --git a/build.sbt b/build.sbt
@@ -20,7 +20,7 @@ lazy val baker: Project = project.in(file("."))
     `bakery-interaction-k8s-interaction-manager`,
     // Examples
     `baker-example`, `bakery-client-example`, `interaction-example-make-payment-and-ship-items`,
-    `interaction-example-reserve-items`, `bakery-kafka-listener-example`
+    `interaction-example-reserve-items`, `bakery-kafka-listener-example`, `docs-code-snippets`
   )
 
 def testScope(project: ProjectReference): ClasspathDep[ProjectReference] = project % "test->test;test->compile"
@@ -39,17 +39,17 @@ lazy val buildExampleDockerCommand: Command = Command.command("buildExampleDocke
       state
 })
 
-lazy val scala212 = "2.12.16"
+//lazy val scala212 = "2.12.16"
 lazy val scala213 = "2.13.8"
 
-lazy val supportedScalaVersions = List(scala213, scala212)
+lazy val supportedScalaVersions = List(scala213)
 val commonSettings: Seq[Setting[_]] = Defaults.coreDefaultSettings ++ Seq(
   organization := "com.ing.baker",
   fork := true,
   testOptions += Tests.Argument(TestFrameworks.JUnit, "-v"),
-  javacOptions := Seq("-source", "1.8", "-target", "1.8"),
+  javacOptions := Seq("-source", "17", "-target", "17"),
   scalacOptions := Seq(
-    s"-target:jvm-1.8",
+    s"-target:jvm-17",
     "-unchecked",
     "-deprecation",
     "-feature",
@@ -175,12 +175,7 @@ lazy val `baker-interface`: Project = project.in(file("core/baker-interface"))
       catsEffect,
       fs2Core,
       fs2Io,
-      CrossVersion.partialVersion(scalaVersion.value) match {
-        case Some((2, n)) if n <= 12 =>
-          scalaJava8Compat091
-        case _ =>
-          scalaJava8Compat100
-      },
+      scalaJava8Compat100,
       javaxInject,
       guava
     ) ++ providedDeps(findbugs) ++ testDeps(
@@ -202,8 +197,8 @@ lazy val `baker-interface-kotlin`: Project = project.in(file("core/baker-interfa
   .settings(Publish.settings)
   .settings(
     moduleName := "baker-interface-kotlin",
-    kotlinVersion := "1.7.22",
-    kotlincJvmTarget := "1.8",
+    kotlinVersion := "1.8.21",
+    kotlincJvmTarget := "17",
     kotlinLib("stdlib-jdk8"),
     kotlinLib("reflect"),
     libraryDependencies ++=
@@ -315,8 +310,8 @@ lazy val `baker-recipe-dsl-kotlin`: Project = project.in(file("core/recipe-dsl-k
   .settings(Publish.settings)
   .settings(
     moduleName := "baker-recipe-dsl-kotlin",
-    kotlinVersion := "1.7.22",
-    kotlincJvmTarget := "1.8",
+    kotlinVersion := "1.8.21",
+    kotlincJvmTarget := "17",
     kotlinLib("stdlib-jdk8"),
     kotlinLib("reflect"),
     libraryDependencies ++=
@@ -340,8 +335,8 @@ lazy val `baker-recipe-compiler`: Project = project.in(file("core/recipe-compile
   .settings(Publish.settings)
   .settings(
     moduleName := "baker-compiler",
-    kotlinVersion := "1.7.22",
-    kotlincJvmTarget := "1.8",
+    kotlinVersion := "1.8.21",
+    kotlincJvmTarget := "17",
     libraryDependencies ++=
       testDeps(scalaTest, scalaCheck, junitJupiter)
   )
@@ -660,27 +655,6 @@ lazy val `bakery-integration-tests`: Project = project.in(file("bakery/integrati
     `interaction-example-make-payment-and-ship-items`,
     `interaction-example-reserve-items`)
 
-lazy val `sbt-bakery-docker-generate`: Project = project.in(file("docker/sbt-bakery-docker-generate"))
-  .settings(scalaVersion := scala212, crossScalaVersions := Nil)
-  .settings(defaultModuleSettings212)
-  .settings(noPublishSettings) // docker plugin can't be published, at least not to azure feed
-  .settings(
-    crossScalaVersions := Nil,
-    // workaround to let plugin be used in the same project without publishing it
-    Compile / sourceGenerators += Def.task {
-      val file = (Compile / sourceManaged).value / "bakery" / "sbt" / "BuildInteractionDockerImageSBTPlugin.scala"
-      val sourceFile = IO.readBytes(baseDirectory.value.getParentFile.getParentFile / "project" / "BuildInteractionDockerImageSBTPlugin.scala")
-      IO.write(file, sourceFile)
-      Seq(file)
-    }.taskValue,
-    addSbtPlugin(("com.github.sbt" % "sbt-native-packager" % "1.9.9") cross CrossVersion.constant(scala212)),
-    addSbtPlugin(("org.vaslabs.kube" % "sbt-kubeyml" % "0.4.0") cross CrossVersion.constant(scala212))
-  )
-  .enablePlugins(SbtPlugin)
-  .enablePlugins(bakery.sbt.BuildInteractionDockerImageSBTPlugin)
-  .dependsOn(`bakery-interaction`, `bakery-interaction-spring`)
-
-
 lazy val `baker-example`: Project = project
   .in(file("examples/baker-example"))
   .enablePlugins(JavaAppPackaging)
@@ -721,6 +695,44 @@ lazy val `baker-example`: Project = project
   )
   .dependsOn(`baker-types`, `baker-akka-runtime`, `baker-recipe-compiler`, `baker-recipe-dsl`, `baker-intermediate-language`)
 
+lazy val `docs-code-snippets`: Project = project
+  .in(file("examples/docs-code-snippets"))
+  .enablePlugins(JavaAppPackaging)
+  .settings(commonSettings)
+  .settings(noPublishSettings)
+  .settings(yPartialUnificationSetting)
+  .settings(crossBuildSettings)
+  .settings(
+    moduleName := "docs-code-snippets",
+    kotlinVersion := "1.8.21",
+    kotlincJvmTarget := "17",
+    kotlinLib("stdlib-jdk8"),
+    kotlinLib("reflect"),
+    libraryDependencies ++=
+      compileDeps(
+        slf4jApi,
+        http4s,
+        http4sDsl,
+        http4sServer,
+        http4sCirce,
+        circe,
+        circeGeneric,
+        akkaPersistenceCassandra,
+        akkaPersistenceQuery
+      ) ++ testDeps(
+        scalaTest,
+        scalaCheck,
+        mockitoScala,
+        junitInterface,
+        slf4jApi,
+        akkaTestKit
+      )
+  )
+  .settings(
+    coverageEnabled := false
+  )
+  .dependsOn(`baker-types`, `baker-akka-runtime`, `baker-recipe-compiler`, `baker-recipe-dsl-kotlin`, `baker-intermediate-language`, `baker-interface-kotlin`)
+
 lazy val `bakery-client-example`: Project = project
   .in(file("examples/bakery-client-example"))
   .enablePlugins(JavaAppPackaging)
@@ -782,7 +794,6 @@ lazy val `bakery-kafka-listener-example`: Project = project
 
 lazy val `interaction-example-reserve-items`: Project = project.in(file("examples/bakery-interaction-examples/reserve-items"))
   .enablePlugins(JavaAppPackaging)
-  .enablePlugins(bakery.sbt.BuildInteractionDockerImageSBTPlugin)
   .settings(noPublishSettings)
   .settings(defaultModuleSettings)
   .settings(yPartialUnificationSetting)
@@ -804,7 +815,6 @@ lazy val `interaction-example-reserve-items`: Project = project.in(file("example
 
 lazy val `interaction-example-make-payment-and-ship-items`: Project = project.in(file("examples/bakery-interaction-examples/make-payment-and-ship-items"))
   .enablePlugins(JavaAppPackaging)
-  .enablePlugins(bakery.sbt.BuildInteractionDockerImageSBTPlugin)
   .settings(noPublishSettings)
   .settings(defaultModuleSettings)
   .settings(yPartialUnificationSetting)

diff --git a/core/akka-runtime/src/main/protobuf/delayed_transition.proto b/core/akka-runtime/src/main/protobuf/delayed_transition.proto
@@ -14,7 +14,7 @@ message DelayedTransitionInstance {
   optional int64 jobId = 3;
   optional int64 transitionId = 4;
   optional string eventToFire = 5;
-  optional bool fired = 6;
+  optional bool fired = 6; //REMOVED
 }
 
 message DelayedTransitionScheduled {
@@ -25,4 +25,8 @@ message DelayedTransitionScheduled {
 message DelayedTransitionExecuted {
   optional string id = 1;
   optional DelayedTransitionInstance delayedTransitionInstance = 2;
+}
+
+message DelayedTransitionSnapshot {
+  map<string, DelayedTransitionInstance> waitingTransitions = 1;
 }
diff --git a/core/akka-runtime/src/main/protobuf/process_instance.proto b/core/akka-runtime/src/main/protobuf/process_instance.proto
@@ -49,6 +49,19 @@ message TransitionFired {
     optional SerializedData data = 8;
 }
 
+message TransitionFailedWithOutput {
+    option (scalapb.message).extends = "com.ing.baker.runtime.akka.actor.serialization.BakerSerializable";
+
+    optional int64 job_id = 1;
+    optional string correlation_id = 9;
+    optional int64 transition_id = 3;
+    optional int64 time_started = 4;
+    optional int64 time_completed = 5;
+    repeated ConsumedToken consumed = 6;
+    repeated ProducedToken produced = 7;
+    optional SerializedData data = 8;
+}
+
 message TransitionDelayed {
     option (scalapb.message).extends = "com.ing.baker.runtime.akka.actor.serialization.BakerSerializable";
     optional int64 job_id = 1;

diff --git a/core/akka-runtime/src/main/resources/reference.conf b/core/akka-runtime/src/main/resources/reference.conf
@@ -13,8 +13,11 @@ baker {
     # The interval that a check is done of processes should be deleted
     retention-check-interval = 1 minutes
 
-    # snapShotInterval
+    # The interval of messages between creating snapshots for the actors
     snapshot-interval = 100
+
+    # The amount of snapshots to keep for the actors
+    snapshot-count = 3
   }
 
   # the default timeout for Baker.bake(..) process creation calls
@@ -47,6 +50,7 @@ baker {
     restart-maxBackoff = 30 seconds
     restart-randomFactor = 0.2
     start-all-shards = true
+    remember-process-duration = null
   }
 
   process-instance {

diff --git a/core/akka-runtime/src/main/scala/com/ing/baker/runtime/akka/AkkaBaker.scala b/core/akka-runtime/src/main/scala/com/ing/baker/runtime/akka/AkkaBaker.scala
@@ -14,7 +14,9 @@ import com.ing.baker.runtime.akka.actor.recipe_manager.RecipeManagerProtocol
 import com.ing.baker.runtime.akka.actor.recipe_manager.RecipeManagerProtocol.RecipeFound
 import com.ing.baker.runtime.akka.internal.CachingInteractionManager
 import com.ing.baker.runtime.common.BakerException._
+import com.ing.baker.runtime.common.RecipeInstanceState.RecipeInstanceMetadataName
 import com.ing.baker.runtime.common.{InteractionExecutionFailureReason, RecipeRecord, SensoryEventStatus}
+import com.ing.baker.runtime.model.recipeinstance.RecipeInstanceState.getMetaDataFromIngredients
 import com.ing.baker.runtime.recipe_manager.{ActorBasedRecipeManager, DefaultRecipeManager, RecipeManager}
 import com.ing.baker.runtime.scaladsl._
 import com.ing.baker.runtime.{javadsl, scaladsl}
@@ -181,7 +183,9 @@ class AkkaBaker private[runtime](config: AkkaBakerConfig) extends scaladsl.Baker
         case None => cats.effect.IO.pure(InteractionExecutionResult(Left(InteractionExecutionResult.Failure(
           InteractionExecutionFailureReason.INTERACTION_NOT_FOUND, None, None))))
         case Some(interactionInstance) =>
-          interactionInstance.execute(ingredients, Map())
+          interactionInstance.execute(
+              ingredients.filter(ingredientInstance => ingredientInstance.name != RecipeInstanceMetadataName),
+              getMetaDataFromIngredients(ingredients).getOrElse(Map.empty))
             .map(executionSuccess => InteractionExecutionResult(Right(InteractionExecutionResult.Success(executionSuccess))))
             .recover {
               case e => InteractionExecutionResult(Left(InteractionExecutionResult.Failure(
@@ -208,6 +212,8 @@ class AkkaBaker private[runtime](config: AkkaBakerConfig) extends scaladsl.Baker
     processIndexActor.ask(CreateProcess(recipeId, recipeInstanceId))(config.timeouts.defaultBakeTimeout).javaTimeoutToBakerTimeout("bake").flatMap {
       case _: Initialized =>
         Future.successful(())
+      case ProcessDeleted =>
+        Future.failed(ProcessDeletedException(recipeInstanceId))
       case ProcessAlreadyExists(_) =>
         Future.failed(ProcessAlreadyExistsException(recipeInstanceId))
       case RecipeManagerProtocol.NoRecipeFound(_) =>
@@ -423,22 +429,22 @@ class AkkaBaker private[runtime](config: AkkaBakerConfig) extends scaladsl.Baker
         case ProcessDeleted(id) => Future.failed(ProcessDeletedException(id))
       }
 
-//  /**
-//    * @param recipeInstanceId The recipeInstance Id.
-//    * @param name The name of the ingredient.
-//    *  @return The provided ingredients.
-//    */
-//  override def getIngredient(recipeInstanceId: String, name: String): Future[Value] =
-//    processIndexActor
-//      .ask(GetProcessIngredient(recipeInstanceId, name))(Timeout.durationToTimeout(config.timeouts.defaultInquireTimeout))
-//      .javaTimeoutToBakerTimeout("getRecipeInstanceState")
-//      .flatMap {
-//        case ingredientFound: IngredientFound => Future.successful(ingredientFound.value)
-//        case IngredientNotFound  => Future.failed(NoSuchIngredientException(name))
-//        case Uninitialized(id) => Future.failed(NoSuchProcessException(id))
-//        case NoSuchProcess(id) => Future.failed(NoSuchProcessException(id))
-//        case ProcessDeleted(id) => Future.failed(ProcessDeletedException(id))
-//      }
+  /**
+    * @param recipeInstanceId The recipeInstance Id.
+    * @param name The name of the ingredient.
+    *  @return The provided ingredients.
+    */
+  override def getIngredient(recipeInstanceId: String, name: String): Future[Value] =
+    processIndexActor
+      .ask(GetProcessIngredient(recipeInstanceId, name))(Timeout.durationToTimeout(config.timeouts.defaultInquireTimeout))
+      .javaTimeoutToBakerTimeout("getRecipeInstanceState")
+      .flatMap {
+        case ingredientFound: IngredientFound => Future.successful(ingredientFound.value)
+        case IngredientNotFound  => Future.failed(NoSuchIngredientException(name))
+        case Uninitialized(id) => Future.failed(NoSuchProcessException(id))
+        case NoSuchProcess(id) => Future.failed(NoSuchProcessException(id))
+        case ProcessDeleted(id) => Future.failed(ProcessDeletedException(id))
+      }
 
   /**
     * Returns all provided ingredients for a given process id.