Project | Maven Central | Maven Central (JS) | Bintray |
---|---|---|---|
hedgehog-core | |||
hedgehog-runner | |||
hedgehog-sbt | |||
hedgehog-minitest |
Hedgehog will eat all your bugs.
Hedgehog is a modern property-based testing system, in the spirit of QuickCheck (and ScalaCheck). Hedgehog uses integrated shrinking, so shrinks obey the invariants of generated values by construction.
This project is still in some form of early release. The API may break during this stage until (if?) there is a wider adoption.
Please drop us a line if you start using scala-hedgehog in anger, we'd love to hear from you.
- Integrated shrinking, shrinks obey invariants by construction.
- Abstract state machine testing.
- Range combinators for full control over the scope of generated numbers and collections.
- SBT test runner
- Currently no external dependencies in the core module
In your build.sbt
you will unfortunately need to add a
custom resolver.
Hedgehog is released for every commit and so the "version" will be a git commit hash.
You can find the bintray repository here.
val hedgehogVersion = "${COMMIT}"
libraryDependencies ++= Seq(
"qa.hedgehog" %% "hedgehog-core" % hedgehogVersion,
"qa.hedgehog" %% "hedgehog-runner" % hedgehogVersion,
"qa.hedgehog" %% "hedgehog-sbt" % hedgehogVersion
)
resolvers += "bintray-scala-hedgehog" at "https://dl.bintray.com/hedgehogqa/scala-hedgehog"
This project can be added as an SBT subproject.
// This can also be a branch name, like 'master'`, if you want to live on the edge
val hedgehogVersion = "${COMMIT}"
val hedgehogUri = uri("https://github.com/hedgehogqa/scala-hedgehog.git#" + hedgehogVersion)
lazy val root =
(project in file("."))
.dependsOn(ProjectRef(hedgehogUri, "core"))
.dependsOn(ProjectRef(hedgehogUri, "runner"))
.dependsOn(ProjectRef(hedgehogUri, "sbt-test"))
NOTE: Depending on your scala version(s) SBT might not resolve.
Scala Hedgehog comes with a very primitive runner interface, and supports the SBT testing extension.
testFrameworks += TestFramework("hedgehog.sbt.Framework")
The IntelliJ scala plugin only has
hard-coded support for the most popular test frameworks.
While Hedgehog is obviously not included in that list, an may never be, by extending the runner
Properties
tests can be run as an application (as Properties
includes a handy main
function).
NOTE: This requires the test to be an object
and not a class
.
See the examples module for working versions.
import hedgehog._
import hedgehog.runner._
object PropertyTest extends Properties {
def tests: List[Test] =
List(
property("reverse", testReverse)
)
def testReverse: Property =
for {
xs <- Gen.alpha.list(Range.linear(0, 100)).forAll
} yield xs.reverse.reverse ==== xs
}
Scala Hedgehog provides an integration module for minitest. This allows you to define property-based and example-based Hedgehog tests within a minitest test suite. If you use this integration, you won't need to Scala Hedgehog sbt testing extension, because you're using the one provided by minitest:
val hedgehogVersion = "${COMMIT}"
libraryDependencies ++= "qa.hedgehog" %% "hedgehog-minitest" % hedgehogVersion
resolvers += "bintray-scala-hedgehog" at "https://dl.bintray.com/hedgehogqa/scala-hedgehog"
testFrameworks += new TestFramework("minitest.runner.Framework")
Here's an example of using hedgehog-minitest
:
import minitest.SimpleTestSuite
import hedgehog.minitest.HedgehogSupport
import hedgehog._
object ReverseTest extends SimpleTestSuite with HedgehogSupport {
property("reverse alphabetic strings") {
for {
xs <- Gen.alpha.list(Range.linear(0, 100)).forAll
} yield xs.reverse.reverse ==== xs
}
example("reverse hello") {
"hello".reverse ==== "olleh"
}
}
The background and motivation for Hedgehog in general is still best described by the original author in this excellent presenation:
A very quick summary is that the original QuickCheck and it's derivatives (like ScalaCheck) separate the generation of data from the shrinking, which results in something that cannot be composed easily. It turns out it's fairly simple to combine them in a single data-type.
If you've used ScalaCheck before, it's exactly the same as writing your normal Gen
functions,
but now those generated value will shrink without any extra information. Magic!
As a general rule, the current Scala API is intended to be direct port of haskell-hedgehog, much like scalacheck was for QuickCheck. The idea being that people familiar with one of the libraries will be comfortable with the other. It also makes it easier not having to re-invent any wheels (or APIs). There will obviously be exceptions where Scala forces us to make a different trade-off. See haskell-differences for examples and more explanation.
Fortunately there is much in common across property-testing material, and as such the following are still relevant despite whatever language they are presented with.
- Choosing properties for property-based testing (sharpforfunandprofit.com)
- An introduction to property-based testing (sharpforfunandprofit.com)
- Integrated versus Manual Shrinking (well-typed.com)
- Property-Based Testing in a Screencast Editor (wickstrom.tech)
- Much Ado About Testing - Nicolas Rinaudo at ScalaWorld 2019
- Introduction to stateful property based testing - Tomasz Kowal at Lambda Days 2019
- Building on Developers' Intuitions to Create Effective Property-Based Tests - John Hughes at Lambda Days 2019
- Property-Based Testing The Ugly Parts: Case Studies from Komposition - Oskar Wickström at flatMap (Oslo) 2019
- Types vs Tests - Julien Truffaut at Scala eXchange 2018
- Appetite for dysfunction - Andrew McCluskey at Compose Melbourne 2018
- Property-based State Machine Testing - Andrew McCluskey at YOW! Lambda Jam 2018.
- Gens N’ Roses: Appetite for Reduction - Jacob Stanley at YOW! Lambda Jam 2017.
- Find More Bugs with Less Effort - Charles O'Farrell at YOW! Night Singapore 2017.
- Property-based Testing in Practice - Alex Chan at QCon 2017.
- Functions and Determinism in Property-based Testing - Erik Osheim at Philly ETE 2017
- Practical Property-Based Testing - Charles O’Farrell at YOW! Lambda Jam 2015.
- Property-Based Testing for Better Code - Jessica Kerr at Midwest.io 2014.
- I Dream of Gen’ning: ScalaCheck Beyond the Basics - Kelsey Gilmore-Innis at Scala By The Bay 2014.
- Testing the Hard Stuff and Staying Sane - John Hughes on property-based testing using Quviq QuickCheck.
- Testing Stateful Systems with ScalaCheck - Rickard Nilsson at ScalaDays 2014.
- Property-Based Testing in a Screencast Editor by Oskar Wickström (2019)
- Property-Based Testing with PropEr, Erlang, and Elixir by Fred Hebert (2018)
- ScalaCheck: The Definitive Guide by Rickard Nilsson (2014)
In Scala there are other property-testing alternatives:
-
https://github.com/rickynils/scalacheck
The original port of QuickCheck in Scala.
-
https://zio.dev/docs/usecases/usecases_testing
A testing library build on ZIO that includes integrated shrinking.
-
https://github.com/melrief/sonic
This is another port of Hedgehog, based on cats and monix.
-
https://github.com/scalaprops/scalaprops
Makes some improvements on the ScalaCheck implementation.
-
https://github.com/japgolly/nyaya
Another fast data generator and property testing library in Scala.