Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass Result to NextState callback #39

Open
onepunchtech opened this issue Aug 1, 2018 · 4 comments
Open

Pass Result to NextState callback #39

onepunchtech opened this issue Aug 1, 2018 · 4 comments

Comments

@onepunchtech
Copy link

Sometimes the only way to know what the next state should be is if you let the system tell you. So it would be nice to be able to have the Result of Run in the NextState callback. Right now I am modifying the state in the post condition.

For example in a CRUD application you can know all of the parameters except for the unique id that gets generated by the system.

Concrete use case:

  • command: create user

    • next state: save user with id to model state.
    • post condition: ensure creation response conforms
  • command: get user

    • pre condition: at least one user must exist
    • post condition: ensure application response matches model state
@untoldwind
Copy link
Collaborator

untoldwind commented Aug 5, 2018

For stateful testing I used scala-check as template. In this case:
https://www.scalacheck.org/files/scalacheck_2.11-1.14.0-api/index.html#org.scalacheck.commands.Commands$Command

But that does not mean that it is set in stone.

So, if I understand correctly, you need the Result of Run as additional parameter to NextState so that your State-implementation can be immutable?

Or even more specific, the new Command-interface should look like this:

type Command interface {
	Run(systemUnderTest SystemUnderTest) Result
	NextState(state State, result Result) State
	PreCondition(state State) bool
	PostCondition(state State, result Result) *gopter.PropResult
	String() string
}

Alternatively (if just focusing on the immutability) PostCondition might return a modified state as well, like this:

type Command interface {
	Run(systemUnderTest SystemUnderTest) Result
	NextState(state State) State
	PreCondition(state State) bool
	PostCondition(state State, result Result) (State, *gopter.PropResult)
	String() string
}

Though I think the first variant is much cleaner

@onepunchtech
Copy link
Author

I used scalacheck a couple times, and ran into the same issue. I don't remember how exactly I got around it.

Here is a similar library in haskell, which in my opinion has a better api if you are interested. http://hackage.haskell.org/package/hedgehog-0.6/docs/Hedgehog-Internal-State.html#t:Callback. You can see there that the state, input, and result is passed to the update callback.

Back to the original point, yes, I think that the first one is cleaner, and regardless of keeping state immutable (which isn't a super important thing for me to do in go) I think that separating the concerns where my nextState callback is concerned with updating state, and my postCondition callback is only worried about making assertions.

@untoldwind
Copy link
Collaborator

untoldwind commented Aug 6, 2018

I experimented a bit and just realized, that NextState is pretty essential for the shrinker. More precisely: PreCondition and NextState are used to generate valid Command sequences without actually altering/invoking the SystemUnderTest. I suppose that's one of the reasons why scala-check is doing it this was.

Adding the result to NextState would mean that during the creation of the command sequences the SystemUnderTest would have to be resetted and applied on every try before the actual test even runs. That's not only a major redesign but potentially also a huge performance drawback.

Potentially it would make more sense to use the haskell API as template for an extra command package.

@untoldwind
Copy link
Collaborator

untoldwind commented Aug 6, 2018

The main trick of the haskell variant seems to be that they distinguish between a symbolic state and a concrete state, which is pretty nice when used with polymorphic functions.

A go-ish version might look like this (I'm dropping the command input's for now, that would just add even more complexity):

package commands2

// SystemUnderTest resembles the system under test, which may be any kind
// of stateful unit of code
type SystemUnderTest interface{}

// Symbol resembles a symbol in the state
type Symbol interface{}

// SymbolicState resembles the symbolic state (i.e. a state without concrete values)
type SymbolicState map[Symbol]bool

// ConcreteState resembles the concrete state (i.e. a state with concrete values)
type ConcreteState map[Symbol]interface{}

// Output resembles the Output of a command that may or may not be checked
type Output interface{}

// Command is any kind of command that may be applied to the system under test
type Command interface {
	// Execute the command
	Execute(systemUnderTest SystemUnderTest) Output

	Require(state SymbolicState) bool

	UpdateSymbolic(state SymbolicState) SymbolicState

	UpdateConcrete(state ConcreteState, output Output) ConcreteState

	Ensure(prev ConcreteState, next ConcreteState) bool
}

I.e. I declared that a State is always a collection/map of symbols with potential values. Since go not even remotely has such a thing like polymorphic functions (not that I'm aware of at least), the implementor has to provide an Update for the symbolic state and the concrete state.

In your original example it is pretty clear that a GetUser command always require a previous CreateUser command. So you have a symbol "userId" as part of the state. UpdateSymbolic for CreateUser would just add this symbol, while UpdateConcrete would add a concrete userId to it.

I think that something like this might solve your problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants