Should actions without conditions be supported?

[mds1]

Bulloak looks really nice and I'm excited to try it out. I have a function like this OZ function:

function _efficientHash(bytes32 a, bytes32 b) internal pure returns (bytes32 hash) {
    assembly ("memory-safe") {
        mstore(0x00, a)
        mstore(0x20, b)
        hash := keccak256(0x00, 0x40)
    }
}

There's no branching or conditionals here so I expected my .tree file to look very simple:

MyContract._efficientHash.sol
├── it should match the output of a high-level hash
└── it should never revert

But running bulloak against this errors with:

•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
bulloak error: found an action without conditions

├── it should match the output of a high-level hash
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

--- (line 2, column 1) ---•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
bulloak error: found an action without conditions

└── it should never revert
^^^^^^^^^^^^^^^^^^^^^^^^^^

I could add a "when called" condition, but this would add unnecessary clutter to the tree and tests since there's no conditions or paths in this method.

Should bulloak/BTT support specs of this style, or should I be doing things differently here? I think a good approach might be to only allow this IF all requirements (it should lines) are at the shallowest level and not preceded by any given's or when's.

Related, I'd like to see a bit more info in the README on the requirements around .tree files, e.g. when are when and given required, can you only have one of the two, can their order be flipped or must it be when > given, etc. The reason I ask here is I want to mirror the given > when > then flow of cucumber vs. the when > given > it should of BTT (perhaps both can be supported?)

cc @PaulRBerg for thoughts too

[alexfertel]

Hello! Thank you for the kind words & am excited that you find bulloak exciting!

I could add a "when called" condition, but this would add unnecessary clutter to the tree and tests since there's no conditions or paths in this method.

Yeah, that makes sense, I didn't think of this use case.

Should bulloak/BTT support specs of this style, or should I be doing things differently here?

I think this is a totally reasonable use case and that the path forward should be for bulloak to be a bit more flexible.

I think a good approach might be to only allow this IF all requirements (it should lines) are at the shallowest level and not preceded by any given's or when's.

Yes, that makes perfect sense 👍🏻 One consideration here is that since there is no root when/given, there is no test name to emit, so I'll just default to test_Root or test_RevertRoot. If you think there is a more sensible default lmk.

On an unrelated note, the errors look wonky, I should fix that and add a test for it. So, thank you for bringing that to my attention hehe.

Related, I'd like to see a bit more info in the README on the requirements around .tree files, e.g. when are when and given required, can you only have one of the two, can their order be flipped or must it be when > given, etc. The reason I ask here is I want to mirror the given > when > then flow of cucumber vs. the when > given > it should of BTT (perhaps both can be supported?)

I tried to refrain from adding opinionated docs because I wanted to see how people might start using BTT first and then iterate towards a good implementation. However, it's true that I should at least explain what is currently supported 👍🏻 when and given are interchangeable right now, and there is no restriction on their use. I would love to hear @PaulRBerg's thoughts.

Thanks again!

[mds1]

Yes, that makes perfect sense 👍🏻 One consideration here is that since there is no root when/given, there is no test name to emit, so I'll just default to test_Root or test_RevertRoot. If you think there is a more sensible default lmk.

Ah good point, I'd probably go with test_Requirements or something like that.

I tried to refrain from adding opinionated docs because I wanted to see how people might start using BTT first and then iterate towards a good implementation. However, it's true that I should at least explain what is currently supported 👍🏻 when and given are interchangeable right now, and there is no restriction on their use.

Yea this is fair. For some context, I had worked on scopelint spec which is a very simplified version of a similar goal (it generates specs from test names/architecture). But I think the opposite approach of generating test names/arch from the spec is stronger, and bulloak is a great start to getting there. Eventually I'd love for it to be opinionated so we (the eth dev community) can have a standardized approach for having specs for tests :)

[PaulRBerg]

Thanks for your questions @mds1. As ever, great feedback.

First, I want to clarify that BTT is still very much a WIP (conceptually). For instance, even in Sablier, we've recently worked on a major refactor to use "given" instead of "when": sablier-labs/v2-core#642

Should bulloak/BTT support specs of this style

It definitely should - test_Requirements where "requirement" is what follows after "it should" sounds like a fine approach.

I'd like to see a bit more info in the README on the requirements around .tree files

I agree that this would be helpful.

Ideally, we would put together a bespoke docs website. But I just can never manage to find the time to do that, so adding a few simple guidelines in the Bulloak README sounds like a good first step in the right direciton.

when are when and given required

The categories are fairly fuzzy but the rule of thumb is this:

• Is it a contract state that is prepared in advance? It's a given
• Is it a mode of execution, e.g. call vs delegatecall? it's a when
• Is it a function parameter that the user is in control when calling the function? it's a when

can their order be flipped

They can. The goal is to follow the revert paths of the function, which may or may not be dependent on function params.

The reason I ask here is I want to mirror the given > when > then flow of cucumber

BTT is inspired by Cucumber, but it is not Cucumber, and it does not purport to be or evolve into Cucumber.

[mds1]

The categories are fairly fuzzy but the rule of thumb is this:

• Is it a contract state that is prepared in advance? It's a given
• Is it a mode of execution, e.g. call vs delegatecall? it's a when
• Is it a function parameter that the user is in control when calling the function? it's a when

BTT is inspired by Cucumber, but it is not Cucumber, and it does not purport to be or evolve into Cucumber.

This all sounds reasonable to me. FWIW it does align with the gherkin definitions of Given and When (and the it maps to Then).

Even though BTT is not cucumber (we have no mapping from specs to executable code blocks), the BTT syntax is very similar to the gherkin language used in scenario files, to the point where we could and IMO should re-use those definitions/conventions. This would save us from having to reinvent the wheel here since they all have well-defined meanings, prevent user confusion for those already familiar with gherkin, and would also ease the transition to a hypothetical future of mapping requirements to code blocks