Veggies is an awesome cucumberjs library for API/CLI testing. Great for testing APIs built upon Express, Koa, HAPI, Loopback and others. It's also the perfect companion for testing CLI applications built with commander, meow & Co.
- Requirements
- Installation
- CLI
- Features
- Extensions
- Helpers
- Examples
- Node.js
>=12.0.0
- cucumber
>=7.0.0
Using npm:
npm install @ekino/veggies
Or yarn:
yarn add @ekino/veggies
Then all you have to do is installing the provided extensions:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state, fixtures, httpApi, cli } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
fixtures.extendWorld(this)
httpApi.extendWorld(this)
cli.extendWorld(this)
})
state.install()
fixtures.install()
httpApi.install({
baseUrl: 'http://localhost:3000',
})
cli.install()
Veggies provides a simple CLI allowing the use of custom options to configure the feature it provides.
As cucumber-js
no longer supports custom options, it's the safest way to use Veggies in full without problem.
To make use of it, you can use either of these commands:
$ yarn veggies
$ npx veggies
$ ./node_modules/.bin/veggies.js
The available options are:
Option | Description |
---|---|
--cleanSnapshots | removes unused snapshots (not recommended while matching tags) |
-u, --updateSnapshots | updates current snapshots if required |
--preventSnapshotsCreation | a snapshot related step that would create one will fail instead (useful on CI environment) |
--help | prints the veggies CLI help then the cucumber-js CLI help |
Please refer to the CucumberJS CLI documentation to see native Cucumber options (or use veggies --help
).
For full feature list, have a look at available gherkin expressions for the dedicated extension.
Scenario: Using GitHub API
Given I set User-Agent request header to veggies/1.0
When I GET https://api.github.com/
Then response status code should be 200
You can easily issue a POST request using json payload
Scenario: Creating a resource using json payload
Given I set request json body
| username | plouc |
| gender | male |
When I POST https://my-api.io/users
Then response status code should be 201
You can also use form encoded values, all you have to do is
to change json
for form
Scenario: Creating a resource using json payload
Given I set request form body
| username | plouc |
| gender | male |
When I POST https://my-api.io/users
Then response status code should be 201
Putting large data payloads inside your scenarios can reduce legibility, to improve this, you can use the fixtures extension to define it.
# /features/user/fixtures/user.yml
username: plouc
gender: male
# /features/user/create_user.feature
Scenario: Creating a resource using json payload
Given I set request form body from user
When I POST https://my-api.io/users
Then response status code should be 201
Imagine you want to test a resource creation and then that you're able to fetch this new entity through the API.
If resource id is generated by your API, it will be impossible to make the second call because id is unknown.
To solve this problem you have the ability to collect data from a previous response, store it in the state and inject it at various places using placeholders.
The following example calls the root GitHub API endpoint,
extracts the emojis_url
value from the json response and
stores it in the current state under the emojisUrl
key,
then it uses this value to make its next request.
Scenario: Using GitHub API
Given I set User-Agent request header to veggies/1.0
When I GET https://api.github.com/
And I pick response json emojis_url as emojisUrl
And I GET {{emojisUrl}}
Then response status code should be 200
It's even possible to mix this approach with scenario outline to have more concise tests (at the cost of clarity thought).
The following example will generate 3 scenario at runtime using different response values for second request.
Scenario Outline: Fetching <key> API endpoint from root endpoint
Given I set User-Agent request header to veggies/1.0
When I GET https://api.github.com/
Then response status code should be 200
When I pick response json <key> as <key>
And I GET {{<key>}}
Then response status code should be 200
Examples:
| key |
| emojis_url |
| feeds_url |
| public_gists_url |
Pick a value from response header
Syntax:
I pick response header <key> as <key>
Example:
Scenario: Setting json body from .json fixture file
And set request json body from json_file
When I POST https://examples.com/users
Then response status code should be 201
And I pick response header location as location
And I clear request body
And I GET {{location}}
And response status code should be 200
Syntax:
I replace (placeholder) <search> in <key> to <value> with regex option <flags>
placeholder
andwith regex option <flags>
are optional<value>
does not support spaces
Example:
And I replace {token} in URLPage to e1c401d5c
And I replace placeholder {stateUpMode} in URLPage to live with regex option gi
Cookies are disabled by default, but you've got the ability to enable/disable the feature using a gherkin Given
expression.
Be warned that cookies do not persist across scenarios in order to ensure they're autonomous.
If you really need to keep a cookie for multiple scenarios, you should consider using a custom step definition
and/or using the state extension to store it.
Scenario: Enabling cookies
Given I enable cookies
# …
Scenario: Disabling cookies
Given I disable cookies
# …
See definitions for all available cookies related gherkin expressions.
veggies gives you the ability to check json responses, the corresponding gherkin expression is:
/^(?:I )?json response should (fully )?match$/
Checking json response properties equal value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| name | equal | thing |
| address.country | equal | Japan |
Checking json response properties start with value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| name | start with | ing |
| address.country | starts with | Jap |
Checking json response properties contain value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| name | contain | ing |
| address.country | contain | Jap |
Checking json response properties end with value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| name | end with | ing |
| address.country | ends with | pan |
Checking json response properties match value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| name | match | ^(.+)ing$ |
| address.country | match | ^Jap(.+)$ |
Checking json response properties equalRelativeDate value:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should match
| field | matcher | value |
| endDate | equalRelativeDate | 2,days,fr,dddd |
| beginDate | equalRelativeDate | -1,week,fr,[Aujourd'hui] YYYY-MM-DD hh[h]mm |
By default, this assertion does not check for full match. Properties not listed will just be ignored, if you want a full match:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should fully match
| field | matcher | value |
| name | match | ^(.+)ing$ |
| address.country | equal | Japan |
Now if the json contains extra properties, the test will fail.
Available matchers are:
matcher | short matcher | description |
---|---|---|
match |
~= |
property must match given regexp |
matches |
see match |
see match |
start with |
^= |
property must start with given value |
starts with |
see start with |
see startWith |
contain |
*= |
property must contain given value |
contains |
see contain |
see contain |
end with |
$= |
property must end with given value |
ends with |
see end with |
see endWith |
defined |
? |
property must not be undefined |
present |
see defined |
see defined |
equal |
= |
property must equal given value |
equals |
see equal |
see equal |
type |
#= |
property must be of the given type |
equalRelativeDate |
n/a | property must be equal to the computed date |
Any of these matchers can be negated when preceded by these : !
, not
, does not
, doesn't
, is not
and isn't
.
The short version of each matcher is intended to be used that way:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should fully match
| expression |
| name ~= ^(.+)ing$ |
| address.country = Japan |
| address.city ? |
| address.postalCode #= string |
If it eases the reading, you can also pad your expressions:
Scenario: Fetching some json response from the internets
When I GET http://whatever.io/things/1
Then json response should fully match
| expression |
| name ~= ^(.+)ing$ |
| address.country = Japan |
| address.city ? |
| address.postalCode #= string |
In order to check response headers, you have the following gherkin expression available:
/^response header (.+) should (not )?(equal|contain|match) (.+)$/
It supports multiple combinations to verify header value conforms to what you expect. This example illustrates its different features:
Scenario: Testing header related expectations
When I GET http://whatever.io/
Then response header X-Whatever-A should equal whatever
And response header X-Whatever-B should not equal whatever
And response header X-Whatever-C should contain part
And response header X-Whatever-D should not contain part
And response header X-Whatever-E should match ^(thing|other)$
And response header X-Whatever-F should not match ^(thing|other)$
If the header does not exist, the test will fail.
When testing APIs using cucumber, we often faced situations were we didn't understand why a given test were failing.
The dirty fix was to add some nasty console.log()
everywhere, that's why veggies provides helpers to dump response properties.
Scenario: Fetching something from the internets
When I GET http://whatever.io/things
And dump response body
And dump response headers
And dump response cookies
You'll now have the response body/headers/cookies dumped in your terminal. You should disable those steps when the test is fixed as it can be noisy enough.
When testing json based APIs, which is a standard nowadays, you have to be aware of data types for sending payloads or making assertions on received responses, that's why veggies provides a lightweight type systems.
The following directives are available:
directive | type | example | output |
---|---|---|---|
((undefined)) |
undefined |
((undefined)) |
undefined |
((null)) |
null |
((null)) |
null |
<value>((string)) |
string |
hi((string)) |
'hi' |
<value>((number)) |
number |
1((number)) |
1 |
<value>((boolean)) |
boolean |
true((boolean)) |
true |
<value>((array)) |
Array |
one,two,three((array)) |
['one', 'two', 'three'] |
You can now use those directive for most of the step definitions accepting data tables.
For example, you can use it to post typed json data:
Scenario: Creating a resource using typed json payload
Given I set request json body
| username | plouc((string)) |
| team_id | 1((number)) |
| is_active | true((boolean)) |
| hobbies | drawing,hacking((array)) |
When I POST https://my-api.io/users
Then response status code should be 201
which will generate the following payload:
{
"username": "plouc",
"team_id": 1,
"is_active": true,
"hobbies": [
"drawing",
"hacking"
]
}
For full feature list, have a look at available gherkin expressions for the dedicated extension.
Scenario: Getting info about installed yarn version
When I run command yarn --version
Then exit code should be 0
Scenario: Running an invalid command
When I run command yarn invalid
Then exit code should be 1
And stderr should contain error Command "invalid" not found.
In order to check file content, you have the following gherkin expression available:
/^file (.+) content (not )?(equal|contain|match) (.+)$/
It supports multiple combinations to verify file content conforms to what you expect. This example illustrates its different features:
Scenario: Testing file content related expectations
Then file sample_A.text content should equal whatever
And file sample_B.text content should not equal whatever
And file sample_C.text content should contain part
And file sample_D.text content should not contain part
And file sample_E.text content should match ^(thing|other)$
And file sample_F.text content should not match ^(thing|other)$
If the file does not exist, the test will fail.
Snapshot testing compare a response / content against a saved snapshot.
Snapshots are stored in a file with same name as the feature file with the extension .snap
in a folder snapshots in the same folder as the feature file.
Folder tree should look like : support/ features/ feature_with_snapshot.feature feature_without_snapshot.feature snapshots/ feature_with_snapshot.feature.snap …
In a snapshot file, snapshot name follow the pattern: SNAPSHOT_NAME NUMBER_OF_TIME_THIS_NAME_WAS_ENCOUNTERED_IN_CURRENT_FILE.NUMBER_OF_TIME_WE_HAD_A_SNAPSHOT_IN_THIS_SCENARIO. For example, this would give: Scenario 1 1.1
If a snapshot doesn't exist, it will be created the first time.
To update snapshot use the cucumber command line option '-u'. If you narrowed the tests with tags, only the snapshots related to the tagged scenarios will be updated.
In case you need to remove unused snapshots, you can use the option --cleanSnapshots
.
:warning: You shouldn't use this option with tags. It may result in used snapshots removed.
:information_source: Snapshot files related to feature files with no snapshots won't get removed. You need to do it manually.
Sometimes, it could be useful to prevent the creation of snapshots, for instance in a CI environment. To do this,
you can use the --preventSnapshotsCreation
flag. An error will be thrown if the snapshot is missing and this option is present.
In order to check an api response against a snapshot, you have the following gherkin expression available:
/^response body should match snapshot$/
This example illustrates it:
Scenario: Creating a resource using typed json payload
Given I set request json body
| username | plouc((string)) |
| team_id | 1((number)) |
| is_active | true((boolean)) |
| hobbies | drawing,hacking((array)) |
When I POST https://my-api.io/users
Then response status code should be 201
And response body should match snapshot
It is sometimes useful to ignore some fields in a response when comparing with the snapshot. In this case and if it's json you can then use with a table:
/^response json body should match snapshot$/
This examples illustrates it:
Scenario: Creating a resource using typed json payload
Given I set request json body
| username | plouc((string)) |
| team_id | 1((number)) |
| is_active | true((boolean)) |
| hobbies | drawing,hacking((array)) |
When I POST https://my-api.io/users
Then response status code should be 201
And response json body should match snapshot
| field | matcher | value |
| url | type | string |
The table supports anything defined in Testing json response
In order to check a CLI output against a snapshot, you have the following gherkin expression available:
/^(stderr|stdout) output should match snapshot$/
This example illustrates it:
Scenario: Getting info about installed yarn version
When I run command yarn --version
Then exit code should be 0
And stdout output should match snapshot
And stderr output should match snapshot
It is sometimes useful to ignore some fields in a cli json formatted output when comparing with the snapshot. In this case and if it's json you can then use with a table:
/^(stderr|stdout) json output should match snapshot$/
This examples illustrates it:
Scenario: Snapshot testing on a json file
Given I set cwd to examples/features/snapshot/fixtures
Then json file file2.json content should match snapshot
| field | matcher | value |
| gender | type | string |
| id | type | number |
The table supports anything defined in Testing json response
In order to check a file content against a snapshot, you have the following gherkin expression available:
/^file (.+) should match snapshot$/
This example illustrates it:
Scenario: Testing file content related expectations
Then file sample_1.text should match snapshot
It is sometimes useful to ignore some fields in a json file when comparing with the snapshot. In this case and if it's json you can then use with a table:
/^json file (.+) content should match snapshot$/
This examples illustrates it:
Scenario: Creating a resource using typed json payload
Then json file sample_1.text content should match snapshot
| field | matcher | value |
| url | type | string |
The table supports anything defined in Testing json response
This module is composed of several extensions.
state | fixtures | http API | CLI | file system | snapshot
The state extension is a simple helper used to persist state between steps & eventually scenarios (but you should try to avoid coupling scenarios).
It's involved for example when you want to collect values issued by a previous request when using the http API extension.
To install the extension, you should add the following snippet to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
})
state.install()
Given:
- /^(?:I )?set state (.+) to (.+)$/
When:
- /^(?:I )?clear state$/
- /^(?:I )?dump state$/
Then:
# No definitions
When installed, you can access it from the global cucumber context in your own step definitions. For available methods on the state, please refer to its own documentation.
const { When } = require('@cucumber/cucumber')
When(/^I do something useful$/, function() {
const stateValue = this.state.get('whatever')
// …
})
The fixtures extension can be used to load data from files during testing.
It supports the following file extensions:
- .yaml, .yml - loads and parses a yaml file, result can be
Object
orArray
- .txt - loads text content, result is a
string
- .json - loads json, result is and
Object
- .js - loads a javascript module, the module must exports load function via
module.exports
, result can be whatever type the function returns
To install the extension, you should add the following snippet to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { fixtures } = require('@ekino/veggies')
setWorldConstructor(function() {
fixtures.extendWorld(this)
})
fixtures.install()
When installed, you can access it from the global cucumber context in your own step definitions. For available methods on the fixtures loader, please refer to its own documentation.
const { When } = require('@cucumber/cucumber')
When(/^I do something useful with fixtures$/, function() {
return this.fixtures.load('whatever')
.then(fixture => {
// …
})
})
The http API extension relies on the state & fixtures extensions, so make sure they're registered prior to installation.
To install the extension, you should add the following snippet
to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state, fixtures, httpApi } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
fixtures.extendWorld(this)
httpApi.extendWorld(this)
})
state.install()
httpApi.install({
baseUrl: 'http://localhost:3000',
})
Given:
- /^(?:I )?set request headers$/
- /^(?:I )?do not follow redirect$/
- /^(?:I )?follow redirect$/
- /^(?:I )?assign request headers$/
- /^(?:I )?set ([a-zA-Z0-9-_]+) request header to (.+)$/
- /^(?:I )?clear request headers/
- /^(?:I )?set request json body$/
- /^(?:I )?set request json body from (.+)$/
- /^(?:I )?set request form body$/
- /^(?:I )?set request form body from (.+)$/
- /^(?:I )?set request multipart body from (.+)$/
- /^(?:I )?clear request body$/
- /^(?:I )?set request query$/
- /^(?:I )?pick response (json|header) (.+) as (.+)$/
- /^(?:I )?replace(?: placeholder)? (.+) in (.+) to ([^\s]+)(?: with regex options? (.+)?)?$/
- /^(?:I )?enable cookies$/
- /^(?:I )?disable cookies$/
- /^(?:I )?set cookie from (.+)$/
- /^(?:I )?clear request cookies$/
When:
- /^(?:I )?reset http client$/
- /^(?:I )?(GET|POST|PUT|DELETE|PATCH) (.+)$/
- /^(?:I )?dump response body$/
- /^(?:I )?dump response headers$/
- /^(?:I )?dump response cookies$/
Then:
- /^response status code should be ([1-5][0-9][0-9])$/
- /^response status should be (.+)$/
- /^response should (not )?have an? (.+) cookie$/
- /^response (.+) cookie should (not )?be secure$/
- /^response (.+) cookie should (not )?be http only$/
- /^response (.+) cookie domain should (not )?be (.+)$/
- /^(?:I )?json response should (fully )?match$/
- /^(?:I )?should receive a collection of ([0-9]+) items?(?: for path )?(.+)?$/
- /^response should match fixture (.+)$/
- /^response header (.+) should (not )?(equal|contain|match) (.+)$/
When installed, you can access its client from the global cucumber context in your own step definitions. For available methods on the client, please refer to its own documentation.
const { When } = require('@cucumber/cucumber')
When(/^I do something useful$/, function() {
return this.httpApiClient.makeRequest(/* … */)
})
The CLI extension relies on the state & fixtures extensions, so make sure they're registered prior to installation.
To install the extension, you should add the following snippet
to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state, fixtures, cli } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
fixtures.extendWorld(this)
cli.extendWorld(this)
})
state.install()
fixtures.install()
cli.install()
Given:
- /^(?:I )?set (?:working directory|cwd) to (.+)$/
- /^(?:I )?set ([^ ]+) (?:env|environment) (?:var|variable) to (.+)$/
- /^(?:I )?set (?:env|environment) (?:vars|variables)$/
- /^(?:I )?kill the process with ([^ ]+) in (\d+)(ms|s)/
When:
- /^(?:I )?run command (.+)$/
- /^(?:I )?dump (stderr|stdout)$/
Then:
- /^(?:the )?(?:command )?exit code should be (\d+)$/
- /^(stderr|stdout) should be empty$/
- /^(stderr|stdout) should contain (.+)$/
- /^(stderr|stdout) should not contain (.+)$/
- /^(stderr|stdout) should match (.+)$/
- /^(stderr|stdout) should not match (.+)$/
When installed, you can access it from the global cucumber context in your own step definitions. For available methods on the client, please refer to its own documentation.
const { When } = require('@cucumber/cucumber')
Then(/^I check something from the CLI output$/, function() {
const out = this.cli.getOutput()
// …
})
The fileSystem extension relies on the cli extension, so make sure it's registered prior to installation.
To install the extension, you should add the following snippet
to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state, fixtures, cli, fileSystem } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
fixtures.extendWorld(this)
cli.extendWorld(this)
fileSystem.extendWorld(this)
})
state.install()
fixtures.install()
cli.install()
fileSystem.install()
Given:
- /^(?:I )?create directory (.+)$/
- /^(?:I )?remove (?:file|directory) (.+)$/
When:
# No definitions
Then:
- /^(file|directory) (.+) should (not )?exist$/
- /^file (.+) content should (not )?(equal|contain|match) (.+)$/
When installed, you can access it from the global cucumber context in your own step definitions. For available methods on the fileSystem, please refer to its own documentation.
const { Then } = require('@cucumber/cucumber')
Then(/^I check something using file system$/, function() {
return this.fileSystem.getFileContent('whatever')
.then(content => {
// …
})
})
The snapshot extension add capabilities to api, cli and file extensions, so you will need these extensions if you want to use snapshot related gherkin definitions.
To install the extension, you should add the following snippet
to your world
file:
// /support/world.js
const { setWorldConstructor } = require('@cucumber/cucumber')
const { state, fixtures, cli, fileSystem, snapshot } = require('@ekino/veggies')
setWorldConstructor(function() {
state.extendWorld(this)
fixtures.extendWorld(this)
cli.extendWorld(this)
fileSystem.extendWorld(this)
snapshot.extendWorld(this)
})
state.install()
fixtures.install()
cli.install()
fileSystem.install()
snapshot.install()
When installed, you can access it from the global cucumber context in your own step definitions. For available methods on the snapshot, please refer to its own documentation.
const { Then } = require('@cucumber/cucumber')
Then(/^Some content should match snapshot$/, function() {
this.snapshot.expectToMatch('whatever')
})
Cast helper can be used to cast values for custom gherkin rules. To find more about casting see Type System.
This must be used on gherkin arrays. Based on your array type you have to use:
step.hashes()
->Cast.objects(step.hashes())
step.rows()
->Cast.array(step.rows())
step.raw()
->Cast.array(step.raw())
step.rowsHash()
->Cast.objects(step.rowsHash())
For example:
const { cast } = require('@ekino/veggies')
const { Given, When, Then } = require('@cucumber/cucumber')
Then(/^User data should be$/, (step) => {
const userData = this.userData
const expectedData = Cast.objects(step.rowsHash())
expect(userData).to.be.deep.equal(expectedData)
})
You can provide your own type. For example:
Cast.addType('newType', value => value === 'true')
Can be used on:
Given I get user id1 profile
Then I should receive
| id | id1 |
| age | 1((number)) |
| name | veggies |
| isPublic | true((newType)) |
This repository comes with few examples, in order to run them, invoke the following script:
yarn run examples
If you want to only run certain examples, you can use tags, for example to run cli extension examples:
yarn run examples -- --tags @cli
There is a special tag which only runs examples not requiring network access:
yarn run examples -- --tags @offline
Due to public API rate limit (e.g. GitHub API), this tag is used when running on CI.