Releases: pester/Pester
5.3.0-alpha4
- Fixes coverage of default values in param block when the new CC is used.
See in the previous releases how to test it and compare the old CC with the new, and let me know please.
5.3.0-alpha3
Broken build, don't use.
5.3.0-alpha2
- Improves the tracer based code coverage, please try it and let me know where it is different from the current CC. This is how you do it:
There is now $configuration.CodeCoverage.UseBreakpoints experimental flag that allows you to use Profiler based code coverage instead of breakpoints based code coverage.
Here is a fuller example of generating CC for your code base using both methods.
https://gist.github.com/nohwnd/3a29c8f4afc21329a146b5cba953d503
If you can please try it and compare the xmls. E.g. open both in vscode and use the Compare active file with... command from the command pallete. You should see only difference on the top in timestamps.
This should work from PS3 PS4 up. At least it does in our CI pipeline. (PS3 has some weird issues I can't figure out, like not being able to enumerate collections and so on, so use at your own risk.)
The tracer based CC also cannot process default values in parameters, so it reports them as missed.
4.10.2-beta1
- Added parameter for Alias in HaveParameter (#1463)
- Fixes #1028 - Code Coverage for proxy functions/steppablePipelines (#1494)
- Many gherkin runner fixes and improvements (#1276)
Full log 4.10.1...master
5.2.2
5.3.0-alpha1
There is now $configuration.CodeCoverage.UseBreakpoints experimental flag that allows you to use Profiler based code coverage instead of breakpoints based code coverage.
Here is a fuller example of generating CC for your code base using both methods.
https://gist.github.com/nohwnd/3a29c8f4afc21329a146b5cba953d503
If you can please try it and compare the xmls. E.g. open both in vscode and use the Compare active file with... command from the command pallete. You should see only difference on the top in timestamps.
This should work from PS3 up. At least it does in our CI pipeline.
5.2.1
5.2.0
Pester 5.2.0 is finally here! 🥳🥳🥳
First of, thanks to all the contributors, especially @fflaten who responded to a ton of issues, made multiple PRs, helped me diagnose problems, and checked my fixes. Thank you!
Code coverage
Coverage report is back, on screen and on the result object
The theme of this release was Code Coverage. I finally fixed the coverage report which is now output to the screen when Detailed (or Diagnostic) output level is specified. And it is always attached to the result object when CodeCoverage is enabled. The CodeCoverage data are attached to the result object as well, if you want to do further processing on it.
Performance is better
I focused on performance as well and all breakpoints are now set in one place, making it 50% faster in my tests, but I would love to see your numbers. There is also new option, CodeCoverage.SingleHitBreakpoints, which will remove the breakpoint as soon as it is hit, lowering the overhead in PowerShell. This option is enabled by default and makes the execution a bit faster as well.
But not as great as it can be
I did some new research and have a proof of concept using my new Profiler to do code coverage which is almost as fast as running without it. This will become a new experimental option soon and should work for all versions of PowerShell that Pester supports. I will announce more details later.
I also implemented CodeCoverage using the still unreleased PowerShell profiler, which I started about half a year ago, and that @iSazonov has been working on. PowerShell/PowerShell#13673 Once (or if) this is merged and released Pester is ready to start using it #1884.
You can specify your desired code coverage percent
Using option CodeCoverage.CoveragePercentTarget you can now specify the desired code coverage. The default is 75%. This has only visual effect for now, it shows as green message in the output when the target is met, or as red when it is not met.
(See it in the gif below)
VSCode + Pester CodeCoverage are now friends
Using CodeCoverage in VSCode is very painful and that's a shame. I added new format based on JaCoCo which is especially catered to Coverage Gutters extension in VSCode. This, plus some boilerplate code enables you to easily see code coverage when developing in VSCode. The format is also compatible with Azure DevOps coverage view.
Full DEMO of the feature is here: https://youtu.be/qeiy8fRMHf8?t=5697
And code here https://gist.github.com/nohwnd/efc339339dc328d93e0fe000249aea25
❗ -CI no longer enables CodeCoverage
Lastly, the -CI switch no longer enables CodeCoverage. At the moment there is no stable way to make CodeCoverage fast on all versions of PowerShell so it is not a good default for beginners, or build pipelines you want to quickly setup. If you don't mind the overhead, use this configuration to get the functionality of the -CI switch:
$configuration = New-PesterConfiguration
$configuration.CodeCoverage.Enabled = $true
$configuration.TestResult.Enabled = $true
$configuration.Run.Exit = $true
Invoke-Pester -Configuration $configurationThere will be also a new option coming to take advantage of the Profiler based code coverage. Please help me test it when it comes so we can make it new default!
Related changes:
- Fix blank code coverage 1621 #1807
- Coverage gutters #1887
- Fix wildcards in dirs #1858
- Add Coverage to result object #1860
- Little improvements to coverage output #1866
- Bunch bps and one hit bps #1865
- CI switch does not enable CodeCoverage by default #1911
Configuration
New-PesterConfiguration
New-PesterConfiguration cmdlet is added which returns [PesterConfiguration]::Default. It is recommended to be used instead of the .NET call because it will auto-load Pester if it was not loaded already.
The help for this cmdlet is generated from the object, so includes all options that are present.
Throw on failed run
New option Run.Throw is added which is similar to Run.Exit (-EnableExit). When enabled Pester will throw when there are any failed tests. When both Run.Exit and Run.Throw are enabled throwing exception is preferred, because it is more informative and because it works better with VSCode where exit is ignored.
Pester.dll version is checked on import
The Dll holding the configuration and other types is now versioned based on the version of Pester that it is released with. There is also a minimal version check that will ensure that you will get an error on module load, when you already have an older version of Pester loaded in the current session. This unfortunately cannot be avoided and throwing a sensible error is better than getting failure during runtime because some property on an object was renamed.
Related changes:
- Extend and export New-PesterConfiguration #1728
- Add assembly version check during module import #1790
- Add option to throw on failure #1908
Should
Should -Be for string
The useful arrow on Should -Be when comparing strings is back. I updated the implementation to show as much as it can based on how wide your window is, without wrapping the lines for big outputs. Notice the difference in the last example is on index 985 in string which is over 4000 characters long.
- Useful string assert message #1880
Mocking
The mocking was not focus of the current sprint, but I made a lot of fixes there as well.
Cleanup
When you cancel test run using Ctrl+C Pester mock functions and aliases may stay in the session. In subsequent Invoke-Pester call we need to clean them up to ensure stale mocks are not making your tests fail. Pester now looks for those stale mocks not just in Pester scope, but also in all currently loaded modules and user scope.
$PesterBoundParameters variable
In -MockWith and -ParameterFilter you can now use $PesterBoundParameters variable which holds all the bound parameters for that function call. This variable is like the $PSBoundParameters which is not correctly populated, and cannot be without breaking Mock debugging.
Logging
The diagnostic output when searching for mock behaviors is much improved. Calling a mock will show all the behaviors that are present for that command, and reason why they are used or not used. There is also a list of all the behaviors that will execute parameter filter, and default behaviors. The log also shows the target module in which a mock was defined more clearly (in the log below it is module m). When mock is in script scope $none is used to denote it in the log. The mock hook function has a clearer name showing for which command the mock is.
In the code example below, mocks are defined in two different scopes (module m and script scope):
Invoke-Pester -Container (
New-PesterContainer -ScriptBlock {
Describe 'Mock logging' {
It 'is more detailed' {
Get-Module m | Remove-Module
New-Module m -ScriptBlock {
function Invoke-Job ($ScriptBlock){
Start-Job -ScriptBlock $ScriptBlock
}
} | Import-Module
Mock Start-Job -ModuleName m -MockWith { "default mock in module m" }
Mock Start-Job -ModuleName m -MockWith {
"parametrized mock in module m"
} -ParameterFilter { $Name -eq "hello-job" }
Mock Start-Job -MockWith { "default mock in script" }
# call Start-Job in script
# this will call the Mock defined in script
Start-Job -ScriptBlock { Write-Host "hello" }
# call mock of Start-Job via Invoke-Job inside of module m
# this will call the Mock defined in module m
Invoke-Job -ScriptBlock { Write-Host "hello" }
}
}
}
) -Output DiagnosticThis is the improved log:
Mock: Setting up default mock for m - Start-Job.
Mock: Resolving command Start-Job.
Mock: ModuleName was specified searching for the command in module m.
Mock: Found module m version 0.0.
Mock: Found the command Start-Job in a different module.
Mock: Mock does not have a hook yet, creating a new one.
Mock: Defined new hook with bootstrap function PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb and aliases Start-Job, Microsoft.PowerShell.Core\Start-Job.
Mock: Adding a new default behavior to m - Start-Job.
Mock: Setting up parametrized mock for m - Start-Job.
[...]
Mock: Found the command Start-Job in a different module and it resolved to PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb.
Mock: Mock resolves to an existing hook, will only define mock behavior.
Mock: Adding a new parametrized behavior to m - Start-Job.
[...]
Mock: Setting up default mock for Start-Job.
Mock: We are in a test. Returning mock table from test scope.
Mock: Resolving command Start-Job.
Mock: Searching for command Start-Job in the script scope.
...
5.2.0-rc2
5.2.0-rc1
Pester 5.2.0 is finally close to release! 🥳🥳🥳
First of, thanks to all the contributors, especially @fflaten who responded to a ton of issues, made multiple PRs, helped me diagnose problems, and checked my fixes. Thank you!
Code coverage
Coverage report is back, on screen and on the result object
The theme of this release was Code Coverage. I finally fixed the coverage report which is now output to the screen when Detailed (or Diagnostic) output level is specified. And it is always attached to the result object when CodeCoverage is enabled. The CodeCoverage data are attached to the result object as well, if you want to do further processing on it.
Performance is better
I focused on performance as well and all breakpoints are now set in one place, making it 50% faster in my tests, but I would love to see your numbers. There is also new option, CodeCoverage.SingleHitBreakpoints, which will remove the breakpoint as soon as it is hit, lowering the overhead in PowerShell. This option is enabled by default and makes the execution a bit faster as well.
But not as great as it can be
I also implemented CodeCoverage using the still unreleased PowerShell profiler, which I started about half a year ago that @iSazonov has been working on. PowerShell/PowerShell#13673 Once this is merged and released Pester is ready to start using it #1884. This will make CodeCoverage almost as fast running without it.
You can specify your desired code coverage percent
Using option CodeCoverage.CoveragePercentTarget you can now specify the desired code coverage. The default is 75%. This has only visual effect for now, it shows as green message in the output when the target is met, or as red when it is not met.
(See it in the gif below)
VSCode + Pester CodeCoverage are now friends
Using CodeCoverage in VSCode is very painful and that's a shame. I added new format based on JaCoCo which is especially catered to Coverage Gutters extension in VSCode. This, plus some boilerplate code enables you to easily see code coverage when developing in VSCode. The format is also compatible with Azure DevOps coverage view.
Full DEMO of the feature is here: https://youtu.be/qeiy8fRMHf8?t=5697
And code here https://gist.github.com/nohwnd/efc339339dc328d93e0fe000249aea25
❗ -CI no longer enables CodeCoverage
Lastly, the -CI switch no longer enables CodeCoverage. There is no way to make CodeCoverage fast on all versions of PowerShell so it is not a good default for beginners, or build pipelines you want to quickly setup. If you don't mind the overhead, use this configuration to get the functionality of the -CI switch:
$configuration = New-PesterConfiguration
$configuration.CodeCoverage.Enabled = $true
$configuration.TestResult.Enabled = $true
$configuration.Run.Exit = $true
Invoke-Pester -Configuration $configurationRelated changes:
- Fix blank code coverage 1621 #1807
- Coverage gutters #1887
- Fix wildcards in dirs #1858
- Add Coverage to result object #1860
- Little improvements to coverage output #1866
- Bunch bps and one hit bps #1865
- CI switch does not enable CodeCoverage by default #1911
Configuration
New-PesterConfiguration
New-PesterConfiguration cmdlet is added which returns [PesterConfiguration]::Default. It is recommended to be used instead of the .NET call because it will auto-load Pester if it was not loaded already.
Throw on failed run
New option Run.Throw is added which is similar to Run.Exit (-EnableExit). When enabled Pester will throw when there are any failed tests. When both Run.Exit and Run.Throw are enabled throwing exception is preferred, because it is more informative and because it works better with VSCode where exit is ignored.
Pester.dll version is checked on import
The Dll holding the configuration and other types is now versioned based on the version of Pester that it is released with. There is also a minimal version check that will ensure that you will get an error on module load, when you already have an older version of Pester loaded in the current session. This unfortunately cannot be avoided and throwing a sensible error is better than getting failure during runtime because some property on an object was renamed.
Related changes:
- Extend and export New-PesterConfiguration #1728
- Add assembly version check during module import #1790
- Add option to throw on failure #1908
Should
Should -Be for string
The useful arrow on Should -Be when comparing strings is back. I updated the implementation to show as much as it can based on how wide your window is, without wrapping the lines for big outputs. Notice the difference in the last example is on index 985 in string which is over 4000 characters long.
- Useful string assert message #1880
Mocking
The mocking was not focus of the current sprint, but I made a lot of fixes there as well.
Cleanup
When you cancel test run using Ctrl+C Pester mock functions and aliases may stay in the session. In subsequent Invoke-Pester call we need to clean them up to ensure stale mocks are not making your tests fail. Pester now looks for those stale mocks not just in Pester scope, but also in all currently loaded modules and user scope.
$PesterBoundParameters variable
In -MockWith and -ParameterFilter you can now use $PesterBoundParameters variable which holds all the bound parameters for that function call. This variable is like the $PSBoundParameters which is not correctly populated, and cannot be without breaking Mock debugging.
Logging
The diagnostic output when searching for mock behaviors is much improved. Calling a mock will show all the behaviors that are present for that command, and reason why they are used or not used. There is also a list of all the behaviors that will execute parameter filter, and default behaviors. The log also shows the target module in which a mock was defined more clearly (in the log below it is module m). When mock is in script scope $none is used to denote it in the log. The mock hook function has a clearer name showing for which command the mock is.
In the code example below, mocks are defined in two different scopes (module m and script scope):
Invoke-Pester -Container (
New-PesterContainer -ScriptBlock {
Describe 'Mock logging' {
It 'is more detailed' {
Get-Module m | Remove-Module
New-Module m -ScriptBlock {
function Invoke-Job ($ScriptBlock){
Start-Job -ScriptBlock $ScriptBlock
}
} | Import-Module
Mock Start-Job -ModuleName m -MockWith { "default mock in module m" }
Mock Start-Job -ModuleName m -MockWith {
"parametrized mock in module m"
} -ParameterFilter { $Name -eq "hello-job" }
Mock Start-Job -MockWith { "default mock in script" }
# call Start-Job in script
# this will call the Mock defined in script
Start-Job -ScriptBlock { Write-Host "hello" }
# call mock of Start-Job via Invoke-Job inside of module m
# this will call the Mock defined in module m
Invoke-Job -ScriptBlock { Write-Host "hello" }
}
}
}
) -Output DiagnosticThis is the improved log:
Mock: Setting up default mock for m - Start-Job.
Mock: Resolving command Start-Job.
Mock: ModuleName was specified searching for the command in module m.
Mock: Found module m version 0.0.
Mock: Found the command Start-Job in a different module.
Mock: Mock does not have a hook yet, creating a new one.
Mock: Defined new hook with bootstrap function PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb and aliases Start-Job, Microsoft.PowerShell.Core\Start-Job.
Mock: Adding a new default behavior to m - Start-Job.
Mock: Setting up parametrized mock for m - Start-Job.
[...]
Mock: Found the command Start-Job in a different module and it resolved to PesterMock_m_Start-Job_a611abe3-203b-42c7-b81f-668945eb29eb.
Mock: Mock resolves to an existing hook, will only define mock behavior.
Mock: Adding a new parametrized behavior to m - Start-Job.
[...]
Mock: Setting up default mock for Start-Job.
Mock: We are in a test. Returning mock table from test scope.
Mock: Resolving command Start-Job.
Mock: Searching for command Start-Job in the script scope.
Mock: Found the command Start-Job in the script scope.
Mock: Mock does not have a hook yet, creating a new one.
Mock: Defined new hook with bootstrap function PesterMock_script_Start-Job_7745c21c-4173-4c11-be80-7274dd7b93ec and aliases Start-Job, Microsoft.PowerShell.Core\Start-Job.
[...]
Mock: Found 0 behaviors in all parent blocks, and 3 behaviors in test.
Mock: Filtering behaviors for command Start-Job, for target module $null (Showing all behaviors for this command, actual filtered list is further in the log, look for 'Filtered parametriz...