If you have any questions or suggestions around this blog post then please
reach out to @johlju in the Virtual PowerShell User Group #DSC channel.
UPDATE: To convert tests for a repository in DSC Community also see the blog Convert tests to Pester 5 for a DSC Community repository
Pester, the famous DSL and module in the PowerShell community, is used throughout the DSC modules and their common modules to ensure we maintain quality for all new contributions by testing any addition and verify they work with the rest of the code to avoid regression.
A new major version of Pester is about to be released, bringing improvements in some areas along with breaking changes compared to the previous v4 versions. In this post I will explain what may be needed to convert your existing tests from v4 to v5. See the release note under Pester Releases where for example the RC6 release notes contain documentation needed for converting.
Pester is primarily maintained and developed by Jakub Jareš. Pester 5 was made possible thanks to Jakub’s hard work during many weekends, outside of his day job. If you have the means then consider sponsoring Jakub or sponsoring Pester to show your appreciation for Jakub’s dedication to the community and improving this OSS tool.
I thought I’d start with the module SqlServerDsc.Common that contains
helper functions for the DSC resources in the SqlServerDsc module. I knew
it used a lot of different techniques and also had older tests from before
Pester 4. But after I started looking at the tests I realized that they
have been refactor over the years to use modern Pester techniques such as
having most setup and teardown done in Before*- and After*-blocks. So
for these tests I needed to refactor just a few tests to use the Before-
and After-block way.
Even though the tests is doing setup and teardown in Before*- and
After*-blocks, don’t expect the tests to just run under Pester 5.
It took me still over 8 hours to convert 175 test for 3000 lines of code.
If the tests had not used the Before*- and After*-blocks as much as
they did the conversion would have taken considerably longer.
It took, and will take some time, to learn new practices around how Pester
does Discovery and Run which is new to Pester 5, especially around
InModuleScope-block.
Pester 5 is mostly backward compatible with the syntax used in Pester v4, but it is recommended to use the new cmdlets name as backward compatibility is achieved with aliases in v5.
What took most time was to understand the error messages that was outputted when running test that was not “Pester 5 compatible”. When the tests was wrongly written for Pester 5 it took some time to understand if the error was from Pester, bug in tests, or bug in the code being tested. 99.9% of the time it was just the tests that needed to be slightly modified to accommodate Pester’s new way of working.
Converting to Pester 5 did help found a bug in the tests that we had missed with Pester 4. There was a need for mocking a line in the code that messed up the PowerShell session. That needed a change in the code being testing.
After writing Pester tests in Pester 3 and 4 I was used to some error messages returned by Pester when messing up tests, the same will happen for Pester 5, eventually.
I also had tremendous help from the community in the Virtual PowerShell User Group #testing channel when doing this conversion. It is the same place where the #DSC channel is located.
From the conversion there was changes to the unit tests that still worked in Pester 4. Those changes was merged into master, and the changes can be seen here: https://github.com/dsccommunity/SqlServerDsc/pull/1548/files#diff-05195506e4856646b3925b7788341a92
And here is the PR so you can see the actually difference for the tests for them to run in Pester 5 (linking to reviewable since it has a better diff engine than GitHub): https://reviewable.io/reviews/dsccommunity/sqlserverdsc/1550
Here you can find the resulting tests that passing in Pester 5: https://github.com/dsccommunity/SqlServerDsc/blob/9aff05501051e77e64b9a0081d2d16dabd562052/tests/Unit/SqlServerDsc.Common.Tests.ps1
Please note that some of these might not actually be an issue or should be solved in some other way.
Describe- or Context-block should be inside a It-,
Before*-, or After*-block. The exception to this is variables that
contain test cases, those variables should be placed inside the
Describe-block and before the It-block, but outside of the
Before*-blocks.
they must be outside Before*-blocks and It-blocks, but inside Describe-
or Context-blocks.Should -Throw 'error message' has a breaking change. It no longer
uses -contains but instead -like which breaks some tests that depend
on this when comparing error message strings. It possible to add wildcard
to the error string to workaround this, e.g. 'error message*'.
New-InvalidOperationException the expected
message no longer compares correctly since the error message contain
'System.InvalidOperationException: error message'. It can be solved
by using wildcard as previously mentioned, but it can also be solved
by adding the test helper function Get-InvalidOperationRecord.
Using this helper function will return the the same error message
and no wildcard is needed.Assert-MockCalled is alias to Should -Invoke and uses the same
parameters (backward compatible) so there is no need to replace
Assert-MockCalled initially. But a suggestion is replace this as it
is easy with a search and replace.InModuleScope-block outside of an It-block then the module
must be loaded into the session outside the top BeforeAll-block.
Read more about this in issue pester/Pester#1543.
We use InModuleScope-block to reach the variable in the
module scope that contain the localized strings ($script:localizedData).Should -Throw should preferably be updated to Should -Throw -ExpectedMessage
to use named parameters. But is is not a requirement for running the tests.Should -Throw $null will pass even if the code being tested actually
threw an exception. So if we get the expected error message from a localized
string and that for some reason resultet in a $null value instead of the
correct string the test will pass even if the code being test throw the
wrong error message. To handle this the expected error message variable
should be assert to contain a value.
$errorMessage = $script:localizedString.LocalizedErrorMessage
$errorMessage | Should -Not -BeNullOrEmpty
{ Set-Something } | Should -Throw $errorMessage
BeforeEach behaves differently (better) than Pester 4. In Pester 4
if a mock was declared in a previous BeforeAll- or BeforeEach-block
the BeforeEach block override the previous mock. That is not happening
in Pester 5 meaning that the wrong mock could be called.
It is more important to make Context-blocks self-sustaining and avoid
inheriting test setup and or teardown.param-block inside a -MockWith should be removed.
Se more information in issue pester/Pester#1554.foreach-blocks that was used did not work since it didn’t see the
variable from the foreach() inside the It-block because the
foreach-block was outside the Before*- and It-block.
The foreach-blocks was removed and replaced with test cases.Should Be
and Should Throw (without dash), the tests are failing with the error
message:
ParameterBindingException: Parameter set cannot be resolved using the specified named parameters.
One or more parameters issued cannot be used together or an insufficient number of parameters were provided.
[Parameter()] the test will fail. Remove the decoration
[Parameter()] and the tests work with both Pester 4 and 5. In Pester 5
it is possible to remove the entire param-block, which is preferably but
then there is not backward compatibility with Pester 4.
It 'Should...if value is <ParameterValue>' -TestCase @( ... ) {
param
(
[System.String]
$ParameterValue
)
Get-Something | Should -Be $ParameterValue
}
Should -Invoke (or alias Assert-MockCalled)
without having declared a mock for the command. Then Should -Invoke is
just ignored without any error that there was no mock declared as Pester 4
did.$PSBoundParameters in a -ParameterFiler
for a Should -Invoke. This is being tracked in issue pester/Pester#1542
and won’t be fixed before 5.0.0 GA.Mock’s are outside the Before*- or It-blocks then things are
not mocked so make sure prior running the tests that all code is
correctly placed! There is no check that you incorrectly placed a Mock
where it serves no purpose.A Pester 5 test should basically look like this. All test code must be
be inside Before*-, After*-, and It-blocks. There are an exception
to the rule when it comes to test cases.
Note: This is just an example from me which may change in the future when new practices are learned and old habits are improved or replaced.
<#
Import module if using InModuleScope. There are other possible solution for
this, see issue https://github.com/pester/Pester/issues/1543.
#>
Import-Module -Name '.\output\SqlServerDsc\14.0.0\Modules\SqlServerDsc.Common'
BeforeAll {
# More initialization code like loading stub cmdlets or stub classes
}
Describe 'SqlServerDsc.Common\Set-Something' {
Context 'When...' {
BeforeAll {
# Test setup. Mock and mock variables as needed.
}
BeforeEach {
# Test setup per It-block. Mock and mock variable setup.
}
AfterAll {
# Test teardown.
}
AfterEach {
# Test teardown per It-block.
}
It 'Should...' {
{ Set-Something } | Should -Not -Throw
Should -Invoke -CommandName Get-Something -Exactly -Times 1 -Scope It
}
}
Context 'When...' {
# Test cases can be added directly to the It-block.
It 'Should...' -TestCases @(
{
MockVersion = '11'
}
{
MockVersion = '12'
}
) {
{ Set-Something -Version $MockVersion } | Should -Not -Throw
Should -Invoke -CommandName Get-Something -Exactly -Times 1 -Scope It
}
<#
If test cases are defined in a variable then the variable
is not allowed to be inside a Before*-block because then
it is not executed during Discovery when the test cases are
evaluated. Correct placement of these variables are inside
the Describe-block and before the It-block, but outside of
the Before*-blocks.
#>
$testCases = @(
{
MockVersion = '11'
}
{
MockVersion = '12'
}
)
# Test cases can be added directly like this to the It-block-
It 'Should...' -TestCases $testCases {
{ Set-Something -Version $MockVersion } | Should -Not -Throw
Should -Invoke -CommandName Get-Something -Exactly -Times 1 -Scope It
}
}
<#
Write tests so you only need to have InModuleScope around as little
code as possible, preferably only inside the It-block. But if the
It-block needs variables in the test setup then that is not possible
since those will not be able in the module scope.
See issue https://github.com/pester/Pester/issues/1543.
#>
InModuleScope $script:subModuleName {
Context 'When...' {
BeforeAll {
# Test setup. Mock and mock variables as needed.
}
BeforeEach {
# Test setup per It-block. Mock and mock variable setup.
}
AfterAll {
# Test teardown.
}
AfterEach {
# Test teardown per It-block.
}
It 'Should throw the correct error message' {
# The variable $localizedData is coming from the module being tested.
$mockErrorMessage = $localizedData.LocalizedErrorMessage
# Assert that the localized string was fetched.
$mockErrorMessage.Exception.Message | Should -Not -BeNullOrEmpty
# Assert that the correct error message was thrown.
{
Set-Something
} | Should -Throw -ExpectedMessage $mockErrorMessage
}
}
}
}