From 9b88ca0a0fdeb9260995a26fd4141b1bc18c28d1 Mon Sep 17 00:00:00 2001 From: Heinrich Gantenbein <6719941+techlake@users.noreply.github.com> Date: Mon, 22 Apr 2024 14:53:07 -0500 Subject: [PATCH] Multiple bug fixes (#576) --- Docs/ci-cd-github-actions.md | 2 +- Docs/ci-cd-overview.md | 4 +- Docs/operational-scripts-reference.md | 4 +- Docs/start-changes.md | 21 -- Scripts/Deploy/Build-DeploymentPlans.ps1 | 20 +- Scripts/Deploy/Deploy-PolicyPlan.ps1 | 243 +++--------------- Scripts/Helpers/Add-ErrorMessage.ps1 | 4 +- .../Build-AssignmentDefinitionAtLeaf.ps1 | 4 + .../Build-AssignmentParameterObject.ps1 | 2 +- Scripts/Helpers/Build-AssignmentPlan.ps1 | 2 +- Scripts/Helpers/Build-ExemptionsPlan.ps1 | 2 +- .../Confirm-ObjectValueEqualityDeep.ps1 | 44 ++-- .../Confirm-ParametersDefinitionMatch.ps1 | 67 +++-- .../Confirm-ParametersUsageMatches.ps1 | 39 +-- Scripts/Helpers/Get-AzPolicyResources.ps1 | 152 +++-------- .../Helpers/Get-AzPolicyResourcesDetails.ps1 | 4 +- Scripts/Helpers/Get-ClonedObject.ps1 | 18 +- Scripts/Helpers/Get-GlobalSettings.ps1 | 121 +++------ Scripts/Helpers/Remove-GlobalNotScopes.ps1 | 52 ++-- Scripts/Helpers/Select-PacEnvironment.ps1 | 43 +++- Scripts/Helpers/Write-ErrorsFromErrorInfo.ps1 | 6 +- .../Operations/Build-PolicyDocumentation.ps1 | 17 +- .../Operations/Export-AzPolicyResources.ps1 | 5 +- 23 files changed, 271 insertions(+), 605 deletions(-) diff --git a/Docs/ci-cd-github-actions.md b/Docs/ci-cd-github-actions.md index 222ae886..f6bfca66 100644 --- a/Docs/ci-cd-github-actions.md +++ b/Docs/ci-cd-github-actions.md @@ -19,7 +19,7 @@ You will need one [GitHub deployment environment](https://docs.github.com/en/act | TENANT-DEPLOY-POLICY | Deploy Policy resources for `tenant` | ci-cd-root-policy-contributor | | TENANT-DEPLOY-ROLES | Deploy Roles for `tenant` | ci-cd-root-user-assignments | -For each environment, [add to the environment secrets](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-secrets) for the tenant id, client id and client secret for the SPN. The secrets must be named `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET` respectively. +For each environment, [add to the environment secrets](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-secrets) for the tenant id, client id and client secret for the SPN. The secrets must be named `TENANT_ID`, `CLIENT_ID` and `CLIENT_SECRET` respectively. ### Hardening each Environment diff --git a/Docs/ci-cd-overview.md b/Docs/ci-cd-overview.md index 931478ab..807e2a82 100644 --- a/Docs/ci-cd-overview.md +++ b/Docs/ci-cd-overview.md @@ -163,7 +163,7 @@ For saving the output related to ```Build-DeploymentPlans``` there is global var | `OutputFolder` | Output folder path for plan files. Defaults to environment variable `$env:PAC_OUTPUT_FOLDER` or `./Output`. | | `DevOpsType` | If set, outputs variables consumable by conditions in a DevOps pipeline. Default: not set. | | `BuildExemptionsOnly` | If set, only builds the Exemptions plan. This useful to fast-track Exemption when utilizing [Release Flow](#advanced-cicd-with-release-flow) Default: not set. | -| `VirtualCores` | Number of (virtual) cores available to calculate the deployment plan. Defaults to 4. | +| `VirtualCores` | **Deprecated - DO NOT USE** -- Number of (virtual) cores available to calculate the deployment plan. Defaults to 4. | ### Deploy-PolicyPlan.ps1 @@ -173,7 +173,7 @@ Deploys Policies, Policy Sets, Policy Assignments, and Policy Exemptions at thei |Parameter | Explanation | |----------|-------------| | `InputFolder` | Input folder path for plan files. Defaults to environment variable `$env:PAC_INPUT_FOLDER`, `$env:PAC_OUTPUT_FOLDER` or `./Output`. | -| `VirtualCores` | Number of (virtual) cores available to deploy Policy objects in parallel. Defaults to 4. | +| `VirtualCores` | **Deprecated - DO NOT USE** -- Number of (virtual) cores available to deploy Policy objects in parallel. Defaults to 4. | ### Deploy-RolesPlan.ps1 diff --git a/Docs/operational-scripts-reference.md b/Docs/operational-scripts-reference.md index ee344820..6f0e0404 100644 --- a/Docs/operational-scripts-reference.md +++ b/Docs/operational-scripts-reference.md @@ -5,7 +5,7 @@ Builds documentation from instructions in policyDocumentations folder reading the delployed Policy Resources from the EPAC envioronment. ```ps1 -Build-PolicyDocumentation [[-DefinitionsRootFolder] ] [[-OutputFolder] ] [-WindowsNewLineCells] [[-Interactive] ] [-SuppressConfirmation] [-IncludeManualPolicies] [[-VirtualCores] ] [] +Build-PolicyDocumentation [[-DefinitionsRootFolder] ] [[-OutputFolder] ] [-WindowsNewLineCells] [-Interactive ] [-SuppressConfirmation] [-IncludeManualPolicies] [] ``` ### Parameters @@ -35,7 +35,7 @@ Suppresses prompt for confirmation to delete existing file in interactive mode Include Policies with effect Manual. Default: do not include Polcies with effect Manual. -#### `-VirtualCores ` +#### **Deprecated - DO NOT USE** -- `-VirtualCores ` Number of virtual cores to use for the operation. Default is 4. diff --git a/Docs/start-changes.md b/Docs/start-changes.md index a2da2648..1ef05e79 100644 --- a/Docs/start-changes.md +++ b/Docs/start-changes.md @@ -130,24 +130,3 @@ Updating JSON schema to the latest [specification 2020-12](https://json-schema.o ### Documentation Updates Reorganized the documentation to make it easier to find information. Added a new section on how to use the starter kit and how to use the Microsoft release flow. - -### Code Cleanup - -Ongoing cleanup of code: Removed unused code and improved code quality. - -### Performance - -Multiple lengthy sections of the code have been converted to parallel execution to improve performance. The change maybe ineffective if you limit the CI/CD agent to a single vCore or use the Azure DevOps provided CI/CD agents. - -The scripts `Build-DeploymentPlan`, `Deploy-PolicyPlan`, and `Build-PolicyDocumentation` have a new parameter `VirtualCores` to control the number of parallel threads and allowing you to optimize your performance. The code applies the following formula to adjust the `For-Each -Parallel` throttle limits (threads) based on the number of VirtualCores. - -- Threads = 1 x VirtualCores for pre-processing (pure compute) Policy and Policy Set parameters during Policy Assignment plan calculations -- Threads = 2 x VirtualCores for Policy object deployment since it executes many REST calls to the Azure resource manager and therefore spends much of its time waiting on I/O. -- Threads = 4 (fixed) for reading and processing Policy resources; one each for - - Policy definitions - - Policy Set definitions - - Policy Assignments, Role Assignments, and Role Definitions - - Policy Exemptions - -Setting VirtualCores to zero (0) disables parallel processing. The default value is 4. EPAC also uses a minimum chunk size for deployments to avoid unnecessary overhead for small number of items. - diff --git a/Scripts/Deploy/Build-DeploymentPlans.ps1 b/Scripts/Deploy/Build-DeploymentPlans.ps1 index f9ad6516..aa5488e4 100644 --- a/Scripts/Deploy/Build-DeploymentPlans.ps1 +++ b/Scripts/Deploy/Build-DeploymentPlans.ps1 @@ -19,9 +19,6 @@ .PARAMETER DevOpsType If set, outputs variables consumable by conditions in a DevOps pipeline. Valid values are '', 'ado' and 'gitlab'. -.PARAMETER VirtualCores - Number of virtual cores available to calculate the deployment plan. Defaults to 4. - .EXAMPLE .\Build-DeploymentPlans.ps1 -PacEnvironmentSelector "dev" @@ -58,15 +55,19 @@ param ( [ValidateSet("ado", "gitlab", "")] [string] $DevOpsType = "", - [Parameter(HelpMessage = "Number of virtual cores available to calculate the deployment plan. Defaults to 4. A value of 0 disables parallel processing.")] - [Int16] $VirtualCores = 4 - + [Parameter(HelpMessage = "Deprecated.")] + [Int16] $VirtualCores = 0 ) $PSDefaultParameterValues = @{ "Write-Information:InformationVariable" = "+global:epacInfoStream" } +if ($VirtualCores -gt 0) { + Write-Warning "VirtualCores parameter is deprecated. parallel processing is no longer supported. Please remove the parameter!" -WarningAction Continue + $VirtualCores = 0 +} + Clear-Variable -Name epacInfoStream -Scope global -Force -ErrorAction SilentlyContinue # Dot Source Helper Scripts @@ -240,14 +241,11 @@ if ($buildSelections.buildAny) { $scopeTable = Build-ScopeTableForDeploymentRootScope -PacEnvironment $pacEnvironment $skipExemptions = -not $buildSelections.buildPolicyExemptions $skipRoleAssignments = -not $buildSelections.buildPolicyAssignments - $NoParallelProcessing = $VirtualCores -eq 0 - # $NoParallelProcessing = $true # for debugging, disable parallel processing $deployedPolicyResources = Get-AzPolicyResources ` -PacEnvironment $pacEnvironment ` -ScopeTable $scopeTable ` -SkipExemptions:$skipExemptions ` - -SkipRoleAssignments:$skipRoleAssignments ` - -NoParallelProcessing:$NoParallelProcessing + -SkipRoleAssignments:$skipRoleAssignments # Calculate roleDefinitionIds for built-in and inherited Policies $readOnlyPolicyDefinitions = $deployedPolicyResources.policydefinitions.readOnly @@ -318,7 +316,7 @@ if ($buildSelections.buildAny) { $combinedPolicyDetails = Convert-PolicyResourcesToDetails ` -AllPolicyDefinitions $allDefinitions.policydefinitions ` -AllPolicySetDefinitions $allDefinitions.policysetdefinitions ` - -VirtualCores $VirtualCores + -VirtualCores 4 # Populate allAssignments $deployedPolicyAssignments = $deployedPolicyResources.policyassignments.managed diff --git a/Scripts/Deploy/Deploy-PolicyPlan.ps1 b/Scripts/Deploy/Deploy-PolicyPlan.ps1 index 88e20b38..a7be3203 100644 --- a/Scripts/Deploy/Deploy-PolicyPlan.ps1 +++ b/Scripts/Deploy/Deploy-PolicyPlan.ps1 @@ -16,9 +16,6 @@ .PARAMETER Interactive Use switch to indicate interactive use -.PARAMETER VirtualCores - Number of virtual cores available to deploy Policy objects in parallel. Defaults to 4. - .EXAMPLE Deploy-PolicyPlan.ps1 -PacEnvironmentSelector "dev" -DefinitionsRootFolder "C:\git\policy-as-code\Definitions" -InputFolder "C:\git\policy-as-code\Output" -Interactive Deploys Policy resources from a plan file. @@ -49,10 +46,15 @@ param ( [Parameter(HelpMessage = "Use switch to indicate interactive use")] [switch] $Interactive, - [Parameter(HelpMessage = "Number of virtual cores available to deploy Policy objects in parallel. Defaults to 4.")] - [Int16] $VirtualCores = 4 + [Parameter(HelpMessage = "Deprecated.")] + [Int16] $VirtualCores = 0 ) +if ($VirtualCores -gt 0) { + Write-Warning "VirtualCores parameter is deprecated. parallel processing is no longer supported. Please remove the parameter!" -WarningAction Continue + $VirtualCores = 0 +} + $PSDefaultParameterValues = @{ "Write-Information:InformationVariable" = "+global:epacInfoStream" } @@ -65,7 +67,6 @@ Clear-Variable -Name epacInfoStream -Scope global -Force -ErrorAction SilentlyCo $InformationPreference = "Continue" $pacEnvironment = Select-PacEnvironment $PacEnvironmentSelector -DefinitionsRootFolder $DefinitionsRootFolder -InputFolder $InputFolder -Interactive $Interactive $null = Set-AzCloudTenantSubscription -Cloud $pacEnvironment.cloud -TenantId $pacEnvironment.tenantId -Interactive $pacEnvironment.interactive -DeploymentDefaultContext $pacEnvironment.defaultContext -$throttleLimit = $VirtualCores * 2 # Telemetry if ($pacEnvironment.telemetryEnabled) { @@ -101,28 +102,10 @@ else { Write-Information "===================================================================================================" Write-Information "Delete orphaned, deleted, expired and replaced Exemptions ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyExemptions - } - } - else { - $funcRemoveAzResourceByIdRestMethod = ${function:Remove-AzResourceByIdRestMethod}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Remove-AzResourceByIdRestMethod}) { - ${function:Remove-AzResourceByIdRestMethod} = $using:funcRemoveAzResourceByIdRestMethod - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyExemptions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Write-Information "$($entry.displayName) - $($id)" + Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyExemptions } } @@ -133,28 +116,10 @@ else { Write-Information "===================================================================================================" Write-Information "Delete removed and replaced Assignments ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyAssignments - } - } - else { - $funcRemoveAzResourceByIdRestMethod = ${function:Remove-AzResourceByIdRestMethod}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Remove-AzResourceByIdRestMethod}) { - ${function:Remove-AzResourceByIdRestMethod} = $using:funcRemoveAzResourceByIdRestMethod - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyAssignments - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Write-Information "$($entry.displayName) - $($id)" + Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyAssignments } } @@ -165,28 +130,10 @@ else { Write-Information "===================================================================================================" Write-Information "Delete removed and replaced Policy Sets ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions - } - } - else { - $funcRemoveAzResourceByIdRestMethod = ${function:Remove-AzResourceByIdRestMethod}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Remove-AzResourceByIdRestMethod}) { - ${function:Remove-AzResourceByIdRestMethod} = $using:funcRemoveAzResourceByIdRestMethod - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Write-Information "$($entry.displayName) - $($id)" + Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions } } @@ -196,28 +143,10 @@ else { Write-Information "===================================================================================================" Write-Information "Delete replaced Policies ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } - else { - $funcRemoveAzResourceByIdRestMethod = ${function:Remove-AzResourceByIdRestMethod}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Remove-AzResourceByIdRestMethod}) { - ${function:Remove-AzResourceByIdRestMethod} = $using:funcRemoveAzResourceByIdRestMethod - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Write-Information "$($entry.displayName) - $($id)" + Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions } } @@ -231,28 +160,9 @@ else { Write-Information "===================================================================================================" Write-Information "Create and update Policies ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Set-AzPolicyDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } - else { - $funcSetAzPolicyDefinitionRestMethod = ${function:Set-AzPolicyDefinitionRestMethod}.ToString() - $funcRemoveNullFields = ${function:Remove-NullFields}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Set-AzPolicyDefinitionRestMethod}) { - ${function:Set-AzPolicyDefinitionRestMethod} = $using:funcSetAzPolicyDefinitionRestMethod - ${function:Remove-NullFields} = $using:funcRemoveNullFields - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Set-AzPolicyDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Set-AzPolicyDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policyDefinitions } } @@ -264,28 +174,9 @@ else { Write-Information "===================================================================================================" Write-Information "Create and update Policy Sets ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Set-AzPolicySetDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions - } - } - else { - $funcSetAzPolicySetDefinitionRestMethod = ${function:Set-AzPolicySetDefinitionRestMethod}.ToString() - $funcRemoveNullFields = ${function:Remove-NullFields}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Set-AzPolicySetDefinitionRestMethod}) { - ${function:Set-AzPolicySetDefinitionRestMethod} = $using:funcSetAzPolicySetDefinitionRestMethod - ${function:Remove-NullFields} = $using:funcRemoveNullFields - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Set-AzPolicySetDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Set-AzPolicySetDefinitionRestMethod -Definition $entry -ApiVersion $pacEnvironment.apiVersions.policySetDefinitions } } @@ -296,28 +187,10 @@ else { Write-Information "===================================================================================================" Write-Information "Delete Policies ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Write-Information $entry.displayName - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } - else { - $funcRemoveAzResourceByIdRestMethod = ${function:Remove-AzResourceByIdRestMethod}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Remove-AzResourceByIdRestMethod}) { - ${function:Remove-AzResourceByIdRestMethod} = $using:funcRemoveAzResourceByIdRestMethod - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Write-Information "$($entry.displayName) - $($id)" - Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Write-Information $entry.displayName + Remove-AzResourceByIdRestMethod -Id $id -ApiVersion $pacEnvironment.apiVersions.policyDefinitions } } @@ -329,28 +202,9 @@ else { Write-Information "===================================================================================================" Write-Information "Create and update Assignments ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($id in $chunk.Keys) { - $entry = $chunk.$id - Set-AzPolicyAssignmentRestMethod -Assignment $entry -ApiVersion $pacEnvironment.apiVersions.policyAssignments - } - } - else { - $funcSetAzPolicyAssignmentRestMethod = ${function:Set-AzPolicyAssignmentRestMethod}.ToString() - $funcGetDeepClone = ${function:Get-ClonedObject}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Set-AzPolicyAssignmentRestMethod}) { - ${function:Set-AzPolicyAssignmentRestMethod} = $using:funcSetAzPolicyAssignmentRestMethod - ${function:Get-ClonedObject} = $using:funcGetDeepClone - } - $pacEnvironment = $using:pacEnvironment - foreach ($id in $_.Keys) { - $entry = $_.$id - Set-AzPolicyAssignmentRestMethod -Assignment $entry -ApiVersion $pacEnvironment.apiVersions.policyAssignments - } - } + foreach ($id in $table.Keys) { + $entry = $table.$id + Set-AzPolicyAssignmentRestMethod -Assignment $entry -ApiVersion $pacEnvironment.apiVersions.policyAssignments } } @@ -362,28 +216,9 @@ else { Write-Information "===================================================================================================" Write-Information "Create and update Exemptions ($($table.psbase.Count))" Write-Information "---------------------------------------------------------------------------------------------------" - $chunks = Split-HashtableIntoChunks -Table $table -NumberOfChunks $throttleLimit -MinChunkingSize 10 - if ($chunks.psbase.Count -eq 1) { - $chunk = $chunks[0] - foreach ($exemptionId in $chunk.Keys) { - $entry = $chunk.$exemptionId - Set-AzPolicyExemptionRestMethod -ExemptionObj $entry -ApiVersion $pacEnvironment.apiVersions.policyExemptions - } - } - else { - $funcSetAzPolicyExemptionRestMethod = ${function:Set-AzPolicyExemptionRestMethod}.ToString() - $funcRemoveNullFields = ${function:Remove-NullFields}.ToString() - $chunks | ForEach-Object -ThrottleLimit $throttleLimit -Parallel { - if ($null -eq ${function:Set-AzPolicyExemptionRestMethod}) { - ${function:Set-AzPolicyExemptionRestMethod} = $using:funcSetAzPolicyExemptionRestMethod - ${function:Remove-NullFields} = $using:funcRemoveNullFields - } - $pacEnvironment = $using:pacEnvironment - foreach ($exemptionId in $_.Keys) { - $entry = $_.$exemptionId - Set-AzPolicyExemptionRestMethod -ExemptionObj $entry -ApiVersion $pacEnvironment.apiVersions.policyExemptions - } - } + foreach ($exemptionId in $table.Keys) { + $entry = $table.$exemptionId + Set-AzPolicyExemptionRestMethod -ExemptionObj $entry -ApiVersion $pacEnvironment.apiVersions.policyExemptions } } Write-Information "" diff --git a/Scripts/Helpers/Add-ErrorMessage.ps1 b/Scripts/Helpers/Add-ErrorMessage.ps1 index 835476d9..fa09b64c 100644 --- a/Scripts/Helpers/Add-ErrorMessage.ps1 +++ b/Scripts/Helpers/Add-ErrorMessage.ps1 @@ -7,10 +7,10 @@ function Add-ErrorMessage { ) if ($EntryNumber -ne -1) { - $null = $ErrorInfo.errorStrings.Add(" $($EntryNumber): $ErrorString") + $null = $ErrorInfo.errorStrings.Add("$($EntryNumber): $ErrorString") } else { - $null = $ErrorInfo.errorStrings.Add(" $ErrorString") + $null = $ErrorInfo.errorStrings.Add("$ErrorString") } $ErrorInfo.errorsInFile++ $ErrorInfo.hasErrors = $true diff --git a/Scripts/Helpers/Build-AssignmentDefinitionAtLeaf.ps1 b/Scripts/Helpers/Build-AssignmentDefinitionAtLeaf.ps1 index 225d0f59..b987fc82 100644 --- a/Scripts/Helpers/Build-AssignmentDefinitionAtLeaf.ps1 +++ b/Scripts/Helpers/Build-AssignmentDefinitionAtLeaf.ps1 @@ -516,6 +516,10 @@ function Build-AssignmentDefinitionAtLeaf { $parameterObject = $null $parametersInPolicyDefinition = @{} + if ($displayName -eq "Allowed Locations") { + $null = $null + } + if ($isPolicySet) { $parametersInPolicyDefinition = $policySetDetails.parameters if ($useCsv) { diff --git a/Scripts/Helpers/Build-AssignmentParameterObject.ps1 b/Scripts/Helpers/Build-AssignmentParameterObject.ps1 index 3b48aa6a..25851e96 100644 --- a/Scripts/Helpers/Build-AssignmentParameterObject.ps1 +++ b/Scripts/Helpers/Build-AssignmentParameterObject.ps1 @@ -12,7 +12,7 @@ function Build-AssignmentParameterObject { $assignmentParameterValue = $AssignmentParameters.$parameterName if ($null -eq $assignmentParameterValue) { # Incorrect case last chance to match - $assignmentParameterValue = $AssignmentParameters.Keys | Where-Object { $_.ToLower() -eq $parameterName.ToLower() } | ForEach-Object { $AssignmentParameters.$_ } + $assignmentParameterValue = $AssignmentParameters.Keys | Where-Object { $_ -eq $parameterName } | ForEach-Object { $AssignmentParameters.$_ } } $parameterDefinition = $ParametersInPolicyDefinition.$parameterName $defaultValue = $parameterDefinition.defaultValue diff --git a/Scripts/Helpers/Build-AssignmentPlan.ps1 b/Scripts/Helpers/Build-AssignmentPlan.ps1 index acbc00e2..cd7232d8 100644 --- a/Scripts/Helpers/Build-AssignmentPlan.ps1 +++ b/Scripts/Helpers/Build-AssignmentPlan.ps1 @@ -136,7 +136,7 @@ function Build-AssignmentPlan { $metadataMatches, $changePacOwnerId = Confirm-MetadataMatches ` -ExistingMetadataObj $deployedPolicyAssignmentProperties.metadata ` -DefinedMetadataObj $metadata - $enforcementModeMatches = $enforcementMode -eq $deployedPolicyAssignmentProperties.EnforcementMode + $enforcementModeMatches = $enforcementMode -eq $deployedPolicyAssignmentProperties.enforcementMode $nonComplianceMessagesMatches = Confirm-ObjectValueEqualityDeep ` $deployedPolicyAssignmentProperties.nonComplianceMessages ` $nonComplianceMessages diff --git a/Scripts/Helpers/Build-ExemptionsPlan.ps1 b/Scripts/Helpers/Build-ExemptionsPlan.ps1 index b2ce5e6e..d87aefa7 100644 --- a/Scripts/Helpers/Build-ExemptionsPlan.ps1 +++ b/Scripts/Helpers/Build-ExemptionsPlan.ps1 @@ -888,7 +888,7 @@ function Build-ExemptionsPlan { #endregion process each row if ($errorInfo.hasErrors) { - Write-ErrorsFromErrorInfo -ErrorInfo $errorInfo + Write-ErrorsFromErrorInfo -ErrorInfo $errorInfo -ErrorAction Continue $numberOfFilesWithErrors++ continue } diff --git a/Scripts/Helpers/Confirm-ObjectValueEqualityDeep.ps1 b/Scripts/Helpers/Confirm-ObjectValueEqualityDeep.ps1 index caf02380..c0a74d78 100644 --- a/Scripts/Helpers/Confirm-ObjectValueEqualityDeep.ps1 +++ b/Scripts/Helpers/Confirm-ObjectValueEqualityDeep.ps1 @@ -8,13 +8,11 @@ function Confirm-ObjectValueEqualityDeep { $Object2 ) - - if ($Object1 -eq $Object2) { - # $Object1 and $Object2 are equal (includes both $null) - return $true - } - elseif ($null -eq $Object1 -or $null -eq $Object2) { - # $Object1 and $Object2 are not equal, but one of them is $null + if ($null -eq $Object1 -or $null -eq $Object2) { + if ($Object1 -eq $Object2) { + # $Object1 and $Object2 are both $null + return $true + } if ($null -eq $Object1) { # $Object1 is $null, swap $Object1 and $Object2 to ensure that Object1 (the old Object2) is not $null and Object2 (the old Object1) is $null (setting it to $null is ommited because it is not used in the subsequent code) $Object1 = $Object2 @@ -26,7 +24,7 @@ function Confirm-ObjectValueEqualityDeep { return $Object1.Count -eq 0 } elseif ($Object1 -is [string]) { - # $Object1 is a string, if it is empty or only conatins whitespace treat it as $null and therefore equal to Object2 + # $Object1 is a string, if it is empty or only contains whitespace treat it as $null and therefore equal to Object2 return [string]::IsNullOrWhiteSpace($Object1) } else { @@ -54,7 +52,7 @@ function Confirm-ObjectValueEqualityDeep { $foundMatch = $false for ($i = 0; $i -lt $object2List.Count; $i++) { $item2 = $object2List[$i] - if ($item1 -eq $item2 -or (Confirm-ObjectValueEqualityDeep $item1 $item2)) { + if (Confirm-ObjectValueEqualityDeep $item1 $item2) { # if either the array item values are equal or a deep inspection shows equal, continue to the next item by: # 1. Setting $foundMatch to $true # 2. Remove the matching item from the Object2 list, therefore reducing the computing complexity of the next iteration @@ -73,23 +71,30 @@ function Confirm-ObjectValueEqualityDeep { return $true } } + elseif ($Object1 -eq $Object2) { + # $Object1 and $Object2 are the same object; not correct for a list (IList) + # therefore check fo a list be first + # @("S") -eq @("T","S","N") is an empty array => false if used in an if + # @("T","S","N") -eq @("S") is @("S") => true if used in an if + return $true + } elseif ($Object1 -is [datetime] -or $Object2 -is [datetime]) { # $Object1 or $Object2 is a DateTime # Note: this must be done prior to the next test, since [DateTime] is a [System.ValueType] $dateString1 = $Object1 $dateString2 = $Object2 - if ($typeName1 -eq "DateTime") { + if ($typeName1 -is [datetime]) { $dateString1 = $Object1.ToString("yyyy-MM-dd") } - if ($typeName2 -eq "DateTime") { + if ($typeName2 -is [datetime]) { $dateString2 = $Object2.ToString("yyyy-MM-dd") } return $dateString1 -eq $dateString2 } - elseif ($Object1 -is [string] -or $Object2 -is [string] -or $Object1 -is [System.ValueType] -or $Object2 -is [System.ValueType]) { - # Will have caused $true by the first "if" statement if they match - # Note: this must be done prior to the next test, since [string and [System.ValueType] are both [PSCustomObject] (PSCustomObject is PowerShells version of [object]) - return $false + elseif ($Object1 -is [string] -or $Object2 -is [string]) { + # Will have caused $true by the second "if" statement if they match + # Note: this must be done prior to the next test, since [string and [System.ValueType] + # are both [PSCustomObject] (PSCustomObject is PowerShells version of [object]) } elseif (($Object1 -is [System.Collections.IDictionary] -or $Object1 -is [psobject]) ` -and ($Object2 -is [System.Collections.IDictionary] -or $Object2 -is [psobject])) { @@ -109,8 +114,8 @@ function Confirm-ObjectValueEqualityDeep { else { $normalizedKeys2 = $Object2.PSObject.Properties.Name } - $key1IsNotArray = $normalizedKeys1 -isnot [array] - $key2IsNotArray = $normalizedKeys2 -isnot [array] + $key1IsNotArray = $normalizedKeys1 -isnot [System.Collections.ICollection] + $key2IsNotArray = $normalizedKeys2 -isnot [System.Collections.ICollection] if ($key1IsNotArray) { $normalizedKeys1 = @($normalizedKeys1) } @@ -123,7 +128,7 @@ function Confirm-ObjectValueEqualityDeep { # if there are no keys, return true return $true } - if ($uniqueKeys -isnot [array]) { + if ($uniqueKeys -isnot [System.Collections.ICollection]) { $uniqueKeys = @($uniqueKeys) } @@ -132,7 +137,7 @@ function Confirm-ObjectValueEqualityDeep { $item1 = $Object1.$key $item2 = $Object2.$key - if ($item1 -eq $item2 -or (Confirm-ObjectValueEqualityDeep $item1 $item2)) { + if (Confirm-ObjectValueEqualityDeep $item1 $item2) { # if either the property values are equal or a deep inspection shows equal, continue to the next property } else { @@ -144,7 +149,6 @@ function Confirm-ObjectValueEqualityDeep { return $true } else { - # equality of other types handled in the then clause of the outer else clause return $false } } diff --git a/Scripts/Helpers/Confirm-ParametersDefinitionMatch.ps1 b/Scripts/Helpers/Confirm-ParametersDefinitionMatch.ps1 index 5d190a30..cf8178f5 100644 --- a/Scripts/Helpers/Confirm-ParametersDefinitionMatch.ps1 +++ b/Scripts/Helpers/Confirm-ParametersDefinitionMatch.ps1 @@ -7,52 +7,44 @@ function Confirm-ParametersDefinitionMatch { $match = $true $incompatible = $false - $existingParameters = ConvertTo-HashTable $ExistingParametersObj - $definedParameters = ConvertTo-HashTable $DefinedParametersObj - $addedParameters = Get-ClonedObject $definedParameters -AsHashTable -AsShallowClone + $existingParameters = Get-ClonedObject $ExistingParametersObj -AsHashTable + $definedParameters = Get-ClonedObject $DefinedParametersObj -AsHashTable + $addedParameters = Get-ClonedObject $definedParameters -AsHashTable foreach ($existingParameterName in $existingParameters.Keys) { - if ($definedParameters.Keys -contains $existingParameterName) { - # Remove key from $addedParameters - $addedParameters.Remove($existingParameterName) + # ignore paramer name case + $definedParameterNameArray = $definedParameters.Keys -eq $existingParameterName + if ($definedParameterNameArray.Count -gt 0) { + # found a matching parameter name (case insensitive) + $definedParameterName = $definedParameterNameArray[0] + $addedParameters.Remove($definedParameterName) $existing = $existingParameters.$existingParameterName - $defined = $definedParameters.$existingParameterName - - # Analyze parameter defaultValue - $thisMatch = Confirm-ObjectValueEqualityDeep $existing.defaultValue $defined.defaultValue - if (!$thisMatch) { - $match = $false - if ($null -eq $defined.defaultValue) { - $incompatible = $true - break - } - } - - # Analyze parameter allowedValues - $thisMatch = Confirm-ObjectValueEqualityDeep $existing.allowedValues $defined.allowedValues - if (!$thisMatch) { - $match = $false - if ($null -eq $defined.defaultValue) { - $incompatible = $true - break - } + $defined = $definedParameters.$definedParameterName + $thisMatch = Confirm-ObjectValueEqualityDeep $existing $defined + if ($thisMatch) { + continue } + $match = $false - # Analyze type + # analyze parameter type if ($existing.type -ne $defined.type) { $match = $false $incompatible = $true break } + # analyze parameter metadata strongType $existingMetadata = $existing.metadata $definedMetadata = $defined.metadata - $thisMatch = Confirm-ObjectValueEqualityDeep $existingMetadata $definedMetadata + if ($existingMetadata.strongType -ne $definedMetadata.strongType) { + $incompatible = $true + break + } + + # analyze parameter allowedValues + $thisMatch = Confirm-ObjectValueEqualityDeep $existing.allowedValues $defined.allowedValues if (!$thisMatch) { - $match = $false - if ($existingMetadata.strongType -ne $definedMetadata.strongType) { - $incompatible = $true - break - } + $incompatible = $true + break } } else { @@ -62,9 +54,10 @@ function Confirm-ParametersDefinitionMatch { break } } - if ((-not $incompatible) -and ($addedParameters.psbase.Count -gt 0)) { + + if ($match -and !$incompatible -and ($addedParameters.psbase.Count -gt 0)) { $match = $false - # If no defaultValue, added parameter makes it incompatible requiring a delete followed by a new. + # added parameter without defaultValue is and incompatible change foreach ($addedParameterName in $addedParameters.Keys) { $added = $addedParameters.$addedParameterName if ($null -eq $added.defaultValue) { @@ -74,5 +67,9 @@ function Confirm-ParametersDefinitionMatch { } } + if (!$match) { + Write-Verbose "Parameters definition mismatch detected, incompatible: $incompatible" + } + return $match, $incompatible } diff --git a/Scripts/Helpers/Confirm-ParametersUsageMatches.ps1 b/Scripts/Helpers/Confirm-ParametersUsageMatches.ps1 index c92a9194..6ab85691 100644 --- a/Scripts/Helpers/Confirm-ParametersUsageMatches.ps1 +++ b/Scripts/Helpers/Confirm-ParametersUsageMatches.ps1 @@ -9,50 +9,33 @@ function Confirm-ParametersUsageMatches { $existingParameters = ConvertTo-HashTable $ExistingParametersObj $definedParameters = ConvertTo-HashTable $DefinedParametersObj + $allKeys = $existingParameters.Keys + $definedParameters.Keys if ($existingParameters.psbase.Count -ne $definedParameters.psbase.Count) { # parameter count changed return $false } $uniqueKeys = $allKeys | Sort-Object -Unique - if ($existingParameters.psbase.Count -ne $uniqueKeys.Count) { + if ($existingParameters.psbase.Count -ne $uniqueKeys.Count -or $definedParameters.psbase.Count -ne $uniqueKeys.Count) { # parameter names do not match return $false } foreach ($existingParameterName in $existingParameters.Keys) { - $existingParameter = $existingParameters.$existingParameterName - $definedParameter = $definedParameters.$existingParameterName - if ($null -eq $existingParameter) { - # maybe case of key does not match, find key for existingParameters without considering case - $key1 = $existingParameters.Keys | Where-Object { $_.ToLower() -eq $existingParameterName.ToLower() } - if ($null -ne $key1) { - Write-Debug "key '$existingParameterName' exists with a different case '$key1' in Object1 '$($existingParameters | ConvertTo-Json -Depth 100 -Compress)'" - $existingParameter = $existingParameters.$key1 - } - else { - # this is a coding error - Write-Error "Code bug: key '$existingParameterName' does not exist in existingParameters '$($existingParameters | ConvertTo-Json -Depth 100 -Compress)'" -ErrorAction Stop - } - } - if ($null -eq $definedParameter) { - # maybe case of key does not match, find key for definedParameters without considering case - $key2 = $definedParameters.Keys | Where-Object { $_.ToLower() -eq $existingParameterName.ToLower() } - if ($null -ne $key2) { - Write-Debug "key '$existingParameterName' exists with a different case '$key2' in Object2 '$($definedParameters | ConvertTo-Json -Depth 100 -Compress)'" - $definedParameter = $definedParameters.$key2 - } - else { - # this is a coding error - Write-Error "Code bug: key '$existingParameterName' does not exist in definedParameters '$($definedParameters | ConvertTo-Json -Depth 100 -Compress)'" -ErrorAction Stop - } + $definedParameterNameArray = $definedParameters.Keys -eq $existingParameterName + if ($definedParameterNameArray.Count -eq 0) { + # No matching parameter name found (case insensitive) + return $false } + $definedParameterName = $definedParameterNameArray[0] + $existingParameter = $existingParameters.$existingParameterName + $definedParameter = $definedParameters.$definedParameterName $existingParameterValue = $existingParameter - if ($CompareValueEntryForExistingParametersObj) { + if ($null -ne $existingParameterValue.value) { $existingParameterValue = $existingParameter.value } $definedParameterValue = $definedParameter - if ($CompareValueEntryForDefinedParametersObj) { + if ($null -ne $definedParameterValue.value) { $definedParameterValue = $definedParameter.value } diff --git a/Scripts/Helpers/Get-AzPolicyResources.ps1 b/Scripts/Helpers/Get-AzPolicyResources.ps1 index 70a5038f..a03c299b 100644 --- a/Scripts/Helpers/Get-AzPolicyResources.ps1 +++ b/Scripts/Helpers/Get-AzPolicyResources.ps1 @@ -6,8 +6,7 @@ function Get-AzPolicyResources { [switch] $SkipRoleAssignments, [switch] $SkipExemptions, - [switch] $CollectAllPolicies, - [switch] $NoParallelProcessing + [switch] $CollectAllPolicies ) $deploymentRootScope = $PacEnvironment.deploymentRootScope @@ -106,125 +105,40 @@ function Get-AzPolicyResources { "policyExemptions")) } - $deployedPolicyDefinitions = $deployedPolicyResources.policydefinitions - $deployedPolicySetDefinitions = $deployedPolicyResources.policysetdefinitions - if ($NoParallelProcessing) { - foreach ($collectionItem in $collectionList) { - switch ($collectionItem) { - policyDefinitions { - Get-AzPolicyOrSetDefinitions ` - -DefinitionType "policyDefinitions" ` - -PolicyResourcesTable $deployedPolicyResources.policydefinitions ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -CollectAllPolicies $collectAllPoliciesLocal - break - } - policySetDefinitions { - Get-AzPolicyOrSetDefinitions ` - -DefinitionType "policySetDefinitions" ` - -PolicyResourcesTable $deployedPolicyResources.policysetdefinitions ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -CollectAllPolicies $collectAllPoliciesLocal - break - } - policyAssignments { - Get-AzPolicyAssignments ` - -DeployedPolicyResources $deployedPolicyResources ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -SkipRoleAssignments $skipRoleAssignmentsLocal - break - } - policyExemptions { - Get-AzPolicyExemptions ` - -DeployedPolicyResources $deployedPolicyResources ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable - break - } + foreach ($collectionItem in $collectionList) { + switch ($collectionItem) { + policyDefinitions { + Get-AzPolicyOrSetDefinitions ` + -DefinitionType "policyDefinitions" ` + -PolicyResourcesTable $deployedPolicyResources.policydefinitions ` + -PacEnvironment $PacEnvironment ` + -ScopeTable $ScopeTable ` + -CollectAllPolicies $collectAllPoliciesLocal + break } - } - } - else { - $funcGetAzPolicyAssignments = ${function:Get-AzPolicyAssignments}.ToString() - $funcGetAzRoleAssignmentsRestMethod = ${function:Get-AzRoleAssignmentsRestMethod}.ToString() - $funcGetAzRoleDefinitionsRestMethod = ${function:Get-AzRoleDefinitionsRestMethod}.ToString() - $funcGetAzPolicyExemptions = ${function:Get-AzPolicyExemptions}.ToString() - $funcGetAzPolicyExemptionsRestMethod = ${function:Get-AzPolicyExemptionsRestMethod}.ToString() - $funcGetAzPolicyOrSetDefinitions = ${function:Get-AzPolicyOrSetDefinitions}.ToString() - $funcSearchAzGraphAllItems = ${function:Search-AzGraphAllItems}.ToString() - $funcDefGetClonedObject = ${function:Get-ClonedObject}.ToString() - $funcDefConfirmPolicyResourceExclusions = ${function:Confirm-PolicyResourceExclusions}.ToString() - $funcConfirmPacOwner = ${function:Confirm-PacOwner}.ToString() - $funcGetPolicyResourceProperties = ${function:Get-PolicyResourceProperties}.ToString() - $funcSplitAzPolicyResourceId = ${function:Split-AzPolicyResourceId}.ToString() - $funcConvertToHashTable = ${function:ConvertTo-HashTable}.ToString() - $funcSetUniqueRoleAssignmentScopes = ${function:Set-UniqueRoleAssignmentScopes}.ToString() - - $collectionList | Foreach-Object -Parallel { - # Import dot sourced functions into context - if ($null -eq ${function:Get-AzPolicyAssignments}) { - ${function:Get-AzPolicyAssignments} = $using:funcGetAzPolicyAssignments - ${function:Get-AzRoleAssignmentsRestMethod} = $using:funcGetAzRoleAssignmentsRestMethod - ${function:Get-AzRoleDefinitionsRestMethod} = $using:funcGetAzRoleDefinitionsRestMethod - ${function:Get-AzPolicyExemptions} = $using:funcGetAzPolicyExemptions - ${function:Get-AzPolicyExemptionsRestMethod} = $using:funcGetAzPolicyExemptionsRestMethod - ${function:Get-AzPolicyOrSetDefinitions} = $using:funcGetAzPolicyOrSetDefinitions - ${function:Search-AzGraphAllItems} = $using:funcSearchAzGraphAllItems - ${function:Get-ClonedObject} = $using:funcDefGetClonedObject - ${function:Confirm-PolicyResourceExclusions} = $using:funcDefConfirmPolicyResourceExclusions - ${function:Confirm-PacOwner} = $using:funcConfirmPacOwner - ${function:Get-PolicyResourceProperties} = $using:funcGetPolicyResourceProperties - ${function:Split-AzPolicyResourceId} = $using:funcSplitAzPolicyResourceId - ${function:ConvertTo-HashTable} = $using:funcConvertToHashTable - ${function:Set-UniqueRoleAssignmentScopes} = $using:funcSetUniqueRoleAssignmentScopes + policySetDefinitions { + Get-AzPolicyOrSetDefinitions ` + -DefinitionType "policySetDefinitions" ` + -PolicyResourcesTable $deployedPolicyResources.policysetdefinitions ` + -PacEnvironment $PacEnvironment ` + -ScopeTable $ScopeTable ` + -CollectAllPolicies $collectAllPoliciesLocal + break } - - #Action that will run in Parallel. Reference the current object via $PSItem and bring in outside variables with $USING:varname - $PacEnvironment = $using:PacEnvironment - $ScopeTable = $using:ScopeTable - $skipRoleAssignmentsLocal = $using:skipRoleAssignmentsLocal - $deployedPolicyResources = $using:deployedPolicyResources - $deployedPolicyDefinitions = $using:deployedPolicyDefinitions - $deployedPolicySetDefinitions = $using:deployedPolicySetDefinitions - $collectAllPoliciesLocal = $using:collectAllPoliciesLocal - - switch ($_) { - policyDefinitions { - Get-AzPolicyOrSetDefinitions ` - -DefinitionType "policyDefinitions" ` - -PolicyResourcesTable $deployedPolicyDefinitions ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -CollectAllPolicies $collectAllPoliciesLocal - break - } - policySetDefinitions { - Get-AzPolicyOrSetDefinitions ` - -DefinitionType "policySetDefinitions" ` - -PolicyResourcesTable $deployedPolicySetDefinitions ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -CollectAllPolicies $collectAllPoliciesLocal - break - } - policyAssignments { - Get-AzPolicyAssignments ` - -DeployedPolicyResources $deployedPolicyResources ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable ` - -SkipRoleAssignments $skipRoleAssignmentsLocal - break - } - policyExemptions { - Get-AzPolicyExemptions ` - -DeployedPolicyResources $deployedPolicyResources ` - -PacEnvironment $PacEnvironment ` - -ScopeTable $ScopeTable - break - } + policyAssignments { + Get-AzPolicyAssignments ` + -DeployedPolicyResources $deployedPolicyResources ` + -PacEnvironment $PacEnvironment ` + -ScopeTable $ScopeTable ` + -SkipRoleAssignments $skipRoleAssignmentsLocal + break + } + policyExemptions { + Get-AzPolicyExemptions ` + -DeployedPolicyResources $deployedPolicyResources ` + -PacEnvironment $PacEnvironment ` + -ScopeTable $ScopeTable + break } } } diff --git a/Scripts/Helpers/Get-AzPolicyResourcesDetails.ps1 b/Scripts/Helpers/Get-AzPolicyResourcesDetails.ps1 index cd9280ad..f3a3d05d 100644 --- a/Scripts/Helpers/Get-AzPolicyResourcesDetails.ps1 +++ b/Scripts/Helpers/Get-AzPolicyResourcesDetails.ps1 @@ -14,14 +14,12 @@ function Get-AzPolicyResourcesDetails { else { # New root scope found $scopeTable = Build-ScopeTableForDeploymentRootScope -PacEnvironment $PacEnvironment - $NoParallelProcessing = $VirtualCores -eq 0 # $NoParallelProcessing = $true $deployed = Get-AzPolicyResources ` -PacEnvironment $PacEnvironment ` -ScopeTable $scopeTable ` -SkipRoleAssignments ` - -SkipExemptions ` - -NoParallelProcessing:$NoParallelProcessing + -SkipExemptions $policyResourceDetails = Convert-PolicyResourcesToDetails ` -AllPolicyDefinitions $deployed.policydefinitions.all ` diff --git a/Scripts/Helpers/Get-ClonedObject.ps1 b/Scripts/Helpers/Get-ClonedObject.ps1 index 7d73745c..546f8414 100644 --- a/Scripts/Helpers/Get-ClonedObject.ps1 +++ b/Scripts/Helpers/Get-ClonedObject.ps1 @@ -10,22 +10,10 @@ function Get-ClonedObject { $clone = $InputObject if ($AsHashTable) { + # only support deep cloning to hashtable if ($null -ne $InputObject) { - if ($AsShallowClone) { - if ($InputObject -is [psobject] -and $InputObject -isnot [System.ValueType]) { - $clone = [ordered]@{} - foreach ($property in $InputObject.PSObject.Properties) { - $null = $clone.Add($property.Name, $property.Value) - } - } - elseif ($InputObject -is [System.ICloneable]) { - $clone = $InputObject.Clone() - } - } - else { - $json = ConvertTo-Json $InputObject -Depth 100 -Compress - $clone = ConvertFrom-Json $json -NoEnumerate -Depth 100 -AsHashTable - } + $json = ConvertTo-Json $InputObject -Depth 100 -Compress + $clone = ConvertFrom-Json $json -NoEnumerate -Depth 100 -AsHashTable } else { $clone = @{} diff --git a/Scripts/Helpers/Get-GlobalSettings.ps1 b/Scripts/Helpers/Get-GlobalSettings.ps1 index 7e4f78b6..1675e60d 100644 --- a/Scripts/Helpers/Get-GlobalSettings.ps1 +++ b/Scripts/Helpers/Get-GlobalSettings.ps1 @@ -41,112 +41,94 @@ function Get-GlobalSettings { [hashtable] $pacEnvironmentDefinitions = @{} $pacEnvironmentSelectors = [System.Collections.ArrayList]::new() - $hasErrors = $false $pacOwnerId = $settings.pacOwnerId + $errorInfo = New-ErrorInfo -FileName $globalSettingsFile if ($null -eq $pacOwnerId) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: does not contain the required pacOwnerId field. Add a pacOwnerId field with a GUID or other unique id!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: does not contain the required pacOwnerId field. Add a pacOwnerId field with a GUID or other unique id!" } if ($null -ne $settings.globalNotScopes) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: contains a deprecated globalNotScopes field. Move the values into each pacEnvironment!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: contains a deprecated globalNotScopes field. Move the values into each pacEnvironment!" } if ($null -ne $settings.managedIdentityLocations -or $null -ne $settings.managedIdentityLocation) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: contains a deprecated managedIdentityLocations field. Move the values into each pacEnvironment!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: contains a deprecated managedIdentityLocations field. Move the values into each pacEnvironment!" } $pacEnvironments = $settings.pacEnvironments if ($null -eq $pacEnvironments) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: does not contain a pacEnvironments array. Add a pacEnvironments array with at least one environment!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: does not contain a pacEnvironments array. Add a pacEnvironments array with at least one environment!" } elseif ($pacEnvironments -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironments must be an array of objects." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironments must be an array of objects." } elseif ($pacEnvironments.Count -eq 0) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironments array must contain at least one environment." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironments array must contain at least one environment." } else { foreach ($pacEnvironment in $pacEnvironments) { $pacSelector = $pacEnvironment.pacSelector if ($null -eq $pacSelector) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: a pacEnvironments array element does not contain the required pacSelector element." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: a pacEnvironments array element does not contain the required pacSelector element." } $null = $pacEnvironmentSelectors.Add($pacSelector) $cloud = $pacEnvironment.cloud if ($null -eq $cloud) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not define the required cloud element." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not define the required cloud element." } $tenantId = $pacEnvironment.tenantId if ($null -eq $tenantId) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain required tenantId field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain required tenantId field." } # Managed identity location $managedIdentityLocation = $pacEnvironment.managedIdentityLocation if ($null -eq $managedIdentityLocation) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain required managedIdentityLocation field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain required managedIdentityLocation field." } $managingTenantId = $pacEnvironment.managingTenant.managingTenantId $managingTenantRootScope = $pacEnvironment.managingTenant.managingTenantRootScope if ($null -ne $managingTenantId) { if ($null -eq $pacEnvironment.managingTenant.managingTenantRootScope) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector element managingTenantRootScope must have a valid value when managingTenantID has a value." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector element managingTenantRootScope must have a valid value when managingTenantID has a value." } $objectGuid = [System.Guid]::empty # Returns True if successfully parsed, otherwise returns False. $isGUID = [System.Guid]::TryParse($managingTenantId, [System.Management.Automation.PSReference]$objectGuid) if ($isGUID -ne $true) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field managingTenant ($managingTenantId) must be a GUID." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field managingTenant ($managingTenantId) must be a GUID." } } elseif ($null -ne $managingTenantRootScope) { if ($null -eq $managingTenantId) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector element managingTenantID must be a valid GUID when managingTenantRootScope has a value." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector element managingTenantID must be a valid GUID when managingTenantRootScope has a value." } } $defaultSubscriptionId = $pacEnvironment.defaultSubscriptionId if ($null -ne $defaultSubscriptionId) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector contains a deprecated defaultSubscriptionId. Remove it!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector contains a deprecated defaultSubscriptionId. Remove it!" } if ($null -ne $pacEnvironment.rootScope) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector contains a deprecated rootScope. Replace rootScope with deploymentRootScope containing a fully qualified scope id!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector contains a deprecated rootScope. Replace rootScope with deploymentRootScope containing a fully qualified scope id!" } if ($null -ne $pacEnvironment.inheritedDefinitionsScopes) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector contains a deprecated inheritedDefinitionsScopes. To cover the use case see https://aka.ms/epac/settings-desired-state.md#use-case-4-multiple-teams-in-a-hierarchical-organization!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector contains a deprecated inheritedDefinitionsScopes. To cover the use case see https://aka.ms/epac/settings-desired-state.md#use-case-4-multiple-teams-in-a-hierarchical-organization!" } $deploymentRootScope = $pacEnvironment.deploymentRootScope if ($null -eq $pacEnvironment.deploymentRootScope) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain deploymentRootScope field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain deploymentRootScope field." } $policyDefinitionsScopes = @( $deploymentRootScope, "") $defaultContext = $pacEnvironment.defaultContext if ($null -ne $defaultContext) { if ($pacEnvironment.defaultContext -isnot [string]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector has an invalid defaultContext field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector has an invalid defaultContext field." } } else { @@ -171,20 +153,18 @@ function Get-GlobalSettings { $pacEnvironmentGlobalNotScopes = $pacEnvironment.globalNotScopes if ($null -ne $pacEnvironmentGlobalNotScopes) { if ($pacEnvironmentGlobalNotScopes -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field globalNotScopes must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field globalNotScopes must be an array of strings." } else { foreach ($globalNotScope in $pacEnvironmentGlobalNotScopes) { if ($globalNotScope -isnot [string]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field globalNotScopes must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field globalNotScopes must be an array of strings." } elseif ($globalNotScope.Contains("/resourceGroupPatterns/", [System.StringComparison]::OrdinalIgnoreCase)) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field globalNotScopes entry ($globalNotScope) must not contain deprecated /resourceGroupPatterns/.`n`rReplace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/`"" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field globalNotScopes entry ($globalNotScope) must not contain deprecated /resourceGroupPatterns/.`n`r`t`tReplace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/`"" } else { + $null = $globalNotScopesList.Add($globalNotScope) $null = $excludedScopesList.Add($globalNotScope) if ($globalNotScope.StartsWith("/subscriptions/")) { if ($globalNotScope.Contains("/resourceGroups/", [System.StringComparison]::OrdinalIgnoreCase)) { @@ -201,8 +181,7 @@ function Get-GlobalSettings { $null = $globalNotScopesManagementGroupsList.Add($globalNotScope) } else { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field globalNotScopes entry ($globalNotScope) must be a valid scope." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field globalNotScopes entry ($globalNotScope) must be a valid scope." } } } @@ -224,40 +203,33 @@ function Get-GlobalSettings { $desired = $pacEnvironment.desiredState if ($null -eq $desired) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain required desiredState field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain required desiredState field." } else { $strategy = $desired.strategy if ($null -eq $strategy) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain required desiredState.strategy field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain required desiredState.strategy field." } else { $valid = @("full", "ownedOnly") if ($strategy -notin $valid) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.strategy ($strategy) must be one of $(ConvertTo-Json $valid -Compress)." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.strategy ($strategy) must be one of $(ConvertTo-Json $valid -Compress)." } - $desiredState.strategy = $strategy } $includeResourceGroups = $desired.includeResourceGroups if ($null -ne $includeResourceGroups) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.includeResourceGroups is deprecated.`n`rIf set to false, replace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/*`"" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.includeResourceGroups is deprecated.`n`r`t`tIf set to false, replace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/*`"" } $keepDfcSecurityAssignments = $desired.keepDfcSecurityAssignments if ($null -eq $keepDfcSecurityAssignments) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector does not contain required desiredState.keepDfcSecurityAssignments field." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector does not contain required desiredState.keepDfcSecurityAssignments field." } else { if ($keepDfcSecurityAssignments -is [bool]) { $desiredState.keepDfcSecurityAssignments = $keepDfcSecurityAssignments } else { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.keepDfcSecurityAssignments ($keepDfcSecurityAssignments) must be a boolean value." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.keepDfcSecurityAssignments ($keepDfcSecurityAssignments) must be a boolean value." } } $cleanupObsoleteExemptions = $desired.cleanupObsoleteExemptions @@ -266,21 +238,18 @@ function Get-GlobalSettings { $desiredState.cleanupObsoleteExemptions = $cleanupObsoleteExemptions } else { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.cleanupObsoleteExemptions ($cleanupObsoleteExemptions) must be a boolean value." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.cleanupObsoleteExemptions ($cleanupObsoleteExemptions) must be a boolean value." } } $excludedScopes = $desired.excludedScopes if ($null -ne $excludedScopes) { if ($excludedScopes -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedScopes must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedScopes must be an array of strings." } foreach ($excludedScope in $excludedScopes) { if ($null -ne $excludedScope -and $excludedScope -is [string] -and $excludedScope -ne "") { if ($excludedScope.Contains("/resourceGroupPatterns/", [System.StringComparison]::OrdinalIgnoreCase)) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must not contain deprecated /resourceGroupPatterns/.`n`rReplace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/`"" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must not contain deprecated /resourceGroupPatterns/.`n`r`t`tReplace it with excludedScopes pattern `"/subscriptions/*/resourceGroups/`"" } else { $null = $excludedScopesList.Add($excludedScope) @@ -296,8 +265,7 @@ function Get-GlobalSettings { $null = $globalExcludedScopesManagementGroupsList.Add($excludedScope) } else { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must be a valid scope." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must be a valid scope." } } } @@ -306,36 +274,31 @@ function Get-GlobalSettings { $excluded = $desired.excludedPolicyDefinitions if ($null -ne $excluded) { if ($excluded -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedPolicyDefinitions must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedPolicyDefinitions must be an array of strings." } $desiredState.excludedPolicyDefinitions = $excluded } $excluded = $desired.excludedPolicySetDefinitions if ($null -ne $excluded) { if ($excluded -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedPolicySetDefinitions must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedPolicySetDefinitions must be an array of strings." } $desiredState.excludedPolicySetDefinitions = $excluded } $excluded = $desired.excludedPolicyAssignments if ($null -ne $excluded) { if ($excluded -isnot [array]) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.excludedPolicyAssignments must be an array of strings." - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedPolicyAssignments must be an array of strings." } $desiredState.excludedPolicyAssignments = $excluded } $deleteExpired = $desired.deleteExpiredExemptions if ($null -ne $deleteExpired) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.deleteExpiredExemptions is deprecated. Remove it!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.deleteExpiredExemptions is deprecated. Remove it!" } $deleteOrphaned = $desired.deleteOrphanedExemptions if ($null -ne $deleteOrphaned) { - Write-Host -ForegroundColor Red "Error in global-settings.jsonc: pacEnvironment $pacSelector field desiredState.deleteOrphanedExemptions is deprecated. Remove it!" - $hasErrors = $true + Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.deleteOrphanedExemptions is deprecated. Remove it!" } } @@ -361,10 +324,8 @@ function Get-GlobalSettings { $null = $pacEnvironmentDefinitions.Add($pacSelector, $pacEnvironmentDefinition) } } - - if ($hasErrors) { - Write-Error "Global settings contains errors." -ErrorAction Stop - } + + Write-ErrorsFromErrorInfo -ErrorInfo $errorInfo -ErrorAction Stop $prompt = $pacEnvironmentSelectors -join ", " Write-Information "PAC Environments: $($prompt)" diff --git a/Scripts/Helpers/Remove-GlobalNotScopes.ps1 b/Scripts/Helpers/Remove-GlobalNotScopes.ps1 index 23e8467f..96fa916d 100644 --- a/Scripts/Helpers/Remove-GlobalNotScopes.ps1 +++ b/Scripts/Helpers/Remove-GlobalNotScopes.ps1 @@ -1,42 +1,34 @@ function Remove-GlobalNotScopes { [CmdletBinding()] param ( - $NotScopes, + $AssignmentNotScopes, $GlobalNotScopes ) - if ($null -ne $NotScopes -and $NotScopes.Count -gt 0) { - if ($null -ne $GlobalNotScopes -and $GlobalNotScopes.Count -gt 0) { - $modifiedNotScopes = [System.Collections.ArrayList]::new() - foreach ($notScope in $NotScopes) { - $resourceGroup = $null - if ($notScope.StartsWith("/subscriptions/") -and $notScope.Contains("/resourceGroups/")) { - $resourceGroupSplits = $notScope -split "/" - $resourceGroup = $resourceGroupSplits[-1] - } - $found = $false - foreach ($globalNotScope in $GlobalNotScopes) { - if ($notScope -eq $globalNotScope) { - $found = $true - break - } - elseif ($globalNotScope.StartsWith("/resourceGroupPatterns/" -and $null -ne $resourceGroup)) { - $globalNotScopePattern = $globalNotScope -replace "/resourceGroupPatterns/" - if ($resourceGroup -like $globalNotScopePattern) { - $found = $true - break - } - } - } - if (-not $found) { - $null = $modifiedNotScopes.Add($notScope) + if ($null -ne $AssignmentNotScopes -or $AssignmentNotScopes.Count -eq 0) { + $null + } + elseif ($GlobalNotScopes.Count -eq 0) { + Write-Output $AssignmentNotScopes -NoEnumerate + } + else { + $assignmentLevelNotScopes = [System.Collections.ArrayList]::new() + foreach ($assignmentNotScope in $AssignmentNotScopes) { + $found = $false + foreach ($globalNotScope in $GlobalNotScopes) { + if ($assignmentNotScope -like $globalNotScope) { + $found = $true + break } } + if (-not $found) { + $null = $assignmentLevelNotScopes.Add($assignmentNotScope) + } + } + if ($assignmentLevelNotScopes.Count -eq 0) { + $null } else { - return $NotScopes + Write-Output $assignmentLevelNotScopes -NoEnumerate } } - else { - return $null - } } diff --git a/Scripts/Helpers/Select-PacEnvironment.ps1 b/Scripts/Helpers/Select-PacEnvironment.ps1 index 2876bfe3..1b7589bf 100644 --- a/Scripts/Helpers/Select-PacEnvironment.ps1 +++ b/Scripts/Helpers/Select-PacEnvironment.ps1 @@ -58,20 +58,35 @@ function Select-PacEnvironment { $OutputFolder = $globalSettings.outputFolder $InputFolder = $globalSettings.inputFolder - $apiVersions = @{ - policyDefinitions = "2023-04-01" - policySetDefinitions = "2023-04-01" - policyAssignments = "2023-04-01" - policyExemptions = "2022-07-01-preview" - roleAssignments = "2022-04-01" - } - if ($pacEnvironment.cloud -eq "AzureChinaCloud") { - $apiVersions = @{ - policyDefinitions = "2021-06-01" - policySetDefinitions = "2021-06-01" - policyAssignments = "2022-06-01" - policyExemptions = "2022-07-01-preview" - roleAssignments = "2022-04-01" + $apiVersions = @{} + + $apiVersions = switch ($pacEnvironment.cloud) { + AzureChinaCloud { + @{ + policyDefinitions = "2021-06-01" + policySetDefinitions = "2021-06-01" + policyAssignments = "2022-06-01" + policyExemptions = "2022-07-01-preview" + roleAssignments = "2022-04-01" + } + } + AzureUSGovernment { + @{ + policyDefinitions = "2021-06-01" + policySetDefinitions = "2021-06-01" + policyAssignments = "2022-06-01" + policyExemptions = "2022-07-01-preview" + roleAssignments = "2022-04-01" + } + } + default { + @{ + policyDefinitions = "2023-04-01" + policySetDefinitions = "2023-04-01" + policyAssignments = "2023-04-01" + policyExemptions = "2022-07-01-preview" + roleAssignments = "2022-04-01" + } } } $planFiles = @{ diff --git a/Scripts/Helpers/Write-ErrorsFromErrorInfo.ps1 b/Scripts/Helpers/Write-ErrorsFromErrorInfo.ps1 index 4c184699..866fa0d1 100644 --- a/Scripts/Helpers/Write-ErrorsFromErrorInfo.ps1 +++ b/Scripts/Helpers/Write-ErrorsFromErrorInfo.ps1 @@ -4,10 +4,10 @@ function Write-ErrorsFromErrorInfo { [hashtable] $ErrorInfo ) if ($ErrorInfo.hasErrors) { - Write-Host -ForegroundColor Red "Errors in file '$($ErrorInfo.fileName)' list of $($ErrorInfo.errorsInFile):'" + Write-Information "File '$($ErrorInfo.fileName)' has $($ErrorInfo.errorsInFile) errors:" foreach ($errorString in $ErrorInfo.errorStrings) { - Write-Host -ForegroundColor DarkYellow " $errorString" + Write-Information " $errorString" -InformationAction Continue } - Write-Host -ForegroundColor Red "End of errors in file '$($ErrorInfo.fileName)' list of $($ErrorInfo.errorsInFile)." + Write-Error "File '$($ErrorInfo.fileName)' with $($ErrorInfo.errorsInFile) errors (end of list)." } } diff --git a/Scripts/Operations/Build-PolicyDocumentation.ps1 b/Scripts/Operations/Build-PolicyDocumentation.ps1 index c6a16c43..b5a9b9e2 100644 --- a/Scripts/Operations/Build-PolicyDocumentation.ps1 +++ b/Scripts/Operations/Build-PolicyDocumentation.ps1 @@ -20,10 +20,6 @@ .PARAMETER IncludeManualPolicies Include Policies with effect Manual. Default: do not include Polcies with effect Manual. -.PARAMETER VirtualCores - Number of virtual cores to use for the operation. Default is 4. - - .EXAMPLE Build-PolicyDocumentation.ps1 -DefinitionsRootFolder "C:\PAC\Definitions" -OutputFolder "C:\PAC\Output" -Interactive Builds documentation from instructions in policyDocumentations folder reading the delployed Policy Resources from the EPAC envioronment. @@ -61,10 +57,15 @@ param ( [Parameter(Mandatory = $false, HelpMessage = "Include Policies with effect Manual. Default: do not include Polcies with effect Manual.")] [switch] $IncludeManualPolicies, - [Parameter(Mandatory = $false, HelpMessage = "Number of virtual cores to use for the operation. Default is 4.")] - [Int16] $VirtualCores = 4 + [Parameter(HelpMessage = "Deprecated.")] + [Int16] $VirtualCores = 0 ) +if ($VirtualCores -gt 0) { + Write-Warning "VirtualCores parameter is deprecated. parallel processing is no longer supported. Please remove the parameter!" -WarningAction Continue + $VirtualCores = 0 +} + # Dot Source Helper Scripts . "$PSScriptRoot/../Helpers/Add-HelperScripts.ps1" @@ -206,7 +207,7 @@ foreach ($file in $files) { -PacEnvironmentSelector $pacEnvironmentSelector ` -PacEnvironment $pacEnvironment ` -CachedPolicyResourceDetails $cachedPolicyResourceDetails ` - -VirtualCores $VirtualCores + -VirtualCores 4 $policySetDetails = $policyResourceDetails.policySets # Calculate itemList @@ -284,7 +285,7 @@ foreach ($file in $files) { -PacEnvironmentSelector $currentPacEnvironmentSelector ` -PacEnvironment $pacEnvironment ` -CachedPolicyResourceDetails $cachedPolicyResourceDetails ` - -VirtualCores $VirtualCores + -VirtualCores 4 # Retrieve assignments and process information or retrieve from cache is assignment previously processed $assignmentArray = $environmentCategoryEntry.representativeAssignments diff --git a/Scripts/Operations/Export-AzPolicyResources.ps1 b/Scripts/Operations/Export-AzPolicyResources.ps1 index fbbfa750..a46c9608 100644 --- a/Scripts/Operations/Export-AzPolicyResources.ps1 +++ b/Scripts/Operations/Export-AzPolicyResources.ps1 @@ -696,11 +696,8 @@ foreach ($pacSelector in $globalSettings.pacEnvironmentSelectors) { $scope = $policyAssignment.resourceIdParts.scope $notScopes = Remove-GlobalNotScopes ` - -NotScopes $policyAssignment.notScopes ` + -AssignmentNotScopes $policyAssignment.notScopes ` -GlobalNotScopes $pacEnvironment.globalNotScopes - if ($notScopes.Count -eq 0) { - $notScopes = $null - } $additionalRoleAssignments = [System.Collections.ArrayList]::new() foreach ($role in $roles) {