Sunday, September 20, 2020

Azure: Determining the Parameters to Change in Azure Resource Manager (ARM) Templates

Overview

In the post "Azure: Azure Resource Manager (ARM) templates for creating Virtual Machines for Standard Window's SKU's" it was shown how to create an Azure Resource Manager (ARM) template that can be used to create a virtual machine. Also shown was how to generate the template's parameter file. There are dozens of parameters so what this post demonstrates is how to determine which parameters to modify.

Reading the parameters file it can be seen that one parameter, adminPassword is assigned to null as it is a password. The adminPassword parameter's value was excluded when the parameter file was created as is shown below:


The name of the VM specified when the ARM template was created was Machine04. The parameters tied to this machine name are:

  • networkInterfaceName
  • networkSecurityGroupName
  • publicIpAddressName
  • virtualMachineName
  • virtualMachineComputerName

A simple way to determine the parameters requiring modification is to:

  1. Create a copy of the parameters file
  2. In the copy of the parameters file change the text value of Machine04 to Machine123
  3. Perform a diff on the original parameters file and the modified copy of the parameters file
The remainder of this post is a list of tools that are useful in determining the parameters that need to be updated when deploying an ARM templet.

Visual Studio Code: File Compare

Quickdiff.net

A handy online way to diff two text files is to use https://quickdiff.net/. I like this site because the diff has options defined as follows:


QuickDiff.net shows the differences as follows:


Jsondiff.com

The site, http://www.jsondiff.com/, allows to Json objects (the parameters files are just Json objects) to be compared. The aforementioned site identifies how many differences there are between the Json objects:

The site, jsondiff.com, also allows navigation between all differences detected:

Saturday, September 19, 2020

Visual Studio Code: Comparing Text Files

Visual Studio Code has built in, albeit unintuitively, file comparison. To diff two files, from Explorer right click on the first file to compare and choose Select for Compare from the context menu:


In the screen snippet above the ParametersW10.json file was clicked on initially. To choose the second file to diff, from Explorer right click on a file and choose Compare with Selected:


In the screen snippet above the 20200919072819442ParametersW10.json file was selected to be compared to ParametersW10.json. 

The difference between the two files is shown as follows:


The upper right corner of the diff provides some handle tools for managing the file compare:


The options are defined as follows:

  • Up arrow: navigate to previous difference
  • Down Arrow: navigate to next difference
  • Backwards P:Show leading/trailing whitespace differences






Sunday, September 6, 2020

Azure/PowerShell: Finding and Removing Orphaned Network Interfaces

When an Azure virtual machine is deleted via the portal (https://portal.azure.com) any network interfaces associated with the VM are not deleted. This can lead to the pool of IP addresses associated with a subnet to be exhausted and no new VMs can be created as there are no IP addresses to assign to the new VMs.

The Get-AzNetworkInterface cmdlet returns all network instances for an Azure subscription and the Remove-AzNetworkInterface cmdlet removes a specific network interface. The following code uses Get-AzNetworkInterface in conjunction with Where-Object to get all orphaned network interfaces. Each network interface is represented by an instance of type PSNetworkInterface

[string] $subID = 'put subscription ID here'

Select-AzureRmSubscription `
 -Subscriptionid $SubID | Out-Null

[Microsoft.Azure.Commands.Network.Models.PSNetworkInterface []] ` 
 $nics =
   Get-AzNetworkInterface |
   Where-Object {
     ($_.VirtualMachine -eq $null) -And
     (($_.PrivateEndpointText -eq $null) -Or
      ($_.PrivateEndpointText -eq 'null'))}

foreach ($nic in $nics)
{
 Write-Host
     "Removing Orphaned NIC $($nic.Name) $($nic.resourcegroupname)"
 Remove-AzNetworkInterface `
   -Name $nic.Name `
   -ResourceGroupName $nic.resourcegroupname `
   -Force }

The Get-AzNetworkInterface | Where-Object code returns only network interfaces:

  • Not associated with virtual machines
  • Not associated with private endpoints 

This script snippet detects if a network interface is not associated with a virtual machine:

($_.VirtualMachine -eq $null)

This script snippet detects if a network interface is not associated with a private endpoint:

     (($_.PrivateEndpointText -eq $null) -Or 
      ($_.PrivateEndpointText -eq 'null'))}

Not being associated with a virtual machine appears to identify a network interface as orphaned having formally been assigned to a now deleted VM. There are network interfaces that were never associated with a virtual machine such as a network interface associated with a private endpoint. This is why there is an additional check to insure the PSNetworkInterface.PrivateEndpointText property is not assigned. Private endpoints are ancillary germane to detecting/removing orphaned network interfaces. More information on private endpoints can be found at What is Azure Private Endpoint?.


Saturday, September 5, 2020

PowerShell: Expanding Object Properties in Strings using Subexpression Operator

Expanding variables inside a double quoted string is useful as shown below but this approach works with variables and not with object properties:

[string] $hostName = 'Executive Officer Kane'

Write-Output "Host name: $hostName"

The output from the above script snippet is as follows:


The script below expands object properties using the subexpression operator in a string that is passed to the Write-Host cmd-let (see the last line of code, Write-Host, in the script below):


The subexpression operation, $(), means that code in the parenthesis is invoked first hence the property is evaluated and then expanded in the string:

"Orphaned $($nic.Name) $($nic.resourcegroupname)"

The subexpression operator is documented in About Operators as follows:


Double quoted strings are useful for variable and object/property expansion but recall the previous post PowerShell: Use Single Quotes Where Possible,



Saturday, August 29, 2020

Azure: Azure Resource Manager (ARM) templates for creating Virtual Machines for Standard Window's SKU's

Creating Virtual Machines (VMs) with the Azure portal is convenient but it can become tedious if numerous VMs are needed and it can be an error prone process. The New-AzVM PowerShell cmdlet can automate the creation of virtual machines. For rudimentary VMs, New-AzVM is straightforward to use from PowerShell. When this cmdlet is used to create virtual machines with complex restrictions that utilize the numerous parameter combinations, coding with New-AzVM can be a daunting task (see "Azure/PowerShell: Cmdlet Parameter and Result Complexity (Get-AzHost, Get-AzVM, New-AzVM)" for an overview of the parameter sets associated with New-AzVM and an example of the elaborate parameters that can be passed to New-AzVM).

One way to simplify the programmatic creation of a virtual machine is to use Azure Resource Manager (ARM) templates. Using PowerShell in conjunction with the New-AzVM cmdlet or ARM templates is an example of Infrastructure as Code (IAC). Instead of using portal and manual object creation, the code  used to create the infrastructure (Azure objects) can be checked into a source code control repository such as Git and treated like a first class Git citizen.

ARM templates are JSON format and describe how Azure entities are to be created. ARM templates can be created using Azure portal where instead of creating an object such as a VM, the configuration specified in Azure portal can be saved as a template. The templates saved do not magically create infrastructure objects. The template and its corresponding parameters are passed as command-line options to the New-AzResourceGroupDeployment cmdlet. Potentially the templates and their parameters are modified such as to create ten virtual machines named VM00 to VM09 which is simply using PowerShell to update the JSON attribute associated with virtual machine's name.

To demonstrate ARM templates, the steps to create virtual machine with the Azure portal will be demonstrated but instead of creating a VM, a template and parameter file will be generated for later use by New-AzResourceGroupDeployment.

The standard Azure portal appears as follows with a familar Virtual machines icon which when clicked on displays the virtual machine blade:


The virtual machine blade (screen) has an option to add a virtual machine which to no surprise is labeled by the word "Add":


Clicking on the Add option displays the "Create a virtual machine" page:


The desired parameters for a virtual machine can be filled in using the "Create a virtual machine page". For this scenario a VM is created the uses the image Windows Server 2012 R2 Datacenter - Gen 1. The image name "Windows Server 2012 R2 Datacenter - Gen 1" is a particular Windows SKU available to newly created Azure VMs.  Once all the parameters have been specified for the VM to be created the "Review + create" button can be clicked on:

After "Review + create" is clicked on, a set of validations are performed for the virtual machine configuration specified. An example validation might be that a virtual machine for a given Resource Group might not be permitted to expose a public IP address. Not permitting a public IP address  is standard-operation-procedure in environments that use VPNs to access VMs via private IP address. Another validation is the enforcement of password complexity for the default administrator account created.

The following screen is displayed after "Review + create" is clicked and the specified configuration has passed validation:



The command actions on the page above include a link (lower right) labeled as "Download a template for automation" (see below):


Clicking "Download a template for automation" link displays the following:


The Template tab allows the template to be selected in the right pane of the screen (see above). Notice above the options to Download, Add to library (preview), and Deploy are provided. Clicking on the Parameters tab displays the parameters associated with the template (see below):


Clicking on the Scripts tab displays the following:


Clicking on "Start" on the PowerShell tile displays the following (Manage Azure resources by using Azure PowerShell):


The New-AzResourceGroupDeployment cmdlet takes as command-line options a template file and a parameters file and deploys object specified by the template.

Friday, August 28, 2020

PowerShell: Two Interview Questions

 During a recent interview I was asked the following questions.

How do you remove duplicates from an array?

My answer was to perform a Select-Object on the array as there must be an command-line option to return only unique values. The Select-Object cmdlet behaves like C#'s LINQ and removing duplicates is something I would do in LINQ.

The answer is Select-Object -Unique:

[int[]] $arrayWithDuplicates = 1, 2, 3, 3, 3, 2, 1, 1, 3, 1, 2, 3

[int[]] $arrayWithUniqueValues = 
    $arrayWithDuplicates | Select-Object -Unique

$arrayWithUniqueValues -join ' '

The output of the above script is:
1 2 3

How do you read the first five lines of a file?

My answer was to perform a Get-Content on the file as there must be an command-line option to return only finite number of lines from the file. 

The answer is Get-Content <filename> -TotalCount 5 where an example is as follows:

Get-Content 'InterviewQuestions.ps1' -TotalCount 5


Tuesday, August 25, 2020

Azure/PowerShell: Cmdlet Parameter and Result Complexity (Get-AzHost, Get-AzVM, New-AzVM)

Overview

The concept of cmdlet parameter sets was discussed in "PowerShell: Cmdlet Parameter Sets". The motivation for introducing this topic was to show the complexity of certain PowerShell cmdlets and in a future posts how to escape this complexity (foreshadow: Azure Resource Manager templates). An example of a further complexity is that a cmdlet can have different return values and can return single values or arrays. Azure cmdlets can take dozens of parameters and contain multiple parameter sets. Making cmdlets more complicated is that certain cmdlets uses parameters that are created by invoking several different cmdlets which extend the values associated with the parameter. For example the New-AzVM can create an Azure VM using a parameter of type PSVirtualMachine. The PSVirtualMachine parameters could have been setup by invoking multiple cmdlets (New-AzVMConfig, Set-AzVMOperatingSystem, Add-AzVMNetworkInterface, Set-AzVMOSDisk). Debugging an invalid parameter passed to a cmdlet such as New-AzVm can be a nightmare.

In order to better understand cmdlet parameter sets and there complexity consider the documentation of several Azure related cmdlets:

  • Get-AzHost: retrieves a host or a list of all hosts if no host name is specified
  • Get-AzVM: retrieves the properties of an Azure VM
  • New-AzVM: creates an Azure VM
The documentation for each of these cmdlets is broken into sections where each shaded rectangle corresponds to a different parameter set. I am color blind so I have no idea if the shaded rectangles contain a color or are simply shades of gray. 

PowerShell cmdlets such as those referenced above return inconsistent types. For example, the Get-AzVM cmdlet can return either of the following types depending on the parameters specified:
PSVirtualMachine
PSVirtualMachineInstanceView

Even when a PowerShell cmdlet is documented to return a specific type or types this may not be the case. Cmdlets can return a single value or an array of values. 

Get-AzHost

The documentation for Get-AzHost shows two parameter sets each identified by shaded rectangles:
The output for each parameter set appears to be consistent as the return value is documented as follows:
Microsoft.Azure.Commands.Compute.Automation.Models.PSHost

If Get-AzHost returns one value the return type is PSHost. If Get-AzHost returns multiple types the return value is of type array.

The first parameter set contains the following unique parameters:
  • ResourceGroupName
  • HostGroupName
The second parameter set contains a unique parameter:
  • ResourceId
The first parameter set contains a -Name parameter which when excludes retrieves a list of all hosts.

    Get-AzVM

    The documentation for the Get-AzVM cmdlet identifies four parameter sets as follows:

    As mentioned above the Get-AzVM will return different types depending on the parameters specified to the cmdlet:
    PSVirtualMachine
    PSVirtualMachineInstanceView

    The return values can be a single value or an array of values.

    The first parameter set identifies all parameters as optional which means no parameter is an option and hence unique. When passed no parameters, Get-AzVM (the first parameter set) returns all virtual machines for a subscription. The second parameter set has positional parameters ResourceGroupName and Name or their named counterparts as required. DisplayHint is also a unique parameter for the second parameter set. The third parameter set has a unique parameter, Location. The fourth parameter set has a lone unique parameter, NextLink.

    New-AzVM

    The New-AzVM cmdlet defines the following multiple parameters sets and takes over a dozen potential parameters including complex types built up by invoking multiple setup cmdlets. 

    The unique parameter for this New-AzVm parameter set is -Credential:


    The unique parameter for this New-AzVM parameter set is -VM:


    The data type passed to the -VM parameter is PSVirtualMachine. Setting up a PSVirtualMachine can involve invoking multiple cmdlets in order to assign additional settings to the PSVirtualMachine instance.

    The unique parameter for this new-AzVM parameter set is -DiskFile:



    New-AzVM and PSVirtualMachine Configuration

    The following example from New-AzVM demonstrates the level of complexity that can be associated with setting up a cmdlet's parameters. The script marked in boldface shows how many PowerShell cmdlets have to be invoked simply (or not so simply) to set up the New-AzVM's -VM parameter value: